Regular HVAC Maintenance Prevents Costly Downtime in Mission Critical Facilities
HVAC contractors share best practices for keeping data centers in peak operating conditions
Mission critical facilities are exactly what they sound like: facilities that are designed to never shut down. No matter what the operation, it must run without interruption, otherwise it suffers major circumstances. And while regular maintenance is important for all HVAC applications, it is perhaps doubly so in mission critical facilities such as data centers. It reduces unplanned downtime, which prevents loss of service, revenue, and goodwill from the data center’s customers.
“Most HVAC is for comfort, but data center HVAC is geared for production,” said Rab Albert, data center technician, HMC Service Co., Louisville, Kentucky. “Data centers rely on consistently performing HVAC systems to maintain the air quality, which is crucial to their servers and specialty equipment running efficiently.”
Data center clients typically fall into a mission critical category, noted Greg Crumpton, vice president of critical environments and facilities, Service Logic, Charlotte, North Carolina, meaning they must have constant air conditioning to remove heat from the computer equipment for their business purposes.
“An example is MRI units — cooling the magnet, health care records, etc. — things that need to be cooled, versus people, is another way to view it,” he said.
Simply put, data center clients differ from regular clients in the critical nature of HVAC and power systems that are built to support the data center functions, said Mike Balles, account manager, Data Center Services, Murphy Co., St. Louis.
“Most, but not all data centers are now key to everyday business functions, and in some instances, are the primary economic engine for the company,” Balles said. “The cost of downtime for a data center client can be measured in the tens of thousands of dollars a minute and up. That level of critical infrastructure demands a quality preventative maintenance program. Those sort of preventative maintenance programs are not as critical for other types of HVAC clients.”
Because of the extreme loss of revenue that could occur if a system goes down, data center facilities need to know and trust that their HVAC service provider will be there to fix the problem — day or night. As such, many contractors providing data center services are on call 24 hours a day, seven days a week for their clients.
“We have maintenance technicians and backup for data centers,” Albert said. “We also have backup for those technicians who are not available. Most customers have built a relationship with the main technician and will usually call him directly first; if they are not able to reach him, they will call the emergency number.”
Inanimate objects don’t care about the time of day, nor the day of the week — this includes holidays as well, Crumpton explained.
“Historically, my firm always had an employee answering the calls 24/7,” he said. “We rotated the duty. Similar to how many firms rotate an ‘on-call technician,’ we rotated the on-call customer service rep as well as a technician. And from Memorial Day through Labor Day, we kept two technicians on call for the busy season. We’ve found that customers would rather hear a familiar voice at 3 a.m. versus a voice answering from an answering service. The customers told us that they felt secure that we heard their problem and they didn’t have to worry about the hand-off from the answering service to the technician. They were more confident in our CSR taking it and running the issue to ground versus an ‘outsider.’ Other firms handle it as they see fit.”
During an emergency outage, Crumpton will simply roll a technician to the customer.
“This will give the client another set of eyes and ears on the ground addressing the HVAC systems, therefore allowing the customer to handle the other issues that he or she will have on his or her plate,” he said. “That is where continuity planning alongside the client can be a huge benefit — meaning that when an outage occurs, there is a standard operating procedure in place.”
Balles agreed, saying that all data centers operate on a 24/7/365 basis, and even though the staff or end users are not present in the middle of the night, there is a high probability that there are automated functions being performed at all times.
“Being able to respond to data center client demands at all hours can help the company in question prevent a potential outage or, at a minimum, keep their processing environment operational,” Balles said. “Data centers are built with high availability and redundancy in the cooling, power, and network infrastructures. The more critical the business, the higher level of redundancy, which means more equipment to maintain.”
Murphy Co. has staff available 24/7/365 in order to dispatch technicians to a site for any maintenance call, Balles noted.
“Most of our data center clients have preventative maintenance contracts in place, where we schedule maintenance activities in advance of dispatching a tech to the site,” he said. “Our technicians are trained and certified on multiple types of data center HVAC and power systems in order to have the highest working knowledge of the equipment they are likely to encounter while on site. The emergency dispatch technicians are the same individuals that are performing the preventative maintenance services, so in most cases, they are already familiar with the site, and the site personnel are familiar with our technicians. That is an important aspect to our services and one we work hard to maintain.”
Having familiarity with site systems and fostering a good relationship with staff and management are extremely important for contractors when dealing with data center clients.
It’s a good way to stay on top of repairs and issues found during preventative maintenance checks, Albert noted.
According to Crumpton, the No. 1 thing is understanding the mission of the client.
“Think about the times they may need extra ‘touches,’” Crumpton said. “Travel folks need to know you are prepared around their busy holiday season, financial folks know that legal holidays are great days to do planned outage work … so just know how to help and offer to do so. Don’t be a normal contractor — get intimate with the facility, the team that runs it, and know your role. Lastly, offer to educate the client or clients you wish you had. Helping folks think through and create a reaction plan can put you in a great spot, meaning they can depend on you and will do so.
“Maintenance, service, and repair for mission critical customers has to be people- and relationship-based,” he continued. “This is a relationship that can be fruitful for many years. Additionally, it can be a relationship that can go south in a hurry. People lose their jobs over poor or not well-thought-out decisions. Be there for your customers and work side by side to build them a plan.”
According to Balles, the best practice for maintaining a data center is to establish a quality preventive maintenance program.
“After spending millions of dollars for a data center, the best insurance for long-term operation is to start a maintenance program before even starting production computing,” he said.
Following manufacturers’ suggested maintenance activities to build an inspection-maintenance program is the key, according to Balles, but documentation is also vital.
“Documentation begins during the build process and continues over the life of the data center,” he said. “Good documentation includes as-built drawings of the facility that are updated after every single change and proper OEM manuals for all equipment. Commissioning reports and all manufacturer startup documents for a baseline of day-one operating parameters can be instrumental with troubleshooting outages in the future.”
Another key aspect is a quality and all-encompassing building monitoring and automation system.
“A BMS system should have every mechanical, electrical, and fire system monitored and tracked,” said Balles. “Historical data of a device’s history can also be key in troubleshooting. And lastly, keeping a complete log of all maintenance activities, emergency outages, and equipment upgrades is vital to maintaining a complete history of the facility’s operation.”
Publication date: 8/13/2018