Data Center Cooling

Cooling Solutions for Server Racks, Computer Rooms, Server Rooms & Data Centers

Data Center Cooling / Server Room Cooling / Computer Room Cooling / Rack Cooling

How much can your organization save by having a more energy efficient Data Center?
As much as 40% of a data center's energy bill is from cooling equipment. Try our Interactive data center efficiency calculator and find out how reducing PUE will result in significant energy and cost savings!

Issue: Understanding the Efficiency Metrics Best Practice: Adoption and use of PUE/DCiE
In furtherance of its mission, The Green Grid is focused on the following: defining meaningful, user-centric models and metrics; developing standards, measurement methods, processes and new technologies to improve data center performance against the defined metrics.
- The Green Grid

Measurements like watts per square foot, kilowatts per rack, and cubic feet per minute (cfm) are ingrained in data center dialogue. Until recently, no standard measurement existed for data center efficiency. Enter the Green Grid, a consortium promoting responsible energy use within critical facilities. The group has successfully introduced two new terms to the data center lexicon: Power Usage Effectiveness (PUE) and Data Center Infrastructure Efficiency (DCiE).

Power Usage Effectiveness (PUE)
PUE is derived by dividing the total incoming power by the IT equipment load. The total incoming power includes, in addition to the IT load, the data center’s electrical and mechanical support systems such as chillers, air conditioners, fans, and power delivery equipment. Lower results are better, as they indicate more incoming power is consumed by IT equipment instead of the intermediary, support equipment.

While it’s not the only consideration, cooling can be a major player in PUE measurement. Consider the following diagram, where the combination of the chiller, humidifier, and CRAC consume 45% of the total energy coming into the facility.

Where does the energy go? (Source: The Green Grid)

The Uptime Institute approximates an industry average PUE of 2.5. Though there are no tiers or rankings associated with the values, PUE allows facilities to benchmark, measure, and improve their efficiency over time. Companies with large-scale data center operations, like Google and Microsoft, have published their PUE. In 2008, Google had an average PUE of 1.21 across their six company data centers. Microsoft’s new Chicago facility, packed with data center containers, calculated an average annual PUE of 1.22.

The widespread adoption of PUE, left in the hands of marketing departments, leaves the door open for manipulation. Though the equation seems simple, there are many variables to consider, and users should always consider the context of these broadcasted measurements. At its core, however, the measurement encourages the benchmarking and improvement at the site level—the actions individual professionals can take to improve the efficiency of their facilities.

Data Center Infrastructure Efficiency (DCiE)
DCiE is simply the inverse of PUE—Total IT Power/Total Facility Power x 100%. DCiE presents a quick snapshot into the amount of energy consumed by the IT equipment. To examine the relationship between PUE and DCiE, “A DCiE value of 33% (equivalent to a PUE of 3.0) suggests that the IT equipment consumes 33% of the power in the data center.”

ASHRAE temperature and humidity recommendations:
The American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) is an international technical society organized and a leading authority providing recommendations for data center cooling and humidity ranges. ASHRAE TC 9.9 recently released their "2008 ASHRAE Environmental Guidelines for Datacom Equipment" which expanded their recommended environmental envelope as follows:

KW/Ton
KW/Ton measures HVAC efficiency by comparing the energy consumption of the component in kW versus the amount of cooling provided in tons (12,000 BTU/hr). Like PUE, the lower the value, the more efficient the device. A kW/Ton rating is possible for all of the major components of a data center cooling system from the compressors to the server exhaust fans.

As data centers implement best practices and state-of-the-art technologies, they can expect the kW/Ton ratings to improve throughout the energy scheme.

Issue: Understanding the Cooling Components Best Practice: Next generation of cooling solutions
Even in many organizations where IT and facilities staffs are cooperative, budgetary and measurement objectives are often separate and sometimes at cross purposes. Facilities typically pays for data center electricity, and IT often does not know how much electricity is being used in the data center or what it costs. In particular, cooling costs are almost always invisible to IT
- Gartner

A data center professional is not necessarily an HVAC engineer. But he must be mechanically savvy and comprehend the entire energy scheme within the cooling infrastructure. Rising energy costs and a volatile economy will make the corporate hierarchy call for reductions in operational costs. And, based on the PUE discussion, we’ve seen that cooling infrastructure can have a major impact on OpEx.

A brief example of the traditional cooling system:

Computer Room Air Conditioners (CRAC)

• Refrigerant-based (DX), installed within the data center floor and connected to outside condensing units.
• Moves air throughout the data center via fan system- delivers cool air to the servers, returns exhaust air from the room

Computer Room Air Handler (CRAH)

• Chilled water based, installed on data center floor and connected to outside chiller plant.
• Moves air throughout the data center via fan system: delivers cool air to the servers, returns exhaust air from the room.

Humidifier

• Usually installed within CRAC/CRAH and replace water loss before the air exits the A/C units. Also available in standalone units.
• Ensures that humidity levels fall within ASHRAE’s recommended range

Chiller

• Produces chilled water via refrigeration process.
• Delivers chilled water via pumps to CRAH.

The principles of data center cooling—air delivery, movement, and heat rejection—are not complex, but these systems are. There are a number of smaller components like compressors, fans, and pumps, which shape the system’s operation and effectiveness.

Even within the “traditional” generation, nothing is one size fits all; cooling solutions are often dependent on factors like room layout, installation densities, and geographic location. 42U’s cooling technologies provide the data center manager with thorough product overviews, data on ROI, installation and commissioning, and objective product recommendation based strictly on user application, environment, and goals.


Issue: Understanding Airflow Management Best Practice: Measurement, CFD Analysis, Containment
“In most cases, a fully developed air management strategy can produce significant and measurable economic benefits and, therefore, should be the starting point when implementing a data center energy savings program”
- The Green Grid

The cooling components are charged with creating and moving air on the data center floor. From there, the room itself must maintain separate climates—the cool air required by the servers and the hot air they exhaust. Without boundaries, the air paths mix, resulting in both economic and ecological consequences.

In the early 2000s, Robert Sullivan, an Uptime Institute scientist, advanced the concept of hot aisle/cold aisle, attempting to achieve air separation within the server room. The design, which aligns data center cabinets into alternating rows, endures in critical facilities throughout the world and is widely regarded as the first step in improving airflow management.

	Hot Aisle / Cold Aisle Server Rack Configuration

The arrangement, however, lacks precise air delivery and removal, leaving users a new set of challenges. (link to relevant articles)

• Bypass air—Conditioned air that does not reach computer equipment, escaping through cable cut-outs, holes under cabinets, misplaced perforated tiles or holes in the computer room perimeter walls—limits the precise delivery of cold air at the server intake.
• Hot air recirculation, where waste heat enters the cold aisle, ensures that the cooling infrastructure must throw colder air at the equipment to offset this mixing.
• Hot air contamination prohibits the air handlers from receiving the warmest possible exhaust air, rendering their operation less efficient.
• Hot spots may persist as a result of all of the above

Users can address these inefficiencies through measurement, modeling, and analysis. These tools-including Computational Fluid Dynamics (CFD) analysis, air velocity and pressure gauging, provide a snapshot of your environment, pinpointing problem areas. Furthermore, real time measurement provides an immediate analysis of your data center environment. Often significant improvements are achieved with quick, inexpensive remedies, like blanking panels and brush strips.

Cold Aisle Containment
Cold Aisle Containment attempts to maximize the hot aisle/cold aisle arrangement by encasing the cold aisle with barriers made of metal, plastic or fiberglass. This approach eliminates the above challenges, ensuring the cold air stays at the server intake, while the air handlers receive the warmer exhaust error, improving their efficiency.

Hot Aisle Containment
In Hot Aisle Containment, the hot aisle is now enclosed, using the same barriers as its cold aisle counterpart. The design captures exhaust air via In-Row air conditioners, conditions it, and returns it to the cold aisle. AC efficiency is further improved as neither the hot exhaust air nor cold inlet air has far to travel.

Issue: Designing, Implementing, Managing High Density Best Practice: Next Generation Cooling Technology
The trend towards higher density cabinets and racks will continue unabated through 2012, increasing both the density of compute resources on the data center floor, and the density of both power and cooling required to support them.
- Gartner

The industry is exploring progressive cooling solutions because the current generation, discussed earlier, has proven insufficient and inflexible with increased computing requirements. (Chillers, for instance, are estimated to consume 33% of a facility’s total power in current layouts). In its report, the EPA christened some of the latest options as either “best practice” or “state-of the art” in its analysis and approximated gains of 70-80% in infrastructure efficiency through their use. 

In a recent survey conducted by the Uptime Institute, enterprise data center managers responded that 39% of them expected that their data centers would run out of cooling capacity in the next 12-24 months and 21% claimed they would run out of cooling capacity in 12-60 months.  The power required to cool IT equipment in your data center far exceeds the power required to run that equipment.  Overall power in the data center is fast reaching capacity as well and an obvious area that needs to be addressed is implementing cooling best practices where ever possible and utilization of in-row cooling to address hot spots. In the same Uptime survey 42% of these data center managers expected to run out of power capacity within 12-24 months and another 23% claimed that they would run out of power capacity in 24-60 months.  Greater attention to energy efficiency and consumption is critical.

To optimize the cooling in your data center a good first step is an in-depth analysis of your current environment to gain a holistic understanding of your data center's environment, increased Awareness of your critical risk factors, benchmark of performance metrics, and generate a punch list of opportunities for cooling improvement.  For help in this area we recommend our Data CenterCooling Efficiency Services. Heat is just one of the many factors that affects IT performance. Beat the heat with the scalable climate control solutions. From passive air to active liquid, the flexible cooling concepts are designed according to your requirements.  Below are details on some of the latest energy efficient data center cooling solutions available.

Computer Room Cooling Services: Specialized engineering and educational services that enhance data center performance without additional capital investments. These services will help you to boost data center reliability, optimize your current cooling infrastructure, enable precision cooling to eliminate hot spots, dramatically reduce bypass airflow, assist with data center ASHRAE compliance and understand the dynamics of your data center, including cooling requirements and deficiencies
Rittal Liquid Cooling Package Developed to remove high levels of waste heat from server enclosures, this high density cooling solution utilizes LCP air/water heat exchanger providing uniform, effective and affordable cooling for servers and similar IT equipment. The special horizontal airflow of the Rittal LCP represents an adaptation of this widespread cooling principle, providing cooled air uniformly throughout the complete height of the enclosure. The Liquid Cooling unit is a modular, upgradeable, and temperature-neutral cooling concept. Up to 30kW cooling output, with three cooling modules possible per equipment rack Controlled variable speed fan and water flow based on actual heat load generated in cabinet Constant temperature cold air provided at the front intake for optimized equipment use, hot air removed from rear Even air distribution along the entire height of the front 482.6 mm (19") mounting angles Can be bayed between two 42U racks High energy efficiency in removing waste heat with no temperature impact in the room	Rittal TS8 Racks with Rittal Liquid Cooling Package
Blade Cooling Circuit: Distributor Kit Rittals liquid cooling for server racks serves as a basis for this solution. The large number of drip-free taps to the individual computers is now replaced by the connection of horizontal liquid sub-distributors, where the drip-free connectors for connection of the Power blade are located.	Rack Cooling Module
Liebert Water-Based Cooling System Liebert XD Water Based Cooling Module and Racks The Liebert server rack cooling modules work as a system with Liebert XDP-W Coolant Pumping Unit to supply cooling water to high heat density racks. The Liebert XDK and Liebert XDR modules offer an alternative to critical applications where the hot aisle/cold aisle approach to cooling is not practical.
Liebert XD Water Pumping Unit The water-based Liebert XDP-W pumping unit houses the isolating heat exchanger between the Liebert XDR-W or Liebert XDK-W and the building chilled water. It circulates the chilled water to the cooling modules at a temperature always above the actual dew point to prevent condensation.	Liebert XD
Rack Cooling Distributor Accessories
Enclosure Blower Assist airflow in your cabinets with the the 42U Rack Enclosure Blower and be confident that your equipment will run cooler. The Enclosure Blower features a durable, lightweight polymer housing and fits in the bottom of server cabinets where it draws cool air from beneath the floor and boosts delivery of the coldest available air directly to server fronts. This unique blower by 42U Rack is the suggested solution for solid or partially vented front doors with rails set back a minimum of four inches to establish the column of air to raise to the top of the cabinet.
Dual Power Transfer Switch The dual power transfer switch is designed to provide continuous 115 V AC power for the enclosure blower and other single corded fans and motor-driven devices. Capable of automatically switching a 5A load during a power failure on one input circuit, the dual power transfer switch can power other single corded fans and motor-driven devices commonly deployed in data centers.
Rack Cooling Fan Systems
Split Rear Door Fan With today's high heat loads inside server enclosures, it is critical to use  thermal management techniques with optimized enclosure ventilation. With vented front and rear doors and a vented roof, 42U Rack's server cabinets provide such ventilation. 42U Rack's split rear door fan takes advantage of the front to rear airflow of most servers and allows the user to position the fan assembly in the most critical location within the cabinet. Two high-pressure fans, 105 cfm each, feature a shallow depth chassis design and provide the ability to exhaust hot air out the rear of the enclosure at a dramatically higher rate than traditional roof mount fan trays. The result is cooler operating equipment, even with very high enclosure heat loads.
Fan Expansion Kit Designed for retrofitting various 42U Rack fan units or to supplement the fan mounting plate and fan roof, modular. Air Baffle System The 42U Rack Air Baffle System has an air inlet nozzle in the base frame. Cold air is drawn in from below and is routed into the double-walled door. The cold air is then distributed inside the rack with special covers. DC Fan Mounting Plate Exceptionally low-noise thanks to FCS speed control, fully fitted. Suitable for 42U Rack's enclosures with a raised roof (> 20 mm) or 42U Rack's roof plate, vented.  It is installed from above. A cable entry is prepared in the rear section of the plate. Server Fan Assembly The 42U Rack server fan assembly takes advantage of the front-to-rear airflow of most servers and allows the user to position the fan assembly in the most critical location within the cabinet. The result is cooler operating equipment, even with very high enclosure heat loads. Room Air Conditioners Liebert Air Conditioning Solutions - Liebert precision cooling products provide the precise, year-round cooling required by sensitive electronics while protecting them from the environmental hazards of dust, temperature and humidity. Liebert solutions include standalone, ceiling and wall-mount systems for spot, room and telecom shelter applications, and liquid chilling systems for medical imaging and industrial equipment.
Mini Air Conditioners Especially developed for small enclosures but can also be used in large enclosures with low heat dissipating requirements. Rittal air conditioners are available in compact roof mount and powerful wall mount versions. Medium Range Air Conditioners These medium capacity 42U Rack air conditioners are used worldwide and stand out through high dependability, flexible installation options, optimum ease of operation and high safety standards.
Full Sized Air Conditioners Multi-functional wall mount air conditioners fully meet these requirements through their user friendliness, microcontroller technology, and ease of installation.
Mini Chillers Compact size does not mean small performance. These mini chillers are ideal when installation is required in the tightest areas.
Freestanding Chillers With a wide cooling range to satisfy demands for a high cooling output with optimized space requirements, these Freestanding Chillers feature removable panels to ensure easier access for servicing.
Raised Floor Thermal Management: A cost-effective airflow and thermal management way to improve airflow in your data center or server room is to make sure that all cable openings in your raised floor tiles are sealed.  Based on measurements at multiple data centers, 50 - 80% of valuable conditioned air is not reaching the air intake of IT equipment due to unsealed cable openings. This lost air, known as bypass airflow, contributes to equipment hot spots, cooling unit inefficiencies, and increasing infrastructure costs. Sealing cable openings is an essential component of IT equipment reliability and availability. Raised Floor Grommets are a cost-effective remedy.
Freestanding Chillers With an output range from 32 kW to 172 kW (131,245 to 587,411 BTU) to satisfy the demands of high cooling load applications, freestanding chillers are up to the largest task. Servicing is also easy with removable panels for quick access. Chiller Technical design: - Robust industrial standard in 3 enclosure sizes - Identical basic enclosure for oil and water chiller systems - Extra space for the integration of special equipment - Variable air routing is possible via the left or right sidewall - Floating contact for collective fault signal - Level monitor - Multi-coil vaporizer in the tank Chiller Configuration: Chiller system wired and plumbed ready for connection, with multilingual documentation including functional diagram and wiring plans. Cooling Accessories Interface Board The interface board is an extension for 42U Rack cooling units with a comfort controller. In this way it is possible to monitor a master/slave combination of up to 10 cooling units. Optimize Airflow in your Rack Enclosures HotLok - Blanking Panels Optimize air flow within your server racks by using effective sealing quick finger install blanking panels while monitoring temperature within your racks with optional temperature strips. Digital Enclosure Internal Temperature Display and Thermostat For installing on the enclosure door or wall and in a cooling unit or heat exchanger. Featured White Paper Close Coupled Cooling- Liquid Cooling Analysis   Download >>>   Speed Control Temperature dependent speed control for Rack fan and filter units and air/air heat exchangers with a rated operating voltage of 230 V AC for noise reduction and to save energy in part-load operation. Conclusion “Greening” the data center often starts with the cooling infrastructure. This page addresses the basics, hoping to facilitate deeper discussions on current vs. future cooling technologies. From chillers to CRACs to economizers, there is no shortage of vendors who, armed with case studies, tables, and whitepapers, are championing the efficiency of their products. Our goal is to help the data center professional wade through mountains of vendor data to find the most germane, economical, and efficient products for his application. Bibliography Cappuccio, D. (2008). Creating Energy-Efficient, Low-Cost, High Performance Data Centers. Gartner Data Center Conference, (p. 4). Las Vegas. EPA. (2007, August 2). EPA Report to Congress on Server and Data Center Energy Efficiency. Retrieved January 5, 2009, from Energy Star: http://www.energystar.gov/ia/partners/prod_development/downloads/EPA_Report_Exec_Summary_Final.pdf McGuckin, P. (2008). Taming the Data Center Energy Beast. Gartner Data Center Conference, (p. 5). Las Vegas. Sullivan, R. (2002). Alternating Cold and Hot Aisles Provides More Reliable Cooling for Server Farms. Retrieved December 15, 2008, from Open Xtra: http://www.openxtra.co.uk/articles/AlternColdnew.pdf The Green Grid. (2008, October 21). Seven Strategies To Improve Data Center Cooling Efficiency. Retrieved December 18, 2008, from The Green Grid: http://www.thegreengrid.org/gg_content/White_Paper_11_-_Seven_Strategies_to_Cooling_21_October_2008.pdf The Green Grid. (2009). The Green Grid: Home. Retrieved January 5, 2009, from http://www.thegreengrid.org/home About 42U focuses on providing Facilities and Data Center Managers with objective and trusted independent information on Data Center Power and Data Center Cooling best practices along with KVM over IP and Remote Power lights out management to help them reduce their organization's power consumption.  Our Data Center Management website has helped millions of IT professionals globally who search for best practices and technologies to help them make their computer environment more efficient. This work is licensed under a Creative Commons Attribution-No Derivative Works 3.0 Unported License.