| |
Cooling Solutions for Server Racks, Computer Rooms, Server
Rooms & Data Centers
Data Center Cooling / Server Room Cooling
/ Computer Room Cooling / Rack Cooling
|
The cooling infrastructure is a significant
part of a data center. The complex connection of chillers,
compressors and air handlers create the optimal computing
environment, ensuring the longevity of the servers installed
within and the vitality of the organization they support.
Yet, this current ecosystem has come
at a price. The EPA’s oft-cited 2007 report predicted
that data center energy consumption, if left unchecked,
would reach 100 million kWh by 2011 with a corresponding
energy bill of $7.4 billion. This conclusion, however,
isn’t strictly based on Moore’s Law or the need for
greater bandwidth. Their estimate envisions tomorrow’s
processing power will be addressed with yesterday’s
cooling strategies. The shortcomings of these designs,
coupled with demand for more processing power, would
require (10) new power plants to provide the juice for
it all, according to the report.
In light of this news, many industry
insiders are turning a critical eye toward cooling,
recognizing both the inefficiencies of current approaches
and the improvements possible through new technologies.
The information contained herein is designed to assist
the data center professional who, while keeping uptime
and redundancy inviolate, must also balance growing
demand for computing power with pressure to reduce energy
consumption.
|

|
How much can your organization save by
having a more energy efficient Data Center?
As much as 40% of a data center's energy bill is from
cooling equipment. Try our Interactive
data center efficiency
calculator and find out how reducing PUE will result in
significant energy and cost savings!

Issue: Understanding the Efficiency
Metrics Best Practice: Adoption and use of PUE/DCiE
In furtherance of its mission, The Green Grid is focused
on the following: defining meaningful, user-centric models and
metrics; developing standards, measurement methods, processes
and new technologies to improve data center performance against
the defined metrics.
- The Green Grid
Measurements like watts per square foot, kilowatts
per rack, and cubic feet per minute (cfm) are ingrained in data
center dialogue. Until recently, no standard measurement existed
for data center efficiency. Enter the Green Grid, a consortium
promoting responsible energy use within critical facilities.
The group has successfully introduced two new terms to the data
center lexicon: Power Usage Effectiveness (PUE) and Data Center
Infrastructure Efficiency (DCiE).
Power Usage Effectiveness (PUE)
PUE is derived by dividing the total incoming power by the
IT equipment load. The total incoming power includes, in addition
to the IT load, the data center’s electrical and mechanical
support systems such as chillers, air conditioners, fans, and
power delivery equipment. Lower results are better, as they
indicate more incoming power is consumed by IT equipment instead
of the intermediary, support equipment.
While it’s not the only consideration, cooling
can be a major player in
PUE measurement.
Consider the following diagram, where the combination of the
chiller, humidifier, and CRAC consume 45% of the total energy
coming into the facility.

Where does the energy go? (Source: The Green Grid)
The Uptime Institute approximates an industry
average PUE of 2.5. Though there are no tiers or rankings associated
with the values, PUE allows facilities to benchmark, measure,
and improve their efficiency over time. Companies with large-scale
data center operations, like Google and Microsoft, have published
their PUE. In 2008, Google had an average PUE of 1.21 across
their six company data centers. Microsoft’s new Chicago facility,
packed with data center containers, calculated an average annual
PUE of 1.22.
The widespread adoption of PUE, left in the
hands of marketing departments, leaves the door open for manipulation.
Though the equation seems simple, there are many variables to
consider, and users should always consider the context of these
broadcasted measurements. At its core, however, the measurement
encourages the benchmarking and improvement at the site level—the
actions individual professionals can take to improve the efficiency
of their facilities.
Data Center Infrastructure Efficiency (DCiE)
DCiE is simply the inverse of PUE—Total IT Power/Total Facility
Power x 100%. DCiE presents a quick snapshot into the amount
of energy consumed by the IT equipment. To examine the relationship
between PUE and DCiE, “A DCiE value of 33% (equivalent to a
PUE of 3.0) suggests that the IT equipment consumes 33% of the
power in the data center.”
ASHRAE temperature and humidity recommendations:
The American Society of Heating, Refrigerating and Air-Conditioning
Engineers (ASHRAE) is an international technical society organized
and a leading authority providing recommendations for data center
cooling and humidity ranges. ASHRAE TC 9.9 recently released
their "2008 ASHRAE Environmental Guidelines for Datacom Equipment"
which expanded their recommended environmental envelope as follows:
| |
| |
2004 Version |
2008 Version |
| Temperature |
20°C (68°F) to 25°C (77°F) |
18°C (64.4°F) to 27°C (80.6°F) |
| Humidity |
40% RH to 55% RH |
5.5°C DP (41.9°F) to 60% RH
& 15°C DP (59°F DP) |
|
|
| |
** Conditions reflect air entering the
Server & IT equipment ** |
|
KW/Ton
KW/Ton measures HVAC efficiency by comparing the energy consumption
of the component in kW versus the amount of cooling provided
in tons (12,000 BTU/hr). Like PUE, the lower the value, the
more efficient the device. A kW/Ton rating is possible for all
of the major components of a data center cooling system from
the compressors to the server exhaust fans.
As data centers implement best practices and state-of-the-art
technologies, they can expect the kW/Ton ratings to improve
throughout the energy scheme.
Issue: Understanding the Cooling Components
Best Practice: Next generation of cooling solutions
Even in many organizations where IT and facilities staffs
are cooperative, budgetary and measurement objectives are often
separate and sometimes at cross purposes. Facilities typically
pays for data center electricity, and IT often does not know
how much electricity is being used in the data center or what
it costs. In particular, cooling costs are almost always invisible
to IT
- Gartner
A data center professional is not necessarily an HVAC engineer.
But he must be mechanically savvy and comprehend the entire
energy scheme within the cooling infrastructure. Rising energy
costs and a volatile economy will make the corporate hierarchy
call for reductions in operational costs. And, based on the
PUE discussion, we’ve seen that cooling infrastructure can have
a major impact on OpEx.
A brief example of the traditional cooling system:
Computer Room Air Conditioners (CRAC)
- Refrigerant-based (DX), installed within the data center
floor and connected to outside condensing units.
- Moves air throughout the data center via fan system-
delivers cool air to the servers, returns exhaust air from
the room
Computer Room Air Handler (CRAH)
- Chilled water based, installed on data center floor
and connected to outside chiller plant.
- Moves air throughout the data center via fan system:
delivers cool air to the servers, returns exhaust air from
the room.
Humidifier
- Usually installed within CRAC/CRAH and replace water
loss before the air exits the A/C units. Also available
in standalone units.
- Ensures that humidity levels fall within ASHRAE’s recommended
range
Chiller
- Produces chilled water via refrigeration process.
- Delivers chilled water via pumps to CRAH.
The principles of data center cooling—air delivery,
movement, and heat rejection—are not complex, but these systems
are. There are a number of smaller components like compressors,
fans, and pumps, which shape the system’s operation and effectiveness.
Even within the “traditional” generation, nothing
is one size fits all; cooling solutions are often dependent
on factors like room layout, installation densities, and geographic
location. 42U’s cooling technologies provide the data
center manager with thorough product overviews, data on ROI,
installation and commissioning, and objective product recommendation
based strictly on user application, environment, and goals.
Issue: Understanding Airflow Management Best Practice:
Measurement, CFD Analysis, Containment
“In most cases, a fully developed air management strategy
can produce significant and measurable economic benefits and,
therefore, should be the starting point when implementing a
data center energy savings program”
- The Green Grid
The cooling components are charged with creating and moving
air on the data center floor. From there, the room itself must
maintain separate climates—the cool air required by the servers
and the hot air they exhaust. Without boundaries, the air paths
mix, resulting in both economic and ecological consequences.
In the early 2000s, Robert Sullivan, an Uptime Institute scientist,
advanced the concept of hot aisle/cold aisle, attempting
to achieve air separation within the server room. The design,
which aligns data center cabinets into alternating rows, endures
in critical facilities throughout the world and is widely regarded
as the first step in improving airflow management.
| |
 |
|
| |
Hot Aisle / Cold Aisle Server Rack Configuration
|
|
The arrangement, however, lacks precise air delivery and
removal, leaving users a new set of challenges. (link to relevant
articles)
- Bypass air—Conditioned air that does not reach
computer equipment, escaping through cable cut-outs, holes
under cabinets, misplaced perforated tiles or holes in the
computer room perimeter walls—limits the precise delivery
of cold air at the server intake.
- Hot air recirculation, where waste heat enters
the cold aisle, ensures that the cooling infrastructure
must throw colder air at the equipment to offset this mixing.
- Hot air contamination prohibits the air handlers
from receiving the warmest possible exhaust air, rendering
their operation less efficient.
- Hot spots may persist as a result of all of the
above
Users can address these inefficiencies through measurement,
modeling, and analysis. These tools-including Computational
Fluid Dynamics (CFD) analysis, air velocity and pressure gauging,
provide a snapshot of your environment, pinpointing problem
areas. Furthermore, real time measurement provides an
immediate analysis of your data center environment. Often significant
improvements are achieved with quick, inexpensive remedies,
like blanking panels and brush strips.
Cold Aisle Containment
Cold Aisle Containment attempts to maximize the hot aisle/cold
aisle arrangement by encasing the cold aisle with barriers made
of metal, plastic or fiberglass. This approach eliminates the
above challenges, ensuring the cold air stays at the server
intake, while the air handlers receive the warmer exhaust error,
improving their efficiency.
Hot Aisle Containment
In Hot Aisle Containment, the hot aisle is now enclosed,
using the same barriers as its cold aisle counterpart. The design
captures exhaust air via In-Row air conditioners, conditions
it, and returns it to the cold aisle. AC efficiency is further
improved as neither the hot exhaust air nor cold inlet air has
far to travel.
Issue: Designing, Implementing, Managing High Density
Best Practice: Next Generation Cooling Technology
The trend towards higher density cabinets and racks will
continue unabated through 2012, increasing both the density
of compute resources on the data center floor, and the density
of both power and cooling required to support them.
- Gartner
The industry is exploring progressive cooling solutions because
the current generation, discussed earlier, has proven insufficient
and inflexible with increased computing requirements. (Chillers,
for instance, are estimated to consume 33% of a facility’s total
power in current layouts). In its report, the EPA christened
some of the latest options as either “best practice” or “state-of
the art” in its analysis and approximated gains of 70-80% in
infrastructure efficiency through their use.
|
BEST PRACTICE |
STATE OF THE ART |
|
Free Cooling |
Direct Liquid Cooling |
|
Air Side Economizers |
Close-Coupled Cooling |
|
Water Side Economizers |
|
|
- Evaporative Cooling |
|
|
- Dry Cooling |
|
Free Cooling
Free cooling brings Mother Nature into the data center.
When the ambient temperature and humidity are favorable,
an economizer system circumvents some of the cooling infrastructure
and uses the outside air as a cooling mechanism. The economizers
come in two forms.
An air side economizer uses the outside
climate to cool the data center. This outside air is distributed
to the cabinets via the existing air delivery system, except
no mechanical activity is needed for heat rejection.
A water side economizer uses the
outside air in conjunction with a chiller system. Instead
of compressors, the outside air cools the water, which is
then pumped to data center CRAHs. Water side economizers
are marketed as either evaporative coolers or dry coolers.
Economizer use and ROI depend heavily on
climate, meaning the data center manager must thoroughly
review readings like wet bulb temperature, dry bulb temperature,
and relative humidity for his location. For those with optimal
environments (low night-time or seasonal temperatures) there
will be compelling arguments for their use, especially their
impact at the chiller level.
Close-Coupled Liquid Cooling
Close-coupled liquid cooling expands further on air
management and containment. The use of water and the proximity
of the heat transfer increases efficiencies and enables
some close-coupled cooling designs to operate with elevated
chilled water temperatures. Higher inlet water temperatures
can reduce the energy needed for mechanical cooling and
also maximize the amount of free cooling hours per year.
Elevated Chilled Water Temperatures
For facilities with a chiller infrastructure, the EPA
recommends raising the water temperatures. Different sources
estimate a traditional supply temperature between 42-45
deg F. A higher supply temperature yields higher efficiency
for the chiller as it reduces energy consumption. Higher
set points can also segue into water side economizers.
In a recent survey conducted by the Uptime Institute,
enterprise data center managers responded that 39% of them expected
that their data centers would run out of cooling capacity in
the next 12-24 months and 21% claimed they would run out of
cooling capacity in 12-60 months. The power required to
cool IT equipment in your data center far exceeds the power
required to run that equipment. Overall power in the data
center is fast reaching capacity as well and an obvious area
that needs to be addressed is implementing cooling best practices
where ever possible and utilization of in-row cooling to address
hot spots. In the same Uptime survey 42% of these data center
managers expected to run out of power capacity within 12-24
months and another 23% claimed that they would run out of power
capacity in 24-60 months. Greater attention to energy
efficiency and consumption is critical.
To optimize the cooling in your data center a good first
step is an in-depth analysis of your current environment to
gain a holistic understanding of your data center's environment,
increased Awareness of your critical risk factors, benchmark
of performance metrics, and generate a punch list of opportunities
for cooling improvement. For help in this area we recommend
our
Data CenterCooling Efficiency Services. Heat is just one
of the many factors that affects IT performance. Beat the heat
with the scalable climate control solutions. From passive air
to active liquid, the flexible cooling concepts are designed
according to your requirements. Below are details on some
of the latest energy efficient data center cooling solutions
available.
Computer Room Cooling Services:
Specialized engineering and educational services
that enhance data center performance without additional
capital investments. These services will help you to
boost data center reliability, optimize your current
cooling infrastructure, enable precision cooling to
eliminate
hot spots, dramatically reduce bypass airflow, assist
with data center ASHRAE compliance and understand the
dynamics of your data center, including cooling requirements
and deficiencies |
| |
|
Rittal Liquid
Cooling Package
Developed to remove high levels of waste heat from
server enclosures, this high density cooling solution
utilizes LCP air/water heat exchanger providing uniform,
effective and affordable cooling for servers and similar
IT equipment. The special horizontal airflow of the
Rittal LCP
represents an adaptation of this widespread cooling
principle, providing cooled air uniformly throughout
the complete height of the enclosure.The Liquid Cooling
unit is a modular, upgradeable, and temperature-neutral
cooling concept.
- Up to 30kW cooling output, with three cooling
modules possible per equipment rack
- Controlled variable speed fan and water flow
based on actual heat load generated in cabinet
- Constant temperature cold air provided at the
front intake for optimized equipment use, hot air
removed from rear
- Even air distribution along the entire height
of the front 482.6 mm (19") mounting angles
- Can be bayed between two 42U racks
- High energy efficiency in removing waste heat
with no temperature impact in the room
|

Rittal TS8 Racks with
Rittal Liquid
Cooling Package
|
| |
|
| Blade Cooling Circuit: Distributor
Kit
Rittals liquid cooling for server racks serves as
a basis for this solution. The large number of drip-free
taps to the individual computers is now replaced by
the connection of horizontal liquid sub-distributors,
where the drip-free connectors for connection of the
Power blade are located. |

Rack Cooling Module
|
| |
|
| Liebert Water-Based Cooling System
Liebert XD Water Based Cooling Module and Racks
The Liebert server rack cooling modules work as
a system with Liebert XDP-W Coolant Pumping Unit to
supply cooling water to high heat density racks. The
Liebert
XDK and
Liebert
XDR modules offer an alternative to critical applications
where the hot aisle/cold aisle approach to cooling is
not practical.
|

|
| |
|
Liebert XD Water Pumping Unit
The water-based
Liebert
XDP-W pumping unit houses the isolating heat exchanger
between the Liebert XDR-W or Liebert XDK-W and the building
chilled water. It circulates the chilled water to the
cooling modules at a temperature always above the actual
dew point to prevent condensation. |

Liebert XD
|
| Rack
Cooling Distributor Accessories |
|
Enclosure Blower
Assist airflow in your cabinets with the the 42U
Rack Enclosure Blower and be confident that your equipment
will run cooler. The Enclosure Blower features a durable,
lightweight polymer housing and fits in the bottom of
server cabinets where it draws cool air from beneath
the floor and boosts delivery of the coldest available
air directly to server fronts. This unique blower by
42U Rack is the suggested solution for solid or partially
vented front doors with rails set back a minimum of
four inches to establish the column of air to raise
to the top of the cabinet. |
 |
| |
|
Dual Power Transfer
Switch
The dual power transfer switch is designed to provide
continuous 115 V AC power for the enclosure blower and
other single corded fans and motor-driven devices. Capable
of automatically switching a 5A load during a power
failure on one input circuit, the dual power transfer
switch can power other single corded fans and motor-driven
devices commonly deployed in data centers. |
| |
|
|
Rack Cooling Fan
Systems |
|
Split Rear Door Fan
With today's high heat loads inside server enclosures,
it is critical to use thermal management techniques
with optimized enclosure ventilation. With vented front
and rear doors and a vented roof, 42U Rack's server
cabinets provide such ventilation. 42U Rack's split
rear door fan takes advantage of the front to rear airflow
of most servers and allows the user to position the
fan assembly in the most critical location within the
cabinet. Two high-pressure fans, 105 cfm each, feature
a shallow depth chassis design and provide the ability
to exhaust hot air out the rear of the enclosure at
a dramatically higher rate than traditional roof mount
fan trays. The result is cooler operating equipment,
even with very high enclosure heat loads. |

|
| |
|
Fan Expansion Kit
Designed for retrofitting various 42U Rack fan units
or to supplement the fan mounting plate and fan roof,
modular.
Air Baffle System
The 42U Rack Air Baffle System has an air inlet
nozzle in the base frame. Cold air is drawn in from
below and is routed into the double-walled door. The
cold air is then distributed inside the rack with special
covers.
DC Fan Mounting Plate
Exceptionally low-noise thanks to FCS speed control,
fully fitted. Suitable for 42U Rack's enclosures with
a raised roof (> 20 mm) or 42U Rack's roof plate, vented.
It is installed from above. A cable entry is prepared
in the rear section of the plate.Server Fan Assembly
The 42U Rack server fan assembly takes advantage
of the front-to-rear airflow of most servers and allows
the user to position the fan assembly in the most critical
location within the cabinet. The result is cooler operating
equipment, even with very high enclosure heat loads.
Room Air Conditioners
Liebert Air Conditioning Solutions - Liebert
precision cooling products provide the precise, year-round
cooling required by sensitive electronics while protecting
them from the environmental hazards of dust, temperature
and humidity. Liebert solutions include standalone,
ceiling and wall-mount systems for spot, room and telecom
shelter applications, and liquid chilling systems for
medical imaging and industrial equipment.
|
| |
|
Mini Air Conditioners
Especially developed for small enclosures but can
also be used in large enclosures with low heat dissipating
requirements. Rittal air conditioners are available
in compact roof mount and powerful wall mount versions.
Medium Range Air Conditioners
These medium capacity 42U Rack air conditioners
are used worldwide and stand out through high dependability,
flexible installation options, optimum ease of operation
and high safety standards.
|

|
| |
|
Full Sized Air Conditioners
Multi-functional wall mount air conditioners fully
meet these requirements through their user friendliness,
microcontroller technology, and ease of installation. |
| |
|
Mini Chillers
Compact size does not mean small performance. These
mini chillers are ideal when installation is required
in the tightest areas. |
 |
| |
|
Freestanding Chillers
With a wide cooling range to satisfy demands for
a high cooling output with optimized space requirements,
these Freestanding Chillers feature removable panels
to ensure easier access for servicing. |
 |
| |
|
| Raised
Floor Thermal Management: A cost-effective
airflow and thermal management way to improve airflow
in your data center or server room is to make sure that
all cable openings in your raised floor tiles are sealed.
Based on measurements at multiple data centers, 50 -
80% of valuable conditioned air is not reaching the
air intake of IT equipment due to unsealed cable openings.
This lost air, known as bypass airflow, contributes
to equipment hot spots, cooling unit inefficiencies,
and increasing infrastructure costs. Sealing cable openings
is an essential component of IT equipment reliability
and availability.
Raised Floor
Grommets are a cost-effective remedy.
|
 |
| |
|
Freestanding Chillers
With an output range from 32 kW to 172 kW (131,245
to 587,411 BTU) to satisfy the demands of high cooling
load applications, freestanding chillers are up to the
largest task. Servicing is also easy with removable
panels for quick access.

Chiller Technical design:
- Robust industrial standard in 3 enclosure sizes
- Identical basic enclosure for oil and water chiller
systems
- Extra space for the integration of special equipment
- Variable air routing is possible via the left or right
sidewall
- Floating contact for collective fault signal
- Level monitor
- Multi-coil vaporizer in the tank
Chiller Configuration:
Chiller system wired and plumbed ready for connection,
with multilingual documentation including functional
diagram and wiring plans.
Cooling Accessories
Interface Board
The interface board is an extension for 42U Rack
cooling units with a comfort controller. In this way
it is possible to monitor a master/slave combination
of up to 10 cooling units.
Optimize Airflow
in your Rack Enclosures
HotLok
- Blanking Panels
Optimize air flow within your server racks by using
effective sealing quick finger install blanking panels
while monitoring temperature within your racks with
optional temperature strips.
Digital Enclosure Internal Temperature
Display and Thermostat
For installing on the enclosure door or
wall and in a cooling unit or heat exchanger.

|
|
|
Speed Control
Temperature dependent speed control for Rack fan
and filter units and air/air heat exchangers with a
rated operating voltage of 230 V AC for noise reduction
and to save energy in part-load operation.
Conclusion
“Greening” the data center often starts with the
cooling infrastructure. This page addresses the basics,
hoping to facilitate deeper discussions on current vs.
future cooling technologies. From chillers to
CRACs to economizers, there is no shortage of vendors
who, armed with case studies, tables, and whitepapers,
are championing the efficiency of their products. Our
goal is to help the data center professional wade through
mountains of vendor data to find the most germane, economical,
and efficient products for his application.
Bibliography
Cappuccio, D. (2008). Creating Energy-Efficient,
Low-Cost, High Performance Data Centers. Gartner
Data Center Conference, (p. 4). Las Vegas.
EPA. (2007, August 2). EPA Report
to Congress on Server and Data Center Energy Efficiency.
Retrieved January 5, 2009, from Energy Star:
http://www.energystar.gov/ia/partners/prod_development/downloads/EPA_Report_Exec_Summary_Final.pdf
McGuckin, P. (2008). Taming the Data
Center Energy Beast. Gartner Data Center Conference,
(p. 5). Las Vegas.
Sullivan, R. (2002). Alternating
Cold and Hot Aisles Provides More Reliable Cooling for
Server Farms. Retrieved December 15, 2008, from
Open Xtra:
http://www.openxtra.co.uk/articles/AlternColdnew.pdf
The Green Grid. (2008, October 21).
Seven Strategies To Improve Data Center Cooling Efficiency.
Retrieved December 18, 2008, from The Green Grid:
http://www.thegreengrid.org/gg_content/White_Paper_11_-_Seven_Strategies_to_Cooling_21_October_2008.pdf
The Green Grid. (2009). The Green
Grid: Home. Retrieved January 5, 2009, from
http://www.thegreengrid.org/home
About 42U
focuses on providing Facilities and Data Center Managers
with objective and trusted independent information on
Data Center Power and
Data Center Cooling best practices along with
KVM over
IP and
Remote
Power lights out management to help them reduce
their organization's power consumption. Our
Data Center Management
website has helped millions of IT professionals globally
who search for best practices and technologies to help
them make their computer environment more efficient.
This work is licensed under a
Creative Commons Attribution-No Derivative Works 3.0
Unported License.
|
|
|