Water absorbs and transports heat much better than air. However, the coolant of choice in most computing scenarios is air. Why?
Well, an air-based cooling system is easier and cheaper than a water-based system. Besides, water and computers do not mix well. Bringing water near computing equipment worth millions of dollars, even when designed properly is nerve-racking.
However, High-Performance Computing (HPC) is a part of the computing industry where water cooling systems have been the norm. Supercomputers, which have many high-end CPUs and GPUs crammed into every server, constitute a small piece of the industry.
Image of a data center (https://pixabay.com/en/data-center-the-engine-room-2476790/)
Lenovo and other hardware manufacturers argue that very soon, data centers will start implementing liquid cooling. This move will primarily result from increasing demand for resource-hungry machine learning applications, virtualization, and big data analytics.
Lenovo is putting its money where its mouth is by investing in the development of liquid cooling systems. The Hong Kong-based company hopes that more customers will purchase these designs.
Processors Become More Power Hungry
Over time, processor power is increasing exponentially. Companies that design processors are looking for ways to extract more computing power from every watt of electricity consumed. However, these newer processors still consume more power than their predecessors.
On the motherboard, processor chips are surrounded by IO cards, storage devices, and memory. These devices are also becoming more powerful and power-hungry. As servers power increases, air cooling them is going to be tricky.
According to Lenovo, CPUs will soon exceed 240 watts and co-processors such as GPUs will exceed 300 watts. Servers that only consist of CPUs will exceed 1kW.
Image of CPU (https://unsplash.com/photos/eM6WUs4nKMY)
Lenovo Unveils Two Liquid-Cooled Designs
Design One: The Less Intrusive Design
The first design introduces efficient liquid cooling into data centers without the need to replace the cooling system in the data center. So the only change data centers make is bringing in the water.
What Lenovo has designed here is a very special heat sink that’s full of liquid. Through this system, data centers can easily cool higher-density processors, at the same time reducing the amount of air needed to cool the server. Lenovo calls it the Thermal Transfer Module (TTM).
Many two-CPU servers are designed in a way that air travels over one CPU prior to traveling over the other one. Consequently, air reaches the second CPU when it’s already warmer.
With Lenovo’s new heat sink, the liquid inside carries the heat away to a section of the module that has a larger area. This increases the surface area for heat dispersion before expulsion.
Companies that purchase Lenovo’s high-density servers will have an option to include the TTM. Will this increase the price? Yes, by a maximum of $25 for each server. The TTM can cool 205W CPUs using liquid, without the need to install an extra water loop in the data center.
Design Two: The More Intrusive Design
The second design is more invasive in the data center infrastructure. It is designed to maximize energy efficiency. This design cools the servers using warm water piped directly to the chips. It’s no surprise Lenovo calls it Direct to Node (DTN).
The system is efficient because it can use 50-degree Celsius water to cool at least 45kW per rack. At this temperature, there is no need to install chillers into the data center to cool the water.
By bringing the water to the chips, Lenovo is taking advantage of the water’s natural heat-transfer properties. eBay uses such a system in one of its data centers designed by Dell.
Lenovo pushed the DTN design further at the Leibniz Supercomputing Center in Munich. The facility is owned by LRZ, one of Lenovo’s customers. In this design, Lenovo added the capability to capture heat from the water as it cools, and using that heat to add the cooling capacity.
However, not all equipment in the data center can be water-cooled. Storage and networking equipment can only be cooled by air. To boost the capability of the traditional air-cooling system, warm water at 60C carrying heat from the servers is piped into an absorption chiller.
The chiller evaporates the water, making it lose heat. It then pressurizes the vapor to convert it back to liquid state. When the water gets into the chiller, it is already warm and needs little heat to vaporize.
The Beginning of Liquid Cooling in Enterprise Data Centers
Majority of Lenovo’s customers who use liquid cooling are in HPC. However, this is bound to change over time. These two water cooling designs and a rear-door heat exchanger are what Lenovo refers to as Neptune.
Lenovo unveiled Neptune just before the ISC, a big annual conference by the HPC industry scheduled to kick off on Sunday. This is indicative that supercomputers still play a major role in the liquid-cooled data center market.
As hardware increases in power, companies can put more VMs on a single unit that they previously could. Data center operators are under pressure to squeeze as much value as they can from every box they purchase.
NVidia reports that companies are buying a large number of GPUs for AI. This corroborates with reports of increasing demand for GPUs at Google Cloud and Microsoft Azure for use in AI.
Google uses liquid to cool its custom AI chips known as Tensor Processing Units (TPUs). If the systems that businesses use grow to reach similar power levels, then it’s reasonable to imagine that the demand for liquid cooling is bound to grow.
