Intel finds significant savings by pushing the limits of free cooling
Wouldn't it be nice if you could simply shut off the computer-room air conditioning units in your datacenter? You'd no longer be exposed to temperatures found in a meat locker -- but more important, you'd reduce your facility's energy consumption significantly if you didn't have to keep your machines chilled, resulting in lower energy bills and a smaller carbon footprint.
Notably, some datacenter operators have dabbled in running their datacenters sans artificially cooled air, relying instead on an air-side economizer. An economizer draws on outside air to cool the datacenter, then pushes the hot air that exits the machines outdoors. But use of this so-called free cooling tends to be limited to regions with temperate climates, where there's no risk to servers from harsh conditions.
However, Intel last year performed a groundbreaking experiment in free cooling, one that provided some invaluable data as to just how resilient servers can be and how much electricity and money might be saved by reducing datacenter reliance on artificial cooling. The company tested the limits of air-side economization by allowing production servers be cooled by outside air at temperatures as high as 90 degrees Fahrenheit. Furthermore, the company challenged the perception that using outside air can be harmful to servers in that it subjects machines to excessive humidity and contaminants.
"As a leader in both technology and green initiatives, Intel needed to take a risk in the datacenter space to understand what might be possible if we were willing to challenge our long-standing datacenter assumptions," says Don Atwood, regional datacenter manager. "We believed this could change the way Intel and other companies consume energy in the future."
Intel orchestrated its proof of concept at a datacenter location in New Mexico by setting up about 900 heavily used production servers in a 1,000-square-foot trailer, which was split into two 500-foot compartments. One compartment was cooled 24/7 by a relatively low-cost, warehouse-grade direct-expansion air-conditioning unit; the other was cooled almost exclusively by outside air, though it was also equipped to be cooled by the direct-expansion chiller if the need arose.
As explained by Intel's report on this test, "We designed the system to use only the economizer until the supply air exceeded the 90-degree maximum, at which point we began using the chiller to cool the air to 90 degrees. If the temperature dropped below 65 degrees, we warmed the supply air by mixing it with hot return air from the servers."
Ninety degrees is an unusually high temperature for a server room; Intel says in the report that it traditionally sets its chiller temperature to 68 degrees in its datacenters. Conventional wisdom dictates that air-side economization is better for mild, temperate climates, not regions where temperatures can hit the 80s, 90s, and beyond. There was method to Intel's seeming madness, however: "We reasoned that this might be feasible because server manufacturers specify that their products can operate in temperatures as high as 98 degrees."
Not content with simply pushing the temperature boundaries, Intel wanted to gauge how resilient the machines could be to humidity and air contaminants. To that end, Intel used no humidity controls whatsoever and installed just a standard household air filter "that removed only large particles from the incoming air but permitted fine dust to pass through."
Intel didn't go easy on its machines as it conducted this experiment. The chipmaker says that the servers were working at 90 percent utilization during the testing, crunching large production batch silicon design workloads.
As for the type of servers, Intel says they were standard, single-core, dual-socket, Intel-based machines that were roughly two years old. In other words, Intel didn't enlist specially hardened or ruggedized equipment but instead used the same type of machines found in the company's other datacenters.
During the 10-month test, Intel found that the machines cooled by outside air experienced humidity variations from 4 percent to more than 90 percent, and that it changed rapidly at times. Moreover, "the servers and the interior of the compartment became covered in a layer of dust."
Once the experiment was complete, Intel calculated that the air-economized trailer required 74 percent fewer kWh to keep cool then the air-conditioned trailer. Based on the testing, Intel concluded that in this particular region, at least, it could rely on air-side economizers 91 percent of the time and save $2.9 million on cooling at a 10MW datacenter. That also means significant water savings -- 76 million gallons annually -- and a tremendous carbon-footprint reduction.
The other big question, of course, is how well did the server weather the unusually harsh conditions? After all, a datacenter full of fried or dust-clogged machines is pretty much worthless, no matter how inexpensive it is to cool. The answer: surprisingly well. "Despite the dust and variation in humidity and temperature, there was only a minimal difference between the 4.46 percent failure rate in the economizer compartment and the 3.83 percent failure rate in our main datacenter over the same period," Intel reports. "The failure rate in the trailer compartment with DX [direct-expansion] cooling was 2.45 percent, actually lower than in the main datacenter."
In fact, according to Intel's Atwood, the company sent three sets of identical servers (both type and age) to IBM for a blind study to determine the potential long-term affects of the harsh weather conditions on the machines. The three sets of servers included machines running on the free cooling side of the test facility, servers from the closed-loop cooling side of the test facility, and servers from Intel's enterprise datacenter. "IBM was unable to differentiate the enterprise condition servers from the free cooling servers and/or the closed loop cooling side," Atwood says.