We work with clients to choose the most efficient servers and software solutions, but now is coming along an entirely game-changing technology, chips that use dramatically less power, about 1/20th, over existing technologies, and also more importantly, even much less energy in that they can turn off when not in use and immediately turn back on again when needed as processing demand increases again. In steps ARM-based processors for servers—the same technology used in our mobile devices today—that uses much less power and turns on and off much quicker than server processors of today.

Microsoft made a statement that they have been working with ARM based chips since 1997 and are now working with ARM in a new licensing agreement “to enhance our research and development activities for ARM-based products”. Quite a change from the strong partnership Microsft has had with x86 chips from Intel and Advanced Micro Devices. Even Apple has made acquisitions of companies and hiring ARM chip engineers and ARM specialists, likely for use in their growing line of smaller devices—the iPad, iPhone, iPod, etc.

Startup SeaMicro unveiled a server running on Intel Atom chips with a fabric that puts CPUs to sleep, allow for a lower energy use rack. They claim they can have as many as 2,048 CPUs into a 40U rack and use 8 kWs of power. These calculations seem to be with half of the processors off per rack, a good improvement over standard racks, but can we do even better?

Enter Smooth-Stone with a even more advanced approach with ARM-based processors designed for servers. The company is working to build a rack with a similar number of processors, but with ARM based processors, they can shut off and back on again these processors in a much more rapid fashion. These processors should be able to withstand much higher temperatures, vibration and other tolerance—after all, they come from mobile applications where these are requirements unlike the soft-glove approach in most data centers. The advantages of Smooth-Stone’s ARM-based servers should provide a significantly greater amount of onboard memory, compute performance, network bandwidth, lower costs and many other advantages in a chip that only uses single digit Watts, providing a significant performance to Watt advantage. Plus being able to quickly turn on and off every processor in a rack except one running at 1/10 to 1/20th or even less than a traditional processor when the rack is unneeded, and bringing the rack’s processors back up individually within microseconds is a distinct advantage in energy savings, the largest operating expense and driver of capital expense in data centers.

With Smooth-Stone’s SoC and software and experienced team designing the lowest power consuming servers, this may likely be the game-changing technology in IT and data centers.

Later, I was with Google operating and acquiring large data centers, and I had the privilege of running the largest data center in square feet that I’ve ever known. Now fast forward years of data center design, construction, operations and efficiency programs for data centers, and I’ve come to see that while larger may be lower cost to build and operate at the time of construction, in most cases larger data centers cost more over time than ‘medium-sized’ (relative size to time to load up) data centers. Why? Large data centers are rarely future proof. Our server technology leaps ahead a generation every 18-months; software generations are often 6-12 months. New infrastructure solutions are coming along every year, and capacity planning is rarely good more than 6-12 months out and sorely inaccurate at that. Case in point: a data center I built for Yahoo that would take 3-5 years to fill wanted to be modified to accommodate new cooling technologies and wanted to be moved to a different state to take advantage of changing tax laws. Yet modifying a concrete shell and relocating an entire and new data center are impractical and often costly solutions.

If we build data centers that are scalable in that they the are smaller and thus at the scale of 1-3 years of capacity, then we can always implement new technologies, solutions and changing business capacity and needs quickly and at lower cost over time. This may mean we plan for the next ten years when we site select our data center “campus”, but build out smaller shells in more frequent build cycles than one large building that can last for 5+years. I spoke about this in a recent ComputerWorld article.

The article also mentions the “butterfly” data center concept from HP, discussed further in this article. The key tenant of the plan is to build smaller buildings on a campus with shared network, personal and other benefits. Why not also build a small computer factory on the campus to serve the data centers and surrounding area? Another concept to this approach is with containers, which eBay just added themselves into the fold of those considering containers for future modular scalability.

I’ve built many of the lowest cost data centers ($4-8 million per MW of IT load) and the most energy efficient data centers (PUE 1.04-1.10), and I believe these smaller data centers can be built for very comparable cost and efficiency figures, likely the same, and perhaps even lower, especially over time as retrofits are less likely and less costly. I’ve seen examples of this in many of the previous data center projects I’ve been working on, which are 10+ MWs in size, ‘medium’ in relative terms.

Various data center news:
Fast growing company looking for a data center manager position in Santa Clara, IT focused, Linix & storage. Write me if you are interested.

Terremark Worldwide release solid Q1 results, raising annual revenue projections. Guidance for 2011 assumes no federal IT project revenue although that has been accounting for 10% of revenue. Cloud revenue growing and now accounts for 8% of total revenue.

Internap Network Services released its Q2 results, which were on target with revenue and margins, significantly increasing gross margin by exiting partner data centers into company owned data centers. I see this trend over time, as folks move from outsourced into company owned data centers. I can help you with this transition, having completed it and the best strategy to do so many times now with very excellent results.

Investment in “greener data centers” to increase over 5x in 5 years per Pike Research or put another way, capture 28% of global market by 2015 in this story, that I highlighted in a previous blog post.

The main difference of this program and summit is that all presenters talk about their data centers, actual end-users of data centers. No presenter pays to speak via a sponsorship; this event has actual end-users presenting actual data, often validated by a third-party.

This program us put on mostly by a committee of volunteers, including my co-chairs: Ray Pfeifer, Tim Crawford and Brian Brogen. Many thanks to them for helping to pull this program and case studies together. Also many thanks to Kelly Aaron, Ralph Renne and Mukesh Khattar for helping with marketing, agenda and event planning. Zen Kishimoto, Joyce Dickerson and John Noh also helped with their valuable input. Dale Sartor and Bill Tschudi of LBNL helped immensely with leadership and ideas, and Paul Roggensack of the CEC with funding of the program. And most importantly, thanks to Bob Hines and Anne Smart with the SVLG for helping to plan the Summit.

You can view the program of the summit here. If you would like to submit a case study for this year or next year’s program, contact me or send an email to dcespapers@svlg.org. Case studies must be from data center end-users.

If you would like to sponsor the upcoming Summit , contact Bob Hines at the SVLG via the info on the event website. You can register for the Summit here or via the Summit website. The event has sold out the last two years and I expect a sell-out crowd again, so register early. I look forward to seeing you at the SVLG Data Center Efficiency Summit October 14th.

“One of the other lessons learned from the meeting is that central-control systems are more effective at reducing energy consumption than relying on employee practices. Purchasers who implemented employee training programs to have people turn off their machines at the end of the day reported maximum penetration rates of 65 percent, declining rapidly over time.”

“In contrast, most organizations have focused on control solutions, where IT staff program computers to turn off on a timed cycle. This is often matched with settings to turn off monitors or put computers into sleep-modes after a certain period of inactivity. Purchasers report almost no user resistance to these solutions and consider it part of a larger trend of centralizing control of individual computers over a network. Most purchasers have solved common concerns about timed off-cycles with a software-based solution like the NightWatchman or Surveyor Windows server monitoring software.”

When I was at Sun Microsystems (late 90’s-2000), I found similar results. When I asked the 40,000+ Sun community to turn off desktop monitors and computers at night, rarely did they, even though s study I commissioned showed the savings to be well into millions of dollars per year (as most were left on during nights and weekends and on average, employees were only at their desk about 4 hours per work day). But when I had a third party switching device (MonitorMiser) added to all desktops that automatically turned off monitors when desktops were inactive (no mouse or keyboard input for 15 minutes), only three people of over 40,000 complained. Savings = over $3,000,000 a year in US. (Most monitors were 17-24” CRTs, and each employee averaged over two as many had several.)

I then took this a step further and asked that the Operating System turn desktops off when inactive. This was a bit more challenging, as what was inactive to user input might be actively running code all night. So I had the software engineers put in some more code to look at processor state, network activity, and keep it user selectable. This was a very crude “sleep” mode for the OS and a beginning of those for the industry. The industry followed what we did I think not for energy savings, but because Sun sales engineers started selling computers on TCO including energy use and winning deals. The sales team was realizing by the late 90’s that lower total cost, and consequently lower energy use of the equipment, helped to make sales. These and other changes led to over $10,000,000 in annual energy savings I implemented and led to earning my second EPA EnergyStar Partner of the Year Award.

It’s been great to see this early and rather crude OS function automatically put monitors and desktops to sleep and/or off states. Wow! Now look at what our desktops, servers, even networking and storage equipment can do to help it reduce energy use when underutilized.
Take this one more evolutionary step forward, and you have what I call server power management software (1E, PowerAssure, Surveyor, and many others) that automatically determines hardware utilization and state, and either puts it to sleep or off and then automatically turns it back on when needed. How far this has come from our early and rather crude versions of this with desktops at Sun.

Think about it: as a company you want to utilize all of the assets you have to perform work that maximizes revenue (or profit). But you also need assets for peak periods that are underutilized during low demand periods. Think New York City taxis. They are busy as heck on a Friday night or during rush hour yet rather idle at 4 AM on a Sunday. You wouldn’t want every one of those cabs with a paid driver idling their engine burning dollars out the tailpipe now would you? So the cabs are parked, the drivers are home asleep, and they work when demand warrants it. So why do we leave our servers (and storage and network ‘taxis’) idling 8,760 hours a day when average peak times are well less than 1,000 hours per year (often less than 100)? We do this and wonder why we have average processor utilizations of less than 10%. (Processor capacity is rarely the limiting factor in most applications these days, but that is a topic for another blog.) And yet those servers consume about 2/3 of peak power when at 0% processor utilization, so why leave them running, burning precious company dollars out through the power meter? Is it charity to our utility companies? I doubt it. So power down those servers when not needed and save precious dollars for more important tasks than burning power unproductively. After all, those servers do have an on/off button. And call the experts at MegaWatt Consulting for these and more solutions to increase your dollars. Power on…productively.

Savvis also beat quarterly revenue forecasts, seemingly helped by their acquisition of Fusepoint, which closed in June. This acquisition opened up the Canadian market for Savvis, which the Canadian market has been economically very strong and with data centers over the last few years. Over half of the new, greenfield data center site selections I have completed over the last 12 months and almost as much of the new data center designs I have worked on over the last 12-months were for data center projects in Canada. I’ve been saying for a few years now that Canada is a fantastic place to locate data centers with low-cost energy and cooler climates. However, network connectivity and more importantly, taxes, can be a challenge for some corporate customers. All of the data center projects in Canada I worked in the last 12-months were either Canadian-based companies or quasi-government or educational organizations. I’m now quite versed in great locations for data centers across Canada and working with their utilities and government entities. Back to Savvis. I expect we’ll see higher growth in Canada now from Savvis and likely future expansions there as well, not just for Savvis, but also others. Churn was lower than expected at 1.3% and Savvis added growth from the financial services sector, probably mostly driven by the partnership with Thomson Reuters and also new capacity additions in some key markets, which should position them to grow well over the next couple of years in what I believe will be a supply-constrained data center market. Cloud services, which Savvis is going after hard, also grew by 25% for the quarter. I believe that new growth in the financial services sector can be an indicator of a returning/growing economy, as when financial services ramp up, people are investing, and investment fuels economic growth. Secondly, I believe the growth of cloud services, being apparent in Savvis’ quarterly results, is an indicator of growth in this industry segment that we’ll see across the data center industry: cloud will grow. As more people realize the benefits, convert applications and systems and processes to cloud, cloud offerings, services and products will grow to support it. It seems that internal and external cloud will continue to grow and become a standard data center household tool for all data center users. Rackspace’s quarterly results will be released shortly. I believe their results will be telling of the cloud computing space as well, since I see them as the number two provider of cloud to Amazon.

On to the big fish in data center co-location, Equinix. Similar results: revenue was ahead of guidance and consensus. Switch & Data acquisition finalized, ahead of schedule, adding much needed capacity especially in key markets but more importantly, securing a network-neutral co-location company, of which Equinix now has very limited large, network-neutral competitors, reducing network-neutral competition and helping Equinix secure top position as the place to peer and connect to multiple networks. This “eco-system” of networking peering allows for Equinix to continue being the place to connect to various networks and peering partners. When I was with Google and Yahoo, we co-located with Equinix for peering and network connectivity (this is well known so I’m not divulging any secrets here), so Equinix gets to boast big brand marquee names as customers even though in most cases they are small revenue customers (we’d only locate a few racks for peering, network and caching servers). Many enterprises do the same, but often choose to locate large groups of racks (backend functionality) at their own data centers or lower cost wholesale data centers. I contracted this mixed-portfolio approach while with my past employers and also recommend it for many large enterprises, although each solution is unique to the company needs. Another interesting point about Equinix is that they saw significant growth in Europe. It appears that my office neighbor when I was consulting to Equinix, recently moved to Europe to run operations there and is doing a great job. Another interesting point is that Equinix is reporting 78% cabinet utilization in the US, 70% in Europe and 65% of Switch & Data’s; that indicates that Equinix will need to begin adding more capacity again, as they have been recently in the US.

This is only a sampling of data center quarterly reports. I’ll try to add more over time. These are only my opinions and market outlook and not investment advice. All figures reported are from publically available earnings reports.

So, we came up with a compromise, knowing that all data centers and their workloads were different, yet needing something to push us as an industry to higher efficiency. Well, the holly grail of data center metrics got released….P U E. Yes, Power Usage Effectiveness. While it was only a start, and a best compromise, and we knew we needed to improve upon it or come up with something better, yet is has had perhaps more influence on energy efficiency of our data centers than any other metric or industry movement.

While improving PUE only affects the infrastructure side of the data center, not the hardware or software–leaving that always equal to 1.0 with everything else being above one–more power use the higher the PUE. Our data centers have been averaging above 2.0 (meaning that at 2.0 the infrastructure power load is equal to the server load, higher than 2 means it uses more power than the server load). A recent report from EPA of about 200 data centers last year across the US shows that we are averaging north of 2.0. Other studies show that we had been averaging around 3.0 worldwide, so we have improved greatly but still can improve so much more. While 2.0 is much better than 3.0, using 50% less power for the infrastructure, we know we should be able to achieve PUEs of at most 1.5 any where in the world, any TIER level, yet at 2.0 we are using more than double the power we need to support the non-hardware loads.

One problem with the PUE metric is that it is instantaneous since it measures power and not energy and all data center power usage fluctuates with weather and usage, and really what we care about to reduce costs is total energy use over a period of time. Energy is power use over time, while power is instantaneous. Otherwise, PUE could be measured on the coldest day of the year when all systems are running more efficiently but that is not a good gauge of annual energy use and thus costs. In all of the projects I get involved with, and all of the PUEs I quote, I use total annual energy instead of one-time power measurement, and hardware measured at the rack to account within PUE UPS, PDU and other electrical distribution losses. So, PUE should be an annual average, and that is exactly what member representatives of Green Grid, SVLG, 7×24, EPA, DOE, USGBC, ASHRAE and UpTime recommended in December of this year at a meeting in DC. I provided recommendations from the SVLG along with Chris Page, Scott Noteboom and Tim Crawford representing the SVLG at this meeting with input from Olivie Sanche of Apple and many others.

Essentially the outcome was a revised PUE metric that now measures annual usage of infrastructure and IT load, which is fantastic! Also, a little more clarity or definition on how it should be measured and what should and should not be included. (Such as on-site power generation should never reduce one’s PUE, as energy in is energy in, regardless of source.) We’ll soon see PUE and PUE subscript 1, 2 & 3. These clarify where the server load was measured (UPS output, PDU or rack). Ideally, we’d all be measuring at the rack input, but many folks do not have this meter & monitoring capability, so the compromise was to allow for some acceptance of any of these points of measurement.

Even though the location of measurement will affect the measured PUE–meaning different measuring locations will result in different PUEs for the same data center–at least it’s an improvement, and will hopefully drive folks to measure at the rack–the most accurate location of measurement. It will also drive us to think about annual usage and costs, not one time or instantaneous, another big improvement in our thoughts about all buying and operational decisions. These are the key to improving efficiency and reducing costs: long-term measurement, long-term constant improvements, and buying decisions based on long-term economic analysis.

With our new PUE metric, it should re-invigorate the PUE discussions, comparisons, and improvements. Perhaps driving us all to lower PUEs, regardless of actual resulting PUE and type of data center. After all, we all gain when we each improve.

And speaking of low-carbon intensity, Yahoo’s Quincy data center, which I led the completion of the first phase of construction before starting MegaWatt Consulting, recently released a new low-carbon option in managing our data centers: Goat Power. Enjoy this short video with my Yahoo friends Chris, Lisa and Ty and some new goat friends as well. Looks like one of the goats was particularly fond of Chris as well, or at least her shirt.

I spoke at the Green Data Center Conference in San Diego over the last three days. I taught a three-hour energy efficient data center workshop, also a one hour session about energy options and efficiency ideas for data centers, and also joined in on three panels: energy sources moderated by John Diamond, Organizations and Associations moderated by Bruce Myatt while I talked about the SVLG Data Center Efficiency program I co-chair and the McGill University high-performance co-location data center project case study that I helped with site selection and design ideas with Rumsey Engineers. Eric Soladay with Rumsey Engineers did a great job presenting the efficient data center design, with designed annual PUE of 1.06, and the very interesting snow-field concept for cooling this high-density data center without chillers or any other compressor-based cooling through 90% humidity and 90F summer time weather.

After 10+ years of talking about data center energy efficiency being important and concepts to improve the energy efficiency in data centers, as well as sharing my own experiences, I am so glad to hear that these ideas are sticking as well as the importance of energy efficiency. I was even more proud to that many of the ideas I have been pushing for the last several years as well as terms that I believe I coined nearly 10 years ago are sticking and being used in the regular vernacular of the industry: Holistic (designing and operating data centers as an entire system of hardware, software and infrastructure to achieve lowest total cost and highest availability for the intended purpose) and server hugger (the “need” (aka want) to have one’s data center and/or servers located nearby, often an emotional response and not a technical or rational need.)

Remember to look at your specific needs and also be creative with carbon reductions, like how you cut your data center grass!

So, here is one that will not catch you by surprise: a rack that saves energy. We’ve all heard of passive and active cooling racks: those with fans, heat exchangers or direct cooling systems. I explored some of the front & rear door heat exchanger racks back in 2003, which work really well for high-density applications but can be very expensive compared to better-designed data center cooling systems. But how about a rack that not only reduces energy costs but also improves hardware performance?

I’ve had the pleasure of exploring with Green Platform’s CEO, Gus Malek-Madani, their anti-vibration rack (“AVR”), a carbon-fiber composite rack actually designed to remove vibration. Why remove vibration? Green Platform claims that a typical datacenter experiences vibration levels of around 0.2 G-Root Mean Square (GRMS); this, it claims can degrade a disk drive’s performance (both I/O and throughput) by up to 66%; a fact that was borne out during a ‘rigorous’ testing exercise it did in conjunction with Sun Microsystems.

As harddrives get “larger” in capacity, bits get crammed into a smaller space. This along with smaller drives force tolerances between rotating platters and the movement of mechanical actuator arms within the drives to get tighter, and thus, vibration causes drives to slow down or have higher mis-read & writes, slowing down I/O performance. “As a result of this ‘vibration penalty,’ the company believes that up to a third of all US datacenter spending – on both hardware and power – is wasted on vibration, amounting to some $32bn of wastage. The company also says there’s evidence that reducing the impact of vibration will serve to improve the reliability of drives (and improving mean time between failure.)”

In order to back up this figure, early tests with Sun Microsystems (pre-Oracle) and Q Associates (“Effects of Data Center Vibration on Compute System Performance” by Julian Turner) showed IOPS improvement of up to 247% in random I/O. The following chart shows this storage performance degradation:



You can also watch the following video that clearly shows that just yelling into the face of storage hardware causes a very visible degradation of storage performance: http://www.youtube.com/watch?v=tDacjrSCeq4

If the vibration from yelling into a rack causes performance degradation, think about the vibration affects from HVAC systems, thousands of server fans, and even walking thru your data center.

The company says its carbon-composite design massively reduces the vibrations that can cripple hard disk drive performance, boosting performance, efficiency and even reliability. From the results of these tests stated above, they assume that most folks should see a 100% improvement in storage throughput, 50% shorter job times and consequently, 50% less power consumed per job. In testing with Sun Microsystems the AVP dissipated vibration by a factor of 10x to 1000x. In further testing with systems integrator Q-Associates, which pitted the AVP against a regular steel rack – it found that random read IOPS increased by between 56% and 246%, with random write IOPS showing a 34% to 88% improvement with the AVP.  

“The throughput and I/O rate of storage remains a significant performance bottleneck within the datacenter; though hard disk drive (HDD) capacities have increased by several orders of magnitude in the last 30 years, drive performance has improved by a much smaller factor. This issue is exacerbated by the fact that server performance, driven by Moore’s law, has increased massively, to the extent that there’s now a server-storage performance gap. The way most datacenters engineer around this problem is inefficient; typically workloads are striped across many disks in parallel, and disks are ‘short stroked’ — i.e. data is only written to the edge of platters – in order to minimize latency. Although this does address performance, the trade-off is that disk capacity is massively underutilized, wasting datacenter space and energy, not to mention the cost of reliably maintaining an unnecessarily large disk estate.”

In the many data centers I have had the pleasure of working in lately, storage is growing faster than server capacity and the greatest performance limitation is storage throughput. This product works for the high-end video/audio and scientific markets; a niche space where another of Malek-Madani’s company– Composite Products, LLC — is focused. The test results clearly show storage throughput dramatically improved by reducing vibration at the rack of storage hardware. With some 3 million storage racks currently in use inside datacenters worldwide, and growing by the second to probably eventually exceed server racks, this is a very large opportunity to improve performance while reducing energy use, always one of my main mantras. Green Platform expects to have their racks as an option from storage vendors, NetApp, EMC, and others, so that as you purchase and provision new storage systems, you pay a small incremental increase in price of the storage system for a very large improvement in performance and energy reduction. Think of all of those servers waiting so much less for data throughput and how much that can improve the utilization of those systems? Think about it.

I’m looking to conduct an end-user test with their rack; contact me if you’re interested so we can determine results for your organization.

Now, enter 2010. UPSs are still assumed by nearly every data center engineer and operator to be needed or required, yet, power electronics within the computer equipment can ride thru just about any voltage sag or surge a utility would pass on thru their protective equipment. Computer equipment power supplies have been rated for 100-240VAC and 50-60 Hertz for about 10 years now, so a far greater range than an utility will likely every pass on. Furthermore, due to capacitors in the power supplies, these devices can ride thru complete outages of about 15+ cycles, which is roughly 1/4 second. So the UPSs job is really now only to provide ride thru of outages over 1/4 second and until a generator comes on or as needed by the operation.

In many of the data center design charrettes that I have been part of over the last few years, we ask the users what really needs to be on UPS, avoiding the assumption that all computer load must be on UPS. Once we dive into the operations, we always come back with an answer from the data center operators that only a portion of the computer load needs to be on UPS and the rest can go down during a usually irregular utility outage. The reason is that these computers can stop operating for a few hours and not affect the business. Examples might be HR functions, crawlers, back up/long-term data storage, research computers, etc. Computers that might need to be on UPS include sales tools, accounting applications, short-term storage, email, etc. but not every application and function. Think about your own data center operations about what can go down every now and then from a utility outage (usually about once per year for a few hours) and see if you can reduce the total amount of UPS power you require and repurpose that expensive UPS capacity and energy loss to the critical functions.

Some data centers avoid UPSs entirely by putting a small battery on the computer itself, in widely publicized Google’s case, an inexpensive and readily available 9V battery. While this is an excellent idea for those that have custom computer hardware, it is not as easy to implement for most folks buying commodity servers today. Perhaps another idea better for the masses is to locate a capacitor on the computer board or within the server that can ride thru ~20+ seconds until generator(s) can supply load during a utility outage. Capacitor technology of today should make this fairly easy to implement and could be a standard feature on all computer equipment with a minimal added cost, much as the international power supplies did for us 10+ years ago and higher-efficiency power supplies (90+) are today. A great new technology that could make this easy to build on the computer board can be seen here:
http://newscenter.lbl.gov/feature-stories/2010/04/23/micro-supercapacitor/

Using a technology like this we could avoid UPSs entirely in our data centers by having enough ride thru built onto the computer boards, into the hardware, allowing us to save very expensive UPS power capacity, operating and maintenance expenses and space within our data centers for more important functions, compute and storage capacity. My thought for the day. Think about it and you might save some money and energy.

Finally, ASHRAE advanced the standard, which so many data center designers, operators and engineers relied on for operating specs of their data centers. Even though the new standards expanded the humidity and temperature ranges considered “allowable” within our data centers, the hardware specifications allow for even much broader ranges. While ASHRAE TC9.9 allows for up to about 80 degrees F (depending upon the humidity), hardware manufacturers spec their equipment to run up to 95F inlet air and humidity generally between 5-95%. The computer hardware of today lacks vacuum tubes and paper punch cards; circuit boards are coated to prevent arc flashing; everything is solid state except two moving parts: the harddrive, which are hermetically sealed, and cooling fans to draw air thru the server. (And each of these has a limited life, already with solid-state options or complete removal (fans)).

Many studies have been completed to prove that the hardware no longer needs tight temperature and humidity ranges, nor even filtration to prevent dust accumulation. Microsoft proved this point with what I call their data center in a tent experiment, in which they located a couple of racks under a tent canopy in Redmond, WA for many months. The servers sucked in leaves, ran when it rained, snowed, blew, was hot and cold, and even had water leak onto them, without any hardware failures, proving the point that servers really don’t need any environmental controls. A link to this study is here: http://blogs.msdn.com/the_power_of_software/archive/2008/09/19/intense-computing-or-in-tents-computing.aspx

I myself completed very low humidity (10-20%) and high inlet temperature (75-85F) testing, as well as unintentionally water leaks onto servers back in 2002-2003, all without any failures whatsoever.

Another bold move was by chip-maker Intel, in which they completed what I call their data center in a box test, but different than most containers, in which they put two containers, each loaded the same with 900 servers, in Santa Fe for a year. One had air conditioning and it’s inherent humidity control as well as filtration, the other, a fan to draw outside air in with minimal filtration and no air conditioning. What a great test! Santa Fe can get hot (92F) and cold (24F), being in the high desert, normally very dry, but also with sudden thunderstorms on hot summer afternoons that made humidity range from 4-90%. Intel found that the air economized container had a very visible layer of dust on the servers, and that even though temperature and humidity ranged dramatically during this year long study, the server failure rate was 4.46% versus 3.83% in their main air-conditioned data center. A failure rate of this low is ridiculous when it is economic today to refresh (replace) hardware at 18-24 month cycles and the failures seemed to coincide with dramatic humidity changes (10 to 90% in one hour) . This study proves that the hardware is very robust and capable of needing very little to any humidity, temperature and filtration control. Intel’s own engineers say their chips are good up to 135 degrees, Centigrade (275F)! This study is available from Intel, titled “Reducing Data Center Cost with an Air Economizer”, August 2008.

I very much commend the good folks at Microsoft and Intel for completing and publishing these studies. So, why does ASHRAE still limit temperature and humidity so much more tightly than necessary? To add to this, ASHRAE recently released 90.1, a data center energy efficiency standard. Well, I’ve been touting and pushing for data center energy efficiency since the late 90’s, and I have made it a keystone of my career for about a decade, so of all people, I fully support further improving data center energy efficiency. This is why I chair the SVLG data center demonstration efficiency program (http://dce.svlg.org). But what ASHRAE has done via this standard is require air-economization, one technology to improving the mechanical efficiency of our data centers, essentially picking one technology. It would be akin to saying that all homes must have fiberglass insulation instead of saying walls and ceilings that meet a specific R-value, which is really what we want. Let technology and the commercial market compete for the best solutions, not standards that require one technology.

My concern with this standard is not that I don’t want data centers to air-economize to reduce energy use, my qualm is that air-economization is only one method to improving energy efficiency and often not the best. For example, I have been involved with the design of several data centers over the last two years, many with the fine folks at Rumsey Engineers (ww.rumseyengineers.com) and all have achieved annual PUEs of 1.04-1.08, significantly better than most data centers. In all of these low PUE data centers, we studied air-economization, heat wheels, direct and indirect evaporative cooling, chilled-water plants, geothermal exchange, and many other methods to cool the data center. In all cases, hot or cold climates, dry or humid, high elevation or low elevation, we found that air-economization was about twice as energy intensive as other cooling methods, and more expensive to build and often less reliable. So ASHRAE 90.1 could actually INCREASE energy use in our data centers (and first time cost as well) instead of the intended affect of reducing it. Why would a standards group specify a technology to achieve efficiency. Why are the temperature and humidity standards a decade or more out of date with technology? We need impetus to be more efficient, but ability to innovate and challenge each other to be as efficient as we can using a variety of technologies, old and new. Limitations to one technology are out of date the minute they are published in the data center world. With amazing new technologies being released daily, and a generation of servers lasting less than two years, a standards body that takes years to update should not be prescribing any specific technology and allowing greater flexibility in operations and design, enabling us operators, designers and manufacturers to adapt to the most-efficient resource.

I congratulate those who have come out before me about this new standard and ask with them, that ASHRAE revise their thinking once again and provide more flexible standards, not limiting ones, which only hurt our energy efficiency achievements. Thank you for reading.