Archive for December, 2010

Data Center Trends, 2010 and 2011

Wednesday, December 29th, 2010
I was recently asked what I felt were the most significant developments or trends for the data center industry in 2010 and what will be the most interesting trends in 2011.
For 2010, in my opinion, the most significant trend has been a constant focus on improving efficiency of the data center. and improving PUE. PUE has become fantastically successful with a constant focus and monitoring of the result and it is certainly now part of the industry vernacular. In order to improve PUE, data center operators have been exploring new methods of achieving availability to challenging the need for high reliability of every device. Concepts that I’ve been exploring and using for years like chiller-less designs, 100% water economization, UPS-less and even generator-less data centers are beginning to be considered by traditional high-reliability data center users in order to improve energy efficiency and reduce upfront and ongoing costs.
For 2011, total lowest cost of the entire data center from a holistic view will become common. Successor metrics to PUE  and new metrics that focus on the entire data center and everything within as a system will become the de facto tools used to monitor and drive efficiency and lower total costs by the entire operational chain and used by even CIOs and CFOs. Total energy use and cost per final work output will be the end-goal metrics. Moving above PUE to new metrics that encompass all aspects of the data centers, from operations, servers, storage and even network efficiency will be utilized, and a battle of the best metrics will ensue for years to come. By looking at these metrics and working with advanced thinking of the design of the data center and systems, companies will finally realize that improving energy efficiency actually reduces not just total cost but also up front capital costs. (All of the designs we work on cost less than industry average data centers, as building a more efficient data center, when approached holistically costs less than when approached by component or system.) Companies will look to building data centers as a complete system of IT equipment, software and infrastructure, not individual and discretely engineered systems.
These changes in focus on total lowest cost and folks implementing ideas that I’ve been advocating for years such as redundancy of data centers–building two or more lower-reliability data centers that achieve a higher net-availability than the one large high-reliability data center, and for a lower total cost–will be more commonly utilized to reduce total costs. These ideas and reviewing the reliability need by IT application instead of generically across all applications will be considered when building new data centers–especially when combined with high-reliablity utility service locations–to reduce total costs with mixed reliability data centers and those without UPSs, generators and chillers. We’ll also see more data centers that are “fan-less”, meaning using no fans in the data center other than a few air circulation fans and those within the hardware–a technique I’ve been employing in several variants for years to great success at reducing total energy use. These changes and others will shrink the average costs to build data centers from around $10 million per MW of IT where it is today to levels in the range I’ve been working on in the range of $3-7 million per MW of IT load. Remember: designing a holistically more efficient data center costs less to build. Save money up front and save money every month. It’s really quite simple and well worth designing your next data center “differently” than before.
In addition to the trend I was part of starting a decade ago of locating data centers with lower utility costs, and then adding in lower electricity use for a lower cost over time, the focus on total data center costs will result in the industry realizing that taxes and hardware costs can trump even these large costs. This will cause site selection of data centers to add into the decision matrix  business personal property taxes, sales taxes and income taxes for online revenue generation, as these will amount to more savings than energy cost and energy use. The decision matrix will get more complex with these new considerations including more importance on existing ones such as electrical grid reliability, carbon impacts, virtual and physical security risks and privacy laws. I believe that the one item that may be slightly reduced in consideration is network latency and reliability/redundancy as data centers for an organization will become geographically distributed instead of centrally located, improving overall end-user experience and total net uptime of the systems as they fail over between data centers. As more use of the cloud and mobile computing continues, the applications will be written to connect to the data center nearest to the end-user (distributed applications). This will make the data center location less important to network so long as latency is reasonable from every location of end-use, so the applications must be able to promptly support users anywhere in the world and as the users move around, further driving the trend to multiple and distributed data center locations on each continent to best support users and provide highest uptime and lowest cost.
With hardware, total costs per unit of work will become more common than upfront costs, driving many changes to efficiency, creating a revolution of hardware options to materialize over the next year. For one, servers will improve efficiency with ARM processors. All hardware, including storage and network will shut off components or entire systems when underutilized. Data de-duplciation will become common, as will desktop virtualization. Storage thru-put will continue to be an area of focus to improve, as will data center network energy efficiency and cost per unit of work. Some high-performance computing hardware will become more like many standard servers connected together in dozens or even hundreds of rack of commodity-like servers. Fans in hardware will use less energy and fans may go away in many pieces of hardware.
In what I have termed our recent progress as an energy efficiency revolution with data centers–mostly focused on reducing energy costs of the infrastructure–will evolve in 2011 to a total lowest cost of the IT operation, holistically monitoring and developing solutions that improve the total cost per unit of work encompassing the data center, servers, storage, software and networking while focusing on improving the end-users’ experience as they are mobile and rapidly evolving in expectation of connectedness and resource.
2011 will continue our energy efficiency evolution, hardware and applications will become more efficient, and taxes, power reliability and total energy cost will drive data center site selections to more new areas. We should see many advances in hardware design. It will be an exciting and interesting year in the data center and computing industries.

Total Cost of Energy (TCE) and true-TCO with all taxes–calculating all variables saves millions per year with data center site selections

Sunday, December 19th, 2010

I’ve been lecturing and advising on getting the data center site selection process right for many years, and as the old saying goes, sometimes you have to, “put your money where your mouth is.” As many of you know, I’ve been leading site-selection of data centers for over a decade, with the first 7 years mainly focused on the cost of electricity. And if we remove the hardware costs, electricity makes ups the largest single cost of building and operating a data center over a 10-year NPV. With electricity costs varying between about $.02/kWh to $.25/kWh with an average rate of about $.10/kWh, you can see that the TCO or NPV of a data center site selection can change dramatically with the total cost of electricity. I emphasize total cost of electricity, what I term as TCE, as many folks look at the utility rate per kWh not realizing that that is usually a cost for the energy or generation portion of the TCE, while the taxes, demand, transmission, distribution (as applicable) and facilities charges (utility facilities/capacity charge) and the actual use of electricity add into my TCE. Since demand charges vary dramatically by utility and facilities charges vary by site and total connection requirement, the actual rate can vary from the utility’s rate tariff. So looking at the rate tariff is not an accurate estimate of the total rate charged per kWh. Then add in the total USE of electricity—this being the total demand and usage over time, usually I use ten years to determine TCE. Total usage will vary by ambient conditions. Hot climates use more electricity than sites in cooler climates and sites with heat rejection to water sources use less than those with only heat rejection to air.

One should then add in the expected rate increase over a ten-year period, taking into account known rate increases, past rate increases an their frequency, as well as expected future emissions costs (i.e. coal will be more expensive in the future than other sources) and regulatory changes. In all of my site selection reviews, I take all of these items into account, providing a true current and expected Total Cost of Electricity, and once we do, most often sites with lower costs per kWh come out with a higher TCE—so a higher total cost than the seemingly higher unit cost sites and often by a large margin. This brings me to a point that the true and forward cost of electricity (TCE) is a driver of costs and rarely done completely as described above, but even when doing so, taxes often prevail as the decision between one site and another, not TCE.

So what comes next is something that affects me personally as well as being an important topic for discussion: the evolving impact of taxes on data center TCO. TCO – total cost of ownership – is an important stepping off point. When we use the term who exactly are we referring to as the ‘owners’ of a data center? The data center operators? The corporate IT group? The real estate department? The true answer is all of the above and then some – the whole ‘mother ship’ corporation. It’s the business that owns the data center and the entire IT operation and all of its assets are in place to support the execution of the top-level corporate business model. As McKinsey & Company recently explained, the IT operation and its assets must be in full alignment with executing and supporting that business model. This may seem blatantly obvious to some. (“Why business needs should shape IT architecture,” McKinsey Quarterly, April 2010)

Business models are adapting rapidly to changing economic, competitive and technical environments. A glaring case in point is the rapid emergence of cloud services as it impacts service providers and clients alike. Where before, data centers could correctly be viewed as high performance operations processing information in support of a company’s profit making activities, the growth in hosting services (IaaS, SaaS, etc), the expanding consumer interest in digital downloads (think NetFlix and iTunes), and the explosion in the number of mobile platforms demanding real time attention (downloads, uploads, services, etc) means that more and more of what is going in the data center is the company’s profit making activity!

Why is this important to data centers in general and site selection decisions? Because cash-strapped states around the country are putting electronic transactions in their ‘cross hairs’ and are already pulling the trigger to collect taxes they believe are due. As Ernst and & Young recently advised its clients,
“You need to keep in mind that the location of a datacenter can trigger tax implications. If you place your datacenter in a different state, or in a foreign country, the facility and quite probably the transactions which run through the facility will be subject to the tax laws of that jurisdiction.” ( “The transactions which run through the facility,” can and will be interpreted as every minute of SaaS, every download of a music file of movie, and every hosting service a customer pays for. If a state attorney general can make the case that the transaction occurred in a data center in their jurisdiction, they will make the case that the revenue was realized in the state and is subject to any state taxes that apply. That means not only state sales taxes (which the customer pays, but which can make one company’s cloud offerings less attractive than another out-of-state competitor’s), but also corporate income taxes, which directly affect your company’s bottom line. Think this doesn’t matter or won’t surface as a problem. Think again.

Texas recently told Rackspace that all of its hosting operations are subject to the state’s margins tax (Yes Virginia & Texas have a corporate income tax. They just call it something else). “The issue is whether a company is taxed where its data centers are located or where its customers are. In Rackspace’s case, 90 percent of its customers are out of state, but the state applies its business tax to its data centers in San Antonio and Grapevine. “That puts us at a competitive disadvantage” with data centers outside Texas that don’t pay a margins tax, Rackspace lawyer Alan Schoenbaum testified.” (“Rackspace: Business tax is pushing cloud computing firms out of Texas,” AMERICAN-STATESMAN, 8 July, 2010)

So Texas is taking the position that the revenue generating transactions that take place within a data center are going to be taxed at the data center, regardless of where the client is located. Further, as Texas told Amazon when it handed them a $250 million plus tax bill, if you have a data center in our state, Internet sales you conduct with clients in Texas will be recognized as having occurred in those data centers and taxed accordingly. Other states are watching. (I’m writing this issue with a disclaimer. As many of you already know, I’m a principal in a data center campus development 8 miles east of Reno, NV).

So what does this imply for cloud computing business models and what can be done to mitigate the effects? Let’s look at a plausible example. Take a ‘hypothetical’ company selling music and applications on line to customers who have purchased its mobile communications platforms. Those transactions are all being processed in a data center somewhere. Let’s say that company makes a profit on those Internet sales and that the company is interested in minimizing the tax impacts on that profit. If that company recognizes that revenue in California, it is subject to an 8.84% state corporate income tax. In Virginia, it would be 6%, and in Iowa, 12% on annual revenues over $250,000. In Nevada, it is 0%. For every $1 billion in annual revenue, this is a nearly $90 million ANNUAL savings compared to California! Enough money saved that I can build an 18 MW of IT load data center every year on just the tax savings alone!
The clear lesson coming from the Rackspace and Amazon standoff with Texas is that the states are looking to see if a company has a data center within their borders. If it does, then the revenue will be taxed. (The Tax Foundation (

The numbers are more than significant. Rackspace just reported Q3 net revenues of $199.7 million. That’s a $1.99 million QUARTERLY tax bill as far as Texas is concerned. Lucky they aren’t based in California, where the tax bill would be $17.65 million per quarter. Up to now, only sales and use taxes and business personal property taxes factored into data center site selection decisions. With the cloud, a new issue must be considered. Think of companies migrating much larger offerings on to the cloud, with revenues measured in billions of dollars. The state income tax implications can dwarf other metrics. Microsoft is an example of just how big the numbers can get. The company moved its licensing operation out of Washington State to avoid an onerous state income tax levy. They set up shop in a state that has no corporate income tax (there are only 3). Microsoft moves over half of its global licensing revenue through that Nevada office and data center–over $40 billion annually. They pay $0 state corporate income tax. All legal and allowed by the IRS, and they are not alone. Cisco, Oracle, Intuit and many others also run similar operations within a few miles of each other. A company doing 1/10th that amount located in California would be liable for over $300 million in state income taxes every year following the Texas approach. I can build about a 50 MW of IT load data center for that kind of money. You can look up the other two states that also have no corporate income taxes, but I’ll wager you’ll like the Lake Tahoe region better.

In a recent analysis for a company looking at Oregon with its 0% sales tax, we found that the Business and Personal Property taxes in those Oregonian towns far outweighed the sales tax savings when compared with a 2% sales tax and a 0% Business and Personal Property tax in Nevada. Even the electricity rate, TCE, was much lower in Nevada, netting a significant TCO savings compared to several eastern Oregon sites. Already a much lower TCO before adding in the potential massive income tax savings.

So what’s the bottom line here? Calculate TCE and TCO considering all tax (income, sales and property) and energy variables (energy, demand, transmission, facilities/capacity charges, future costs).  A 0% state corporate income tax rate is going to assume more and more importance in very short order as the cloud becomes a bigger factor in businesses strategies across the board. I’ll gladly do a quick analysis for your business to help with your site selection. Save millions more than a data center in Oregon, Idaho, Washington, Utah, even North Carolina, and let me help you be a corporate hero.