HDS Blogs

As noted in previous posts, capacity efficiency has two dimensions: allocation efficiency and utilization efficiency.

Allocation Efficiency

Allocation efficiency is what most people think of first: eliminating the waste of over-allocation. In open systems this has been a major problem since we may not know ahead of time how much capacity an application requires, but we don’t want to run out of capacity and we know the operational difficulties of expanding it. So, the usual practice is to over-allocate by a wide margin.

After all, disk is cheap, isn’t it?

The problem with over-allocation is that that we don’t just make one copy. As Claus Mikkelsen noted in his blog, there may be 10 to 15 copies of that allocation for many valid requirements, like data analysis, data sharing, development test, or point in time snap shots. The most efficient way to eliminate this over-allocation is to use thin provisioning, where you provide virtual space for the requested allocation and only provision the capacity that is actually being used. It also helps to support the APIs for file systems, like VMFS and Symantec file systems that can notify the storage system when files are deleted so that the allocation for those files can be reclaimed by the storage system.

The capacity and time to make copies is also reduced by the elimination of allocated unused space. Reduction of copies can also be reduced further with copy on write so that only the new changes are replicated. HDS storage supports these functions, and can map them to legacy storage systems through virtualization.

The thin provisioning software that Hitachi provides in Hitachi Dynamic Provisioning (HDP) also increases allocation efficiency by providing a pool of preformatted pages. This eliminates the need for the storage administrator to format the drives, carve out the LUNs, and concatenate LUNs for striping performance. HDP will automatically strip a LUN across the width of the HDP pool. Allocation software in our Hitachi Command Suite will enable a user to allocate storage with five clicks of a mouse.

Utilization Efficiency

Utilization Efficiency is about using the capacity in an efficient manner so as to reduce costs and increase performance/availability. The primary Hitachi tool for doing this is Hitachi Dynamic Tiering, where we can dynamically tier pages within a volume across multiple tiers of cost/performance storage capacity. When most people think of tiering they think of volume level tiering, where a volume is moved between tiers of cost/performance storage. While this can match volumes with the right performance tier of storage, it can actually use more storage since you need to have space for the whole volume in all the tiers that are involved. With page level tiering, only the hot pages need to reside on the higher performance tiers. Since only a small amount of pages are hot at any time, you will only need enough high performance for 5% to 10% of your volume rather than for full 100% of your volume. That is utilization efficiency.

Utilization efficiency also depends on the efficiency of the paging process. Paging is the most efficient method of dynamic tiering since it is calculated on a page basis. Chunk/Chunklet methods for paging require the definition of a chunk and then an index into the chunklet. Dynamic tiering requires the handling of more metadata and more processing power within the storage system. VSP was designed with a separate control store for the metadata and a separate pool of Intel quad core processors to offload this processing from the I/O processors.

This function can also be mapped to external storage through storage virtualization through VSP.

These are the primary tools for storage capacity and utilization efficiencies in Hitachi storage systems. I would be interested in hearing about other functions that can be used to enhance capacity efficiencies.

This is part two of a three-part blog series on strategies and tactics during an economic crisis. You can read part one here.

We have already covered what makes a crisis (since your mileage and conditions will vary), so this entry will focus on CAPEX pressures, and how traditional capital will (probably) be replaced with contract/consumption pricing over the next half-decade. The raw cost of procurement alone will not drive our industry to this new acquisition model, but when combined with current and future economic conditions, it will set the stage for new consumption models (i.e. cloud computing).

During an economic crisis, there are some behaviors that can be observed. How many of these apply to your IT department?

• Most of the world is in the middle of a recession, and recoveries tend to be a long, slow process. CEOs and CFOs respond to an economic crisis by building up their cash reserves. See this article on what Apple (for example) could do with their cash reserves.
• When hoarding cash, companies are reluctant to also commit monies to long-term IT investments, and are more open to operating leases, and consumption models to finance new growth and expansion.
• Company leadership and the CFO will slow down or severely restrict capital purchases.
• Operational costs will be reviewed with great scrutiny, and labor costs can be one of the first areas targeted.
• When purchasing is allowed, procurement groups tend to go for the lowest price to maximize the limited funding, and often this can be without understanding the downstream impacts to cost, quality, reliability etc.

Re-stating this strategy in poetic or chiasmus prose, the approach looks something like this….

My next and final entry in this series will discuss technology and attitudinal chances that can be adopted to reclaim what you have, and improve IT efficiencies.

If you Google storage efficiencies, eventually you will get a Wikipedia definition, which describes storage efficiency as “the ability to store and manage data that consumes the least amount of space with little or no impact on performance, resulting in a lower total operational cost.”  Wikipedia also references the SNIA definition, which notes:

storage efficiency = (effective capacity + free capacity)/raw capacity

However, as I noted in my last post, the definition of storage efficiency has expanded beyond capacity.

Since that last post on storage efficiency being more than capacity, I have discovered more tweets and blogs around this topic. Randy Kerns posted this week at IT Knowledge Exchange on storage efficiencies and data center optimization.

Randy identifies storage efficiencies around data reduction, allocation of capacity reduction, performance efficiency, data protection efficiency, scalability efficiency, and increasing automation for administrative efficiency. This matches very well to the list of efficiencies that Jon Toigo provided in the blog that I recently referenced.  One new twist may be the scalability efficiency, in which scaling of capacity and performance is done in equal proportion to support greater consolidation and growth. I think he makes a very good point, so I am adding it to my list of storage efficiencies.

Expanding on Jon Toigo’s base list of efficiencies, I would add Randy’s contribution along with my “storage management efficiency”:

• Capacity Allocation Efficiency
• Capacity Utilization Efficiency
• Performance Efficiency
• Data Protection Efficiency
• Energy Efficiency
• Storage Management Efficiency
• Scalability Efficiency

Does anyone have other efficiency consideration for storage? In my next few blogs I plan to expand on each of these bullets.

I hope you survived the solar flare-up the other day. I did not get my usual number of cell phone calls on Tuesday, so I am blaming the sun. I am convinced that if Chicken Little is ever proven right, it will be because of space junk falling into the earth’s atmosphere (after being melted by solar flares). I did notice some space debris over the weekend, but that could have been spent bottle-rockets from the Chinese New Year (btw happy year of the dragon!).

There is a lot of strange news, frightening news, and pretty funny (given economist humor) stuff out there. Let’s take a look at some of the global economic trends that are happening now:

• Currency de-valuation in India, Vietnam (and forecast for some in Europe), and the impact to purchasing power
• Government debt downgrade
• Austerity programs and all the trickle-down results to government and private sector spending
• Rise in HDD – due in part to natural disasters and other calamities
• Bankruptcy from notable companies like Kodak and American Airlines

One could certainly make the case that the sky is falling. Capital funding is being preserved, even from companies that are flush with cash. What impact of capital spending do you see locally in 2012? Gartner has revised their IT outlook,down to 4% growth with year on year IT purchases. Capital may be tighter, but OPEX is still (and probably always will be) under pressure. Our methods around storage, cloud, VM and VDI economics are very popular with IT planners and executives to identify, measure and reduce the cost of IT infrastructure.

The strange part of any economic crisis is that data and workload growth does not take a holiday. Upward trends continue putting more pressure on CAPEX and OPEX. Storage, servers (VM) and mobile computing (VDI) seem to continue a steep growth curve. New CAPEX, OPEX and procurement ideas need to be factored in to accommodating growth while in the middle of an economic crisis.

I am developing several internal papers on how to position growth options while in the middle of current (or future unknown) economic uncertainties. I will be sharing some of this content in my next 2 blog entries:

• Part 2 will talk about CAPEX pressures, and how CAPEX will (probably) be replaced with contract/consumption pricing over the next half-decade.
• Part 3 will outline storage and server architecture strategies/technologies that can help stretch your limited capital budget, outline work-arounds when your local currency does not buy what it used to, and how to really measure, track and improve real costs. This is often referred to as capacity efficiencies, but goes way beyond reclaiming a few GB of storage.

I do not want to be accused of being an ambulance-chaser, or even a chicken-little, but we have to have a sense of our surroundings (government, social, economic, political) as a backdrop to IT spending, and IT architecture decisions. These next few years may cause some hardship if traditional methods and architectures continue without challenge. IT planners need to step back and have a strong strategy to meet IT business demands, while working within (possibly) new financial restraints.

Oh, and I joined Twitter. Follow @StoragEcon for all of your storage economist needs!

Guest post by Tracey Doyle

After a relaxing break for the holidays, Storage Visions 2012 was a great way to ease back into the swing of things. Sure, many people might not look at a whirlwind trip to a Las Vegas-based conference kicking off CES (the world’s largest consumer technology show of the year) “easing back in,” but well…I do.

SV2012 starts a few days before CES, so there is somewhat of a quiet before the storm. I love the feel of Storage Visions; it has an intimate feel even though the attendance continues to grow each year. This is due to Tom Coughlin and his team. They run the conference like a community. I see many familiar faces each year and there is a heavy emphasis on networking with your peers. Networking is made easy with the laid back atmosphere at the conference, and with all the friendly exchanges, it is very easy to meet new contacts and get excited about what you do. Our HDS cloud vision really seemed to resonate with the people I talked with.

I was able to present during the session “They’re Out There: Opportunities and Challenges for Consumer and Enterprise Cloud Storage”. What a fun panel session it was. It included a variety of presenters (even a competitor), but it was such a lively interactive exchange that it made for an interesting and informative discussion. The session focused on distributing content, as well as how on-line back-up and disaster recovery are driving demand for remote storage. We also addressed storage requirements and trends for online content delivery and remote storage. We covered some new business opportunities and how they’ll impact the growth and use of storage in this growing market.

Another thing that made SV 2012 a bigger and better event this year, for me and HDS: Hitachi Content Platform (HCP)was honored with a Visionary Product Award at the 2012. HDS was recognized in the Enabling Professional Storage Technology category for the benefits HCP brings to organizations, including simplified IT, reduced costs and reduced risks. I swear I thought I was accepting an Oscar! A little too excited maybe? Oh well, a little too much excitement won’t kill anyone.

That’s how I started off my New Year . I know it will be an exciting year here at HDS. I look forward to spreading the HDS cloud vision and continuing to share the many exciting things we have going on!

Follow me on twitter @tdoyle49

Hu and others have recently been blogging about storage efficiencies, in terms of capacity, management, allocation, data protection and energy efficiency. I will add another dimension to this discussion, and that is the methods in which efficiencies can be measured.

Quite often we talk about an efficiency theme, and jump right into the solution or options to improve that efficiency. In my work, I tend to recommend that organizations:

• Identify the measurement system clearly
• Measure the area that is to be optimized
• Improve with specific actions and plans

The recent blogs talk a lot about the step to identify and improve. For example: you can impact storage utilization efficiency with virtualization, thin provisioning, reclamation activities and de-duplication. One can improve energy efficiencies with virtual machines, thin volumes, SSD, storage tiering and an aggressive archive program. As I mentioned, the identify and improve aspects go hand in hand, but we often forget to take the intermediate step to measure (before, during and after) our efficiency program.

A wise colleague shared this idiom with me 15 years ago – “you cannot improve what you cannot measure”. This expression has been core to our storage economics framework from the beginning.

Measuring efficiencies can sometimes be difficult and subjective. That is why financial or economic measurements are most effective. Turn the efficiency or inefficiency into money and measure the costs (or unit costs). The simple table below outlines a series of efficiency options, potential measurements and then the options that can improve the results. If measurements are taken before and after the effort, we can be sure that we have actually achieved a measurable improvement.

This is a simple outline, but you can see how the intermediate step of measurement is key to ensuring that the right actions are taken to produce the efficiencies required. Many differences may exist as to the best approach to achieve improvement, but an organization should have a singular focus on the measurement systems to align them with local goals and objectives.

Here are more posts on capacity efficiency:

• A Consensus on Storage Efficiencies
• How Much Does it Cost to Spend Money?
• The Tipping Point for Hard Disk Prices?
• Buying Disks or Buying Storage Efficiencies
• Look Beyond the Price of HDD
• Capacity Efficiencies, Again and Again
• How Thin Provisioning Contributes to Storage Efficiency

In response to my last blog, Jon Toigo was kind enough to post a training piece that he wrote last year, reminding us that capacity is only one part of storage efficiency.

In addition to capacity allocation efficiency, which most of us are addressing with thin provisioning, Jon points out the need for capacity utilization efficiency, storage performance efficiency, data protection efficiency, and storage energy efficiency. Expanding on his thoughts, I have added storage management efficiency.

Capacity utilization efficiency is about the placement of data on the appropriate tier of storage, based on frequency of access, business value and cost of the storage. This could be addressed by automated tiering based on policies that are triggered by time or events.

Storage performance efficiency could be addressed by automated wide striping or page level tiering, where only the hot pages of a volume—rather than the whole volume—is moved to high performance tiers of storage.  Ray Luccesi has a great take on storage performance efficiency in his IOPs vs Drive Counts chart of the month, which he posted last week.

Data protection efficiency, which is measured in terms of Recovery Time and Recovery Point objectives (RPO/RTO), is a major area for improving efficiency. This has to do with replication, backup, recovery, archive, etc. If most of the data is static data, which is not being updated like most unstructured data, you only need two copies for redundancy. You can eliminate making the many snapshots and backups of the same unchanged data over and over again.  Brad Clarke commented on my post that the most important storage efficiencies to him were the ones which make replication less bandwidth hungry. He makes the point that when data volumes increase, the cost of disks to contain that capacity is relatively cheap—compared to the cost of the increase in bandwidth pipes that is required to replicate it.

Energy efficiency will be a big focus this year based on the record increase in carbon emissions in 2011 (5.9% increase) that was reported by The New York Times. Another factor is the nuclear problems at Fukushima, which has sifted demand from nuclear power to carbon fuels, and is raising the cost of energy, as well as the possibility of carbon taxes on top of the energy bill. Since storage is becoming a greater percent of the power consumption in the data center, storage energy efficiency is becoming a key consideration for buying decisions. Storage energy efficiency benefits from the other efficiencies cited above, but there are also efficiencies of 40% or more with storage systems like VSP, which use Small Form Factor SAS drives, dense drawers, front to back cooling, and the replacement of batteries with flash for protection of the cache.

Another efficiency that comes to mind is Storage management efficiency: the ability to manage heterogeneous storage arrays as a pool of common resources with a common set of tools, so that resources like capacity and performance can be shared rather than isolated in silos.

Are there other areas of storage efficiency that we should be considering?

Here are more posts on capacity efficiency:

• A Consensus on Storage Efficiencies
• How Much Does it Cost to Spend Money?
• The Tipping Point for Hard Disk Prices?
• Buying Disks or Buying Storage Efficiencies
• Look Beyond the Price of HDD
• Capacity Efficiencies, Again and Again
• How Thin Provisioning Contributes to Storage Efficiency

The greatest tool for storage efficiency is storage virtualization, which enables the extension of other storage efficiency tools like tiering and thin provisioning to existing storage systems that do not have that capability. It also reduces operational costs by providing a common pool of dynamic shared resources under a common set of management tools.

In view of Gartner’s recent prediction of a 5% to 20% increase in disk prices and shortages due to last year’s floods in Thailand, Hitachi Data Systems will extend and enhance the Switch It On III promotion that reduces the cost of virtualizing and managing heterogeneous storage on a Virtual Storage Platform. This promotion is targeted to helping customers increase utilization and reclaim capacity on existing third party storage in the face of raising disk prices and shortages. Customers who take advantage of this promotion will be able to reduce storage CAPEX and OPEX and increase their return on assets.

Photo by Kevin Kevin Pelletier

The Switch It On III promotion, which was due to expire at the end of March has been extended to June 30, 2012. The promotion reduces the licensing and maintenance costs for virtualizing heterogeneous storage systems. It also includes price reductions on thin provisioning, dynamic tiering, disaster recovery, and in-system replication software, as well as management tools for tuning and Command Director that provides an end-to-end application to infrastructure view of utilization and service level objectives.

This promotion is available on new purchases of VSP and existing VSP systems that have not already virtualized external storage.  Hitachi storage virtualization enables customers to fully utilize the resources that they already have in their multivendor environment. It creates a single pool of heterogeneous storage capacity that they can control and optimize with a powerful suite of management tools. It lets them take advantage of cost saving features like thin provisioning and data mobility with their existing storage systems and provides them with a sustainable storage architecture that that can protect and grow into the future.

For specific details on this promotion contact your Hitachi Data Systems representative or reseller. Also, check out more details on Switch It On III.

Also, here are some other blog posts on storage efficiencies:

• Look Beyond the Price of HDD
• Buying Disks or Buying Storage Efficiencies
• Capacity Efficiencies, Again and Again

Virtualization in the data center is a stable and proven approach to make IT more efficient from desktops to servers and from networks to storage. Whether storage virtualization is host-based, controller-based or through an appliance, it is a core ingredient in economical IT architectures. As in most new technology investments, you need a clear understanding of the benefits versus the costs. When you can project a positive ROI and fast payback, your projects gain more traction with IT management. Storage virtualization is a critical element for any organization that wants to significantly reduce unit costs in 2012.

Does storage virtualization make sense for you and your storage environment now? Are there economic benefits for you with this technology? What benefits and cost metrics do you need to build your own business case for Storage Economics? Are you aware of the compound advantages that storage and server virtualization make on your data center operations and costs?

Join me, Hitachi Data Systems chief economist and global business consultant to understand:

• Types of costs that virtualization can directly address (and reduce)
• Qualitative benefits of virtualization and how to convert them to cost savings
• Quantitative methods to measure and predict cost savings of virtualization in data migration, space reclamation, storage management tools, storage management effort and consolidations
• Qualitative impact of combining server, desktop and storage virtualization
• Cost savings by large and small IT organizations around the world through virtualization of their storage infrastructures

Speakers: David Merrill, Chief Economist

Feb 1, 9am, 12pm ET

To register for this webinar, click here, or visit: https://hdschannel.webex.com/hdschannel/onstage/g.php?t=a&d=928422534

Hu and I have blogged about rising disk prices, and how virtualization can be a key instrument to hold down costs:

• Buying Disks or Buying Storage Efficiencies
• Look Beyond the Price of HDD
• Supply-side Economics
• Storage Virtualization – A Foundation for Economically Superior Storage Architectures

To learn more, please see below for details about a webinar on this topic tomorrow, Wednesday January 25, 2012.

Webinar: Manage Rising Disk Prices with Storage Virtualization

Wednesday, January 25, 2012 9:00 am Pacific Standard Time (San Francisco)

Technical Session: Learn how storage virtualization can reclaim existing storage on the floor. Extend thin provisioning to existing storage to increase disk utilization and defer capital purchases. Take advantage of zero page reclaim and write same to reclaim storage reclamation. Use Hitachi Dynamic Provisioning over provisioning capability for automatic optimum storage utilization.

We’ll also cover dramatic enhancements to Hitachi Switch IT On III program that make this extremely attractive and affordable. Register for this WebTech and learn from HDS experts how to use these technologies to increase your customer satisfaction and sales despite impending increases in disk prices.

This webinar will also cover:

• Thin provisioning options for storage reclamation and expectations that depend on customer environment
• A “how to” on reclaiming storage from existing systems using Storage Virtualization
• Details of Hitachi Switch IT On III enhanced program

Target Audience: Hitachi Data Systems Customers, TrueNorth Partners, and Employees

Register by either clicking or pasting the link into your Web browser:

https://hdschannel.webex.com/hdschannel/onstage/g.php?t=a&d=928626897

Since I posted my trends for 2012, I have been looking at what other bloggers have been predicting.

The most common theme is the explosion of data, and the need for storage efficiencies. Jon Toigo says that 70% of the capacity on every disk today doesn’t need to be there–40% should be archived and the other 30% should be reviewed and probably deleted.

Joe Kovar of CRN predicts that growth in storage capacities decline as users implement more storage efficiencytechnologies like thin provisioning, deduplication, and cloud. An optimistic prediction is that the impact of the Thailand floods on disk shortages will wane sooner than expected, and the expectation is that disk shortages will wane by the second half of this year. I agree that the impact may wane sooner as users implement storage efficiencies. However, I believe there will be a fundamental change in the pricing of storage capacity, as I posted last week.

David Chapa believes storage is becoming more and more affordable to the masses, through the adoption of small business cloud services. He recognizes that home office users now have several terabytes of data stored locally and the increasing costs of managing that data, similar to the enterprise. While we have seen consumer prices of disks more than double in the last two quarters—$79/TB to $190/TB was quoted at a recent Gartner conference—the total cost of managing storage far exceeds the cost of acquisition. Cloud services can reduce the cost of this management and make the total cost of storage more affordable, even though the cost of the disk may increase. See what David Merrill says about procurement costs.

What do you think about storage efficiencies and the impact of disk prices on storage costs?

Here are more posts on capacity efficiency:

• How Much Does it Cost to Spend Money?
• The Tipping Point for Hard Disk Prices?
• Buying Disks or Buying Storage Efficiencies
• Look Beyond the Price of HDD
• Capacity Efficiencies, Again and Again
• How Thin Provisioning Contributes to Storage Efficiency

Taking some comic relief from normal blog posting, and instead, sharing some of my favorite Dilbert storage (and economics) cartoons:

Source: http://dilbert.com/strips/comic/2004-10-14/

Source: http://dilbert.com/strips/comic/2002-09-27/

Source: http://dilbert.com/strips/comic/2002-06-08/

I have been working for HDS for almost 15 years, so, needless to say, I am very proud that we have been recognized as one of FORTUNE magazine’s 100 best companies to work for in the United States in 2012. This recognition helps to validate one of our stated company goals, which is to be the employer and partner of choice.

FORTUNE worked with Great Place to Work Institute to conduct the most extensive employee survey in Corporate America. While this survey applies to the United States, we have similar results across Hitachi Data Systems globally. The Great Place to Work Institute ranked HDS #5 in Poland and #13 in France.  In Silicon Valley—one of the most exciting places to work in technology—we ranked #3 and placed in the top 10 for the last three consecutive years!

How do you define a great workplace? Great Place to Work Institute, who has been doing research on this for over 25 years, has this to say:

“Trust is the defining principle of great workplaces – created through management’s credibility, the respect with which employees feel they are treated, and the extent to which employees expect to be treated fairly. The degree of pride and levels of authentic connection and camaraderie employees feel with one are additional essential components.”

HDS has an entrepreneurial culture where we are constantly improving and innovating. At the foundation of our innovation is trust, an integral part of the Hitachi culture (Hitachi Spirit) that we have inherited from our parent Company Hitachi, Ltd:

Hitachi Spirit is what distinguishes us as the employer and partner of choice. It’s more than our foundation, more than a poster on the wall. It’s how we operate every day, how we get things done. It’s who we are.

See what our employees are saying on the top workplaces website.

One of the expenses included in storage and IT economics is the cost of procurement. I discovered and documented this cost element about a year ago while working in Australia.

Anyway, I believe that traditional capitalization for IT will diminish over time, and we will move to contract or consumption-based contacts to secure IT resources in the future. I can image an advanced version of an app-store and facebook where I can ‘like’ a storage consumption model, then download it to my infrastructure to upload with my internal apps and systems.

In the meantime, we are left with more traditional procurement models that require a set of methods and rigor to get assets approved, justified, certified and priced correctly. The entire procurement process can take months and significant effort, so IT planners have become experts in forward-planning to meet capacity demands. There are many options to reduce the time and cost of procurement, but I wanted to start with some simple metrics that may helpyou get a jump start on efforts to reduce this time and cost of procurement.

The first step is to dollarize the cost of procurement. Easier said than done.

Last March, Forrester released a paper covering eProcurement software and the impact this can have on the entire process. I found this to be an insightful read on the software automation side of the equation. Then a colleague sent me some information on the efficiencies of procurement personnel, and that the average procurement employee spends US $25M per year. And those employees burdened cost, on average in the US, is $112K.

Simple math then helps us to create a simple metric: to spend a million dollars on IT assets will cost (in labor) about$4,500. Now there are other aspects of procurement that will not be in this number (such as RFI, ITT generation ad review, treasury funds, certifications) but assigning a rough cost of $5K to acquire $1M is an interesting start. This procurement cost can be seen as a tariff of 0.5% (.0005%) of all IT spending. Perhaps this is the kind of metric that can be used to start measuring the cost of spending money.

Q1 2012 marks a major turning point in the storage industry. After 50 years of price declines in the magnetic disk industry, we are seeing what most analysts predict to be a 5% to 20% increase in disk prices due to the catastrophic floods in Thailand, which has had a major impact on the disk supply chain. While the manufacturers are hoping to get their capacity back on line to ease the supply shortages by the second half of this year, the cost of rebuilding their manufacturing capabilities will impact prices for some time. This additional cost will also impact the investments required to deliver next generation higher capacity disk technologies, like Bit Patterned Media or Heat Assisted Magnetic Recording, which are on the roadmap for disk drives.

Above is a chart showing disk drive capacity increases going back to 1980, which is truly phenomenal. Due to this rate of capacity increase, the industry has enjoyed an annual price erosion of about 20% to 30% per year on disk media. However, you can see that the density curve is starting to slow down as we approach the limits of current perpendicular recording technologies.

With this historic price erosion, most data centers depreciate their enterprise storage over three years while midrange storage is typically depreciated over five years considering it has been cheaper to buy new than to maintain the old after that time frame. If this price erosion starts to slow down, data centers may need to extend their depreciation to seven years. By this time, the disks will be in the five to ten TB range, so keeping the media longer may not be a bad idea.

However, there is a lot more technology that goes into storage systems than the disk technology and the rate of that technology has been increasing rapidly with thin provisioning, data mobility, tiering, replication, and closer integration with the application layer through APIs, plugins, client/providers, adapters, and snap-ins. That means that a five to seven year life cycle for storage systems will make your storage system non-competitive. The reason why enterprise storage is capitalized over a shorter period than lower cost modular storage is because of the higher technology cycle of enterprise storage.

An approach to solving this is to separate the disk capacity from the enterprise storage system controllers, so that you can keep the storage system controllers current with systems technology on a three year cycle, while you refresh the disk capacity on a five to seven year cycle. Since the storage media is still the bulk of the cost of a storage system, the longer depreciation cycle will help to reduce the capital costs. You can do this with an enterprise storage control unit, which also has the capability to virtualize external storage systems. This is what we provide with VSP.

What are your thoughts? Are the price increases that we expect this quarter just an anomaly and will we go back to enjoying the price erosion that we have enjoyed for the last 50 years? Or has this changed forever?

Is this a tipping point in the way we capitalize storage assets?

At the top of my list of trends to watch in 2012 was an increased focus on storage efficiency due to economic uncertainty and hard disk supply shortages—stemming from last year’s floods in Thailand. Yesterday IDC and Gartner both reported declines in 4th quarter PC shipment of 1.4 to 0.2%, compared to 2010 that was partly due to disk drive shortages. (My colleague David Merrill also covered this in a recent post.)

The shortages have been felt the most in the consumer markets. At the Gartner Data Center Conference in Las Vegas, a speaker cited the costs of a TB disk at the U.S. retailer Fry’s had gone from $79 to $190. Last November, the Storage Architect reported that a 2TB SATA drive that he had bought before for £65 was then listed at £150 on Amazon. Consumer markets run on very low margins, so the price can increase dramatically in response to any shortages.

In the enterprise space, the shortages have been real but the prices have been more stable. Some drive types have increased on the order of 5-15%. Consult your vendor to see what the current status is. Hopefully we will see the supply situation return to normal by the end of 2Q.

The Storage Architect supports my view of focusing on storage efficiency during this period by using thin provisioning and other efficiency methods in his post Drive Prices Increase – Who Will Suffer Most?

“If your vendor doesn’t offer it, then there are plenty out there who do.  As prices rise, it may be time to look again at implementing these features and fixing the processes that stop you using them today.”

Investing in storage virtualization through VSP adds another dimension to storage efficiency by extending these new capabilities to existing storage systems. If your current storage system does not provide thin provisioning, which can reclaim 40% or more of allocated but unused capacity, you don’t need to rip and replace it. Just by attaching it behind VSP, VSP can see your existing LUNs, and move them into a dynamic provisioning pool where the unused pages in the LUN can be reclaimed (Zero Page Reclaim) while your application is running.

So instead of buying additional disks during this period of shortages, invest in storage virtualization with VSP, which not only frees up the capacity you need today from your existing storage assets, but positions you for sustainable growth into the future. See what Claus Mikkelsen and David Merrill have to say about storage efficiencies.

You may have heard news about the devastating floods in Thailand, and now the ripple effect of increased HDD prices due to mfg and supply shortages. HDS, EMC, NetApp, and IBM have all stated temporary price increases of 5-15% for the next few months. It is at these times that we need to step back and differentiate price and costs, and to look at storage architectures (not arrays or HDD) that produce the lower total cost of ownership.

For years, HDS economic consultants have modeled total costs for customers, thousands of customers, in vertical markets in all parts of the world. We know that price of the hardware and software is just two of the 34 elements that can constitute the total cost. And although most organizations are sensitive to the purchase price, we have seen empirical data that suggests that price (lease or depreciation expense) is just 17-20% of the multi-year total cost of storage ownership. Many other costs (such as labor, power/cooling or migration) can often eclipse the purchase price when looking at four year costs.

So a 5-15% increase in the price of arrays/HDD then translates to a potential 1-3% net increase in total cost:

• At the low end, a 5% rise in HDD price impacting a TCO model where price is 17% of the TCO produces <1% increase in Total Cost (.05 x 17%)
• At the higher end, a 15% rise in price for an environment where HW is 20% of the TCO creates a 3% rise in TCO (.15 x 20%)

This 1-3% increase in cost is not as devastating a number compared to the bad news of a catastrophe-driven price increase.

With the right storage architectures though, total costs can be driven down by reducing the storage footprint, total power consumption, faster (or no) migration time and effort, and reduced wasted capacity. Economically superior storage architectures tend to have some common ingredients to produce these kinds of results:

• Storage Virtualization can significantly reduce migration time, reclaim white space, provide better/unified management and reduce license and maintenance costs for the subordinate (heterogeneous) storage arrays
• Thin provisioning can reduce wasted, allocated capacity and provide improved performance with wide stripping
• Data de-duplication can reduce total disk space requirements
• Dynamic tiering and auto-tiering can put data in the right cost of tier, based on rules setup around access, retention or QoS
• Intelligent archive solutions can cut down backup costs, and move stale data to a lower cost tier for retention or immutability requirements
• Control user behavior or appetites with service catalogs, charge-back/show-back reports and basic consumption metrics

So the price of HDD and arrays may be out of your control due to natural or un-natural events, but you can take better control of your total costs by considering and implementing the right storage architecture that meets your cost sensitivities. 2012-13 may produce other problems that impact price and some of the cost factors, and all of these events cannot be predicted. Therefore the burden of implementing economically superior (and price-fluctuation-tolerant) architectures should be high on your 2012 to-do list.

Also, take a look at Hu Yoshida’s recent post on storage efficiencies.

Recently, I spent a few days with Garth Gibson, a computer scientist at Carnegie Mellon University and the founder of Panasas, an enterprise server and storage company. Garth and I were in Singapore for a review with the Data Storage Institute.

Garth is best known for the research paper that he authored with David Patterson and Randy Katz in 1988, “A Case for Redundant Arrays of Inexpensive Disks (RAID)”, which was the catalyst for the RAID storage industry. Once this paper was published and presented at conferences, it took only a few years for all the major storage vendors to deliver RAID storage systems and for customers to adopt this new technology. The rate of this adoption was phenomenal.

When I asked Garth about this, he said the reason RAID was adopted so quickly was that this was delivered as a paper and not as a patent. It was freely available to the industry. Garth and Randy also included a taxonomy that defined RAID levels 1 to 5 and mathematical calculations to determine Mean Time To Failure (MTTF), a key factor in fault tolerance.

It also was helped by the availability of relatively inexpensive 5.5-inch disk drives and the premise that a RAID array of inexpensive disk drives could replace more expensive enterprise storage systems with the same reliability and performance. The industry was quick to drop the term “Inexpensive” in favor of “Independent” and RAID was redefined as Redundant Array of Independent Disks.

Tracing RAID’s Origins

Actually, the concept of RAID was introduced much earlier. Garth does not claim that he invented RAID. The earliest patent on RAID was filed by Norman “Ken” Ouchi of IBM, who was issued U.S. Patent 4,092,732 titled, “System for recovering data stored in a failed memory unit” in 1978. (Claus Mikkelsen and I worked with Ken at IBM.)

This patent described what Garth and his colleagues later defined as RAID 5. Ken’s patent also mentioned mirroring (RAID 1) and dedicated parity (RAID 4) as prior art at that time. So RAID has been around for some time but was not adopted until the RAID paper in 1988 which gave it a name, taxonomy, and a financial justification.

RAID as a fault tolerance mechanism for storage is running out of gas as the densities of disk media increase and the probability of multi drive failures increase. RAID levels up to RAID 5 only protect against a single drive failure in a RAID group. As densities increase, the probability of a drive failure increases and the RAID rebuild time also increases which affects performance due to drive contention and the increased probability of another drive failing during the rebuild. There is also an increasing problem with uncorrectable read errors as densities increase.

Getting Familiar with Erasure Codes

While everyone is familiar with RAID as a method for protection of a drive failure, many are not familiar with the term “erasure code”.  The image on the left come from IEEE to illustrate the concept.

In information theory, an erasure code is a Forward Error Correction (FEC) code for a binary channel (where data is transmitted as one of two symbols, usually a 0 or 1) that can reconstruct symbols that are erased.  It can be used for networks as wells as storage.

RAID is really a simple form of an erasure code where a parity or check sum is appended to a number of records, so that if one record is lost, it can be reconstructed by summing or XOR the remaining records and parity.

If we want to correct more than one error, additional redundancy must be added and the calculation now becomes a polynomial. This is where you will begin to hear more about erasure codes. RAID 6 is a polynomial erasure code that was introduced with large capacity drives in the last decade to protect against dual drive failures. RAID 6 has two redundancy records, so it requires more overhead than RAID 5 in capacity and processing.  As a result it has not been widely adopted until recently.

RAID 6 also helps with uncorrectable read errors. Today, we strongly recommend the use of RAID 6 with RAID pools where data is stripped across many RAID groups, since a dual drive failure in one RAID group would create data loss in all the applications that are using this pool. The cost of an additional parity drive in each RAID group is relatively inexpensive compared to the application down time and the cost of recovering an entire provisioning pool.

However, RAID 6 is not a long-term panacea since it only protects against dual drive failures. With the increasing rate of drive densities, it won’t be too long before we get concerned over three or more drive failures in a RAID group. Storage vendors are working to address these long term requirements.

While storage systems vendors source their disks from the same disk vendors, the reliability of the disk in a storage system will vary depending on how well the system vendors scrub the drives for errors, the effectiveness of their proprietary error detection and recovery software, their maintenance practices, and their proprietary implementation of erasure codes. Users will need to consider the track record of the vendor’s disk availability and then consider the costs and performance trade offs of different erasure codes.

Happy New Year to all.

I came across this news article from Sweden, where web and data sharing has been recognized as an official state religion.

So, some quick thoughts come to mind:

1. Can your storage investment now be considered a charitable tax-exempt contribution?
2. Do you need to purchase some priestly robes for your storage administrator?
3. Will we only have to limit our time and attention to data management for an hour only on Sundays?
4. How will governments’ IT departments now effectively separate church and state?
5. Will this new religion require some type of sacrifice? (that is a loaded one…)
6. Will the data center now require gold leafing and vaulted ceilings?
7. Will a religious connotation change (raise) your perception of database or storage sales people?
8. Can you now use the freedom of religion argument to increase your mailbox quota?
9. Does that 20-year-old 9-track tape now become a sacred religious relic?
10. Will you have to canonize your storage technical architecture?

Feel free to add to this list.

I do not condone this definition of religion, but it may put a new light on the value and importance of data and information.

Happy 2012! While this year is starting with a lot of uncertainty around the world economy and supply/demand questions, there are still areas of assurance. One is that you can still do more with less to meet your storage needs.

Today, Nikkei, the leading business magazine in Japan, announced that Hitachi VSP was the winner of the 2011 Nikkei Superior Products and Services Awards. This year, over 20,000 products were submitted and only 44 were chosen for the award. Nikkei chose VSP for its usability and high performance through virtualization to reduce CAPEX and OPEX.

This is the latest addition to the list of awards VSP has received, which collectively recognize its innovation. Among these awards are the Information Age Innovation Award for 2011 and the TechTarget Best Enterprise Storage Platform Award for 2010.

The best awards are those that come from our customers, who have made VSP the most successful product in Hitachi Data Systems history. Most customers, who take advantage of the full value of VSP with virtualization of their existing assets, can see a payback in nine months or less, and a 40% reduction in capacity demand.

So if you are looking to increase your storage resources while struggling with budget constraints, contact your HDS account representative or HDS Partner to see what VSP can do for you: http://www.hds.com/corporate/contacts/