Is data growth really propelling storage array deployments?  That's a multi-million dollar question with an impact both on hardware acquisition budgets, where storage technology purchases account for between 30 and 75 cents of every dollar spent on technology hardware annually, and on prospects for greener data center operations.

According to Randy Chalfant, a Chief Technology Officer for Sun Microsystems, and based on thousands of storage assessments performed at client sites by Sun's storage consultants over the past couple of years, the reality of new data growth is far from cut and dried.  Based on his analyses, which are based on empirical observation and data collection, we tend to misuse storage capacity in a significant way. 

According to the study, a clear picture of disk capacity waste comes into focus.  Only about 30 percent of any disk drive deployed in a typical organization contains objects that are actually being used to deliver business value.  By contrast, 40 percent of the data is "inert" -- meaning that it may have some value to the organization, but accesses to the data have dropped to zero.  Chances are that some of this data could be deleted, while some may need to be retained for reasons ranging from historical value to the organization to regulatory compliance.

Additionally, 15 percent of the capacity of a disk drive is allocated-but-unused.  This is space that is reserved by applications for future use, but is basically worthless as active storage capacity unless some sort of thin provisioning software is active on the array.  Most thin provisioning schemes leverage this space, spoofing applications to believe that it is still held in reserve, while making it available to other applications for storing bits.  This is a kind of storage shell game that can work if tightly integrated with space usage forecasting technology.  Generally speaking, however, thin provisioning schemes are proprietary and obfuscate the assessment of actual storage capacity at any given time.

Chalfant also discovered that 10 percent of disk, on average, contained orphaned data -- that is, data written by applications or users that no longer exist.  By his estimates, another 5 percent of capacity on average is being used to store contraband data such as non-business related MP3 music files, video files, images or other junk data.

If Chalfant's data is even close to accurate, it paints a picture of storage as a huge junk drawer, with its contents either poorly managed or unmanaged altogether.  By consequence, companies are deploying as a matter of routine between 60 and 70 percent MORE capacity than they really need for business purposes.  This translates into huge electrical power expense -- both in terms of storage array loading and increased HVAC requirements -- that could be AVOIDED with an effective data management strategy that includes regular data hygeine and intelligent archiving.

Sorting out the storage junk drawer could save billions of dollars wasted to power storage spindles with less than 30 percent capacity utilization efficiency.