Consumers Drive Storage Everywhere!
By: Tom Coughlin, Storage Visions
Digital storage has and continues to play an enormous role in every consumer application, whether it is games, TV, mobile devices, home computing, on-line content viewing or home backup and content sharing. The 2013 Storage Visions Conference (SV 2013, www.storagevisions.com), a partner event to the 2013 CES in Las Vegas, taking place January 6 & 7, 2013 will look at the growing use and future trends in digital storage at home, in your pocket, in your vehicle and in the cloud. The conference theme, based upon the growth in storage demand to meet rich media requirements, is “Petabytes are the New Terabytes.” This article will explore trends driving the storage requirements for consumer content that will be among the topics discussed at SV 2013. Some of the content here is extracted from various reports published by Coughlin Associates sometimes in association with Objective Analysis, as well as talks by Tom Coughlin for various events (http://www.tomcoughlin.com/techpapers.htm).
Consumers—the Ultimate Driver for Digital Storage Demand
Consumers are becoming the largest generator of digital content. Many modern consumer devices include content capture capabilities (as well as sharing capabilities) such as the cameras and microphones in mobile phones, tablets, computers as well as many automobiles and other consumer products. The total number of these products is increasing at a staggering rate. In 2012 it is estimated that over 1.6 B mobile phones have been sold with smart phones showing the greatest growth. At the same time it is projected that tablet computers sales in 2012 is roughly 87 M units. These devices can be used to create and share new content as well as view and listen to commercial content.
The size of content in consumer devices in increasing as well as the greater complexity in the various content formats needed to support various screen sizes and distribution channels. Figure 1 shows some sample music and video formats used in consumer products today as well as the future giving bandwidth and storage capacity comparisons.
Figure 1. Storage and Streaming Bandwidth for Consumer Music and Video Formats
As the size and number of content objects increases the total utilization of various technology resources increases. As an example Sandvine recently estimated that peak aggregate traffic composition in North American fixed network traffic was composed of almost 60% real time entertainment, mostly video such as Netflix and YouTube. Coughlin Associates estimates that the average US home in 2014 could have almost 12 TB of total digital content, including commercial as well as backups and user-generated content. With 100 M American households this translates to about 1.2 Zetabytes of data. Of this amount 26 Exabytes are estimated to be user generated content.
We are not far from people recording their lives more or less continuously. There are many products on the market today that are starting to make this possible. With the advent of popular true life-logs consumer generated content could explode, making this larger than any other source of digital content growth. Figure 2 shows an estimate of the growth in raw content from a single individual’s life-log over a five-year period (starting in 2010). Within 5 years time close to 100 TB of raw content could be captured! Now the trick is to find anything in that vast amount of content and that is the job of metadata and searches.
Figure 2. Projected Growth of Raw Digital Content Capacity for a Single Life-Log over Five Years
Better Together—Trends for Digital Storage Technology in Consumer Electronics
Different digital storage devices have different technical and economic characteristics. Common storage technologies used in consumer electronics are flash memory, hard disk drives and optical discs. Flash memory is used in most mobile consumer devices today such as mobile phones and tablet computers. Hard disk drives are used in many game systems, DVRs and set-top boxes with DVR capabilities as well home computers, backup and network storage devices. Optical discs are used for copying and sharing files, but their greatest use has been for commercial media content distribution.
Optical discs have suffered from competition from electronic downloads and streaming except for the highest resolution content. For this reason only Blu-ray optical discs have shown unit growth in recent years. As shown in Figure 3 there is some hope for physical content distribution for the highest resolution content—such as ultra-HD 4K X 2K or even 8 K X 4K (there is a keynote speaker at the 2013 Storage Visions Conference speaking about 8K content from the 2012 Summer Olympics). At these sorts of storage capacities even with anticipated compression technologies it will be difficult to move this content through the Internet.
Figure 3. Media Size Content Trends and Data Rate and Capacity Requirements
Flash memory is finding more uses in computers but for many users the cost of flash memory capacity is too high and a combined solution using some storage on a separate HDD and SSD or with flash memory combined into a HDD to make a hybrid HDD may be more attractive as it provides performance close to that of an SSD with storage capacity costs close to those of a HDD. There are many variations on this approach such as hybrid HDDs expected to be used in many ultra-thin notebook computers in 2013, Apple’s Fusion Drive and other dual storage computers using a separate HDD and SSD.
In many ways because they may have different uses one form of storage might very well drive the growth of another type of storage. For instance HDDs in computers enabled downloading movies and music to portable players while flash based cameras would have their content saved on a hard disk drive while the consumer reused the flash memory. As a result flash use actually could increase the demand for consumer HDD storage. According to a white paper by Tom Coughlin, Jim Handy and Roger Hoyt (Flash and HDD: Symbiosis or Survival of the Fittest?) by 2014 flash memory applications could drive the demand for HDDs by as much as 50 M HDDs.
Consumers Drive Cloud Storage
Smart mobile device such as smart phones and tablet computers are driving demand for on-line resources, including digital storage. According to a recent talk by Cadence for every 600 mobile phones deployed there is a server at the other end and for every 120 iPads there is a server needed. These servers, and the digital storage they contain, are needed to support the applications that have made these devices so compelling to consumers.
Cloud storage has enabled many of the modern consumer applications. Cloud storage is able to do this because the costs of data center storage has been reduced due to virtualization, deduplication and other technologies that have made physical devices much more efficient and better utilized that in the past. It is not uncommon today for data center storage equipment utilization to have increased from 40% 10 years ago to over 80% today. Increasing utilization drives down the costs of on-line storage.
Lower cost on-line storage supports many consumer applications and also gives consumers and small businesses an opportunity to have cost effective remote back-up and disaster recovery. Streaming video and music are also supported by storage equipment in data centers connected to the Internet.
Corporate Internet clouds may also be hybrid clouds composed of private clouds connected to public clouds. Likewise this concept could become more popular with consumer. There are network storage devices available from many vendors today that can provide local storage to mobile devices such as tablet computers (e.g. from Seagate and Kingston) as well as home and desktop storage systems that can be accessed through the Internet offered by many HDD companies as well as other external storage box suppliers.
The growth of a consumer hybrid cloud environment, combined with the growing connections between the major consumer networks of phone, internet and television could eventually lead to a true consumer storage utility as shown in Figure 4 where remote storage is combined with local storage in consumer products to hid the physical source of content and focus on having content be where a consumer wants it and when the consumer wants it. This would be even more useful if data were automatically backed up both locally and remotely and content were automatically indexed and organized, including automated metadata generation of user-generated content.
Figure 4. A Home Storage Utility
Summary and Conclusions
The growth of digital content is growing as the needs of modern consumers include more ways to create, share and experience digital content. User generated content will soon be the bulk of all human content and that will drive vast needs to organize and index this content to make it useful. At the same time commercial content will increase with higher resolution content as well as the increase in the number of formats to support the vast array of content using applications and devices.
Storage technologies will continue to evolve and it is not clear that any one storage technology will support all consumer needs. Even mobile smart phones and tablets, although they use flash memory internally depend upon data centers running apps and providing more storage that is stored on hard disk drives, and even magnetic tape. Also future computers likely will use flash memory in combination with HDDs to provide performance like that of SSDs with storage cost like those of HDDs.
Serving consumer applications and needs will become the greatest use of cloud based data center storage. Further hybrid clouds with both consumer private storage and storage resources in remote data centers will combine to create tomorrow’s home storage utility where data is automatically managed and available to the consumer whenever and wherever he/she is at.
These are topics that will be explored further at the 2013 Storage Visions Conference where digital storage for every aspect of the digital content value chain will be explored. Whether it is content creation, delivery, archiving or content consumption digital storage is needed and in larger amounts. Different applications may require different combinations of storage technology and this drives the continuous development and growth of storage technologies. The Storage Visions Conference is where you can learn about these trends first, www.storagevisions.com.