Sunday, March 13, 2011

The digital pile up

SOME facts of life are just plain counterintuitive. It can be too cold to snow. Heavy things float. Martinis have calories.

Here’s another one with significantly greater import: Electronic information is tangible. The apps we use, the games on our phones, the messages we incessantly tap — all of it may seem to fly through the air and live in some cloud, but in truth, most of it lands with a thump in the earthly domain.

Because electronic information seems invisible, we underestimate the resources it takes to keep it all alive. The data centers dotting the globe, colloquially known as “server farms,” are major power users with considerable carbon footprints. Such huge clusters of servers not only require power to run but must also be cooled. In the United States, it’s estimated that server farms, which house Internet, business and telecommunications systems and store the bulk of our data, consume close to 3 percent of our national power supply. Worldwide, they use more power annually than Sweden.

But it’s not the giants like Google or Amazon or Wall Street investment banks that are responsible for creating the data load on those servers — it’s us. Seventy percent of the digital universe is generated by individuals as we browse, share, and entertain ourselves.

And the growth rate of this digital universe is stunning to contemplate.

The current volume estimate of all electronic information is roughly 1.2 zettabytes, the amount of data that would be generated by everyone in the world posting messages on Twitter continuously for a century. That includes everything from e-mail to YouTube. More stunning: 75 percent of the information is duplicative. By 2020, experts estimate that the volume will be 44 times greater than it was in 2009. There finally may be, in fact, T.M.I.

Proliferating information takes a human toll, too, as it becomes more difficult to wade through the digital detritus. We’re all breeding (and probably hoarding) electronic information. Insensitive to our data-propagating power, we forward a joke on a Monday that may produce 10 million copies by Friday — probably all being stored somewhere.

Despite the conveniences our online lives provide, we end up being buried by data at home and at work. An overabundance of data makes important things harder to find and impedes good decision-making. Efficiency withers as we struggle to find and manage the information we need to do our jobs. Estimates abound on how much productivity is lost because of information overload, but all of them are in the hundreds of millions of dollars yearly.

In the corporate realm, companies stockpile data because keeping it seems easier than figuring out what they can delete. This behavior has hidden costs and creates risks of security and privacy breaches as data goes rogue.

In addition, large corporations face eye-popping litigation costs when they search for information that may be evidence in a lawsuit — so-called e-discovery — that can add up to millions of dollars a year. Cases are often settled because it’s cheaper to just pay up. With so many resource challenges facing them, most companies postpone the effort and cost of managing their data.

Technological innovation usually carries with it the seeds that spawn solutions. The demand for power by big and small players alike is driving development of energy alternatives and data center innovation. Artificial intelligence and other more sophisticated information retrieval processes are making a dent in the cost of e-discovery and can also help rid companies of their stockpiles. Advances in cloud computing and virtual storage will help consolidate applications and data. But it might still be a question as to whether the planet can continue to feed our digital appetite. Improvements in the digital highway usually just lead to more traffic, and we’re in danger of data asphyxiation as it is.

Is there anything we can do? No one wants to give up the pleasures and benefits that the digital domain provides. But we can at least wake up to the toll that it’s taking and search for solutions. We can live a productive digital life without hoarding information. As stockholders and consumers, we can demand that our companies and service providers aggressively engage in data-reduction strategies. We can clean up the stockpiles of dead data that live around us, be wiser data consumers, text less and talk more. We can try hitting delete more often.

While some will be tempted to argue that it won’t make much of a dent, we have to give it a shot. As with any conservation effort, it’s the small actions of a large group that end up making the difference.

Shelley Podolny works for a company that advises corporations on information management. NYT 13 March 2011.