High tech is watching you

The continuing advances of the digital revolution can be dazzling. But Shoshana Zuboff, professor emerita at Harvard Business School, warns that their lights, bells, and whistles have made us blind and deaf to the ways high-tech giants exploit our personal data for their own ends.

In her new book, “The Age of Surveillance Capitalism,” Zuboff offers a disturbing picture of how Silicon Valley and other corporations are mining users’ information to predict and shape their behavior.

The Gazette recently interviewed Zuboff about her belief that surveillance capitalism, a term she coined in 2014, is undermining personal autonomy and eroding democracy — and the ways she says society can fight back.

Q&A

Shoshana Zuboff

GAZETTE: The digital revolution began with great promise. When did you start worrying that the tech giants driving it were becoming more interested in exploiting us than serving us?

ZUBOFF: In my 2002 book, “The Support Economy,” I looked at the challenges to capitalism in shifting from a mass to an individual-oriented structure of consumption. I discussed how we finally had the technology to align the forces of supply and demand. However, the early indications were that the people framing that first generation of e-commerce were more preoccupied with tracking cookies and attracting eyeballs for advertising than they were in the historic opportunity they faced.

For a time I thought this was part of the trial and error of a profound structural transformation, but, certainly by 2007, I understood that this was actually a new variant of capitalism that was taking hold of the digital milieu. The opportunities to align supply and demand around the needs of individuals were overtaken by a new economic logic that offered a fast track to monetization.

GAZETTE: What are some of the ways we might not realize that we are losing our autonomy to Facebook, Google, and others?

ZUBOFF: I define surveillance capitalism as the unilateral claiming of private human experience as free raw material for translation into behavioral data. These data are then computed and packaged as prediction products and sold into behavioral futures markets — business customers with a commercial interest in knowing what we will do now, soon, and later. It was Google that first learned how to capture surplus behavioral data, more than what they needed for services, and used it to compute prediction products that they could sell to their business customers, in this case advertisers. But I argue that surveillance capitalism is no more restricted to that initial context than, for example, mass production was restricted to the fabrication of Model T’s.

Right from the start at Google it was understood that users were unlikely to agree to this unilateral claiming of their experience and its translation into behavioral data. It was understood that these methods had to be undetectable. So from the start the logic reflected the social relations of the one-way mirror. They were able to see and to take — and to do this in a way that we could not contest because we had no way to know what was happening.

We rushed to the internet expecting empowerment, the democratization of knowledge, and help with real problems, but surveillance capitalism really was just too lucrative to resist. This economic logic has now spread beyond the tech companies to new surveillance–based ecosystems in virtually every economic sector, from insurance to automobiles to health, education, finance, to every product described as “smart” and every service described as “personalized.” By now it’s very difficult to participate effectively in society without interfacing with these same channels that are supply chains for surveillance capitalism’s data flows. For example, ProPublica recently reported that breathing machines purchased by people with sleep apnea are secretly sending usage data to health insurers, where the information can be used to justify reduced insurance payments.

GAZETTE: Why have we failed even now to take notice of the effects of all this surveillance?

ZUBOFF: There are many reasons. I chronicle 16 explanations as to “how they got away with it.” One big reason is that the audacious, unprecedented quality of surveillance capitalism’s methods and operations has impeded our ability to perceive them and grasp their meaning and consequence.

Another reason is that surveillance capitalism, invented by Google in 2001, benefitted from a couple of important historical windfalls. One is that it arose in the era of a neoliberal consensus around the superiority of self-regulating companies and markets. State-imposed regulation was considered a drag on free enterprise. A second historical windfall is that surveillance capitalism was invented in 2001, the year of 9/11. In the days leading up to that tragedy, there were new legislative initiatives being discussed in Congress around privacy, some of which might well have outlawed practices that became routine operations of surveillance capitalism. Just hours after the World Trade Center towers were hit, the conversation in Washington changed from a concern about privacy to a preoccupation with “total information awareness.” In this new environment, the intelligence agencies and other powerful forces in Washington and other Western governments were more disposed to incubate and nurture the surveillance capabilities coming out of the commercial sector.

A third reason is that these methodologies are designed to keep us ignorant. The rhetoric of the pioneering surveillance capitalists, and just about everyone who has followed, has been a textbook of misdirection, euphemism, and obfuscation. One theme of misdirection has been to sell people on the idea that the new economic practices are an inevitable consequence of digital technology. In America and throughout the West we believe it’s wrong to impede technological progress. So the thought is that if these disturbing practices are the inevitable consequence of the new technologies, we probably just have to live with it. This is a dangerous category error. It’s impossible to imagine surveillance capitalism without the digital, but it’s easy to imagine the digital without surveillance capitalism.

A fourth explanation involves dependency and the foreclosure of alternatives. We now depend upon the internet just to participate effectively in our daily lives. Whether it’s interfacing with the IRS or your health care provider, nearly everything we do now just to fulfill the barest requirements of social participation marches us through the same channels that are surveillance capitalism’s supply chains.

GAZETTE: You warn that our very humanity and our ability to function as a democracy is in some ways at risk.

ZUBOFF: The competitive dynamics of surveillance capitalism have created some really powerful economic imperatives that are driving these firms to produce better and better behavioral-prediction products. Ultimately they’ve discovered that this requires not only amassing huge volumes of data, but actually intervening in our behavior. The shift is from monitoring to what the data scientists call “actuating.” Surveillance capitalists now develop “economies of action,” as they learn to tune, herd, and condition our behavior with subtle and subliminal cues, rewards, and punishments that shunt us toward their most profitable outcomes.

What is abrogated here is our right to the future tense, which is the essence of free will, the idea that I can project myself into the future and thus make it a meaningful aspect of my present. This is the essence of autonomy and human agency. Surveillance capitalism’s “means of behavioral modification” at scale erodes democracy from within because, without autonomy in action and in thought, we have little capacity for the moral judgment and critical thinking necessary for a democratic society. Democracy is also eroded from without, as surveillance capitalism represents an unprecedented concentration of knowledge and the power that accrues to such knowledge. They know everything about us, but we know little about them. They predict our futures, but for the sake of others’ gain. Their knowledge extends far beyond the compilation of the information we gave them. It’s the knowledge that they have produced from that information that constitutes their competitive advantage, and they will never give that up. These knowledge asymmetries introduce wholly new axes of social inequality and injustice.

GAZETTE: So how do we change this dynamic?

ZUBOFF: There are three arenas that must be addressed if we are to end this age of surveillance capitalism, just as we once ended the Gilded Age.

First, we need a sea change in public opinion. This begins with the power of naming. It means awakening to a sense of indignation and outrage. We say, “No.” We say, “This is not OK.”

Second, we need to muster the resources of our democratic institutions in the form of law and regulation. These include, but also move beyond, privacy and antitrust laws. We also need to develop new laws and regulatory institutions that specifically address the mechanisms and imperatives of surveillance capitalism.

A third arena relates to the opportunity for competitive solutions. Every survey of internet users has shown that once people become aware of surveillance capitalists’ backstage practices, they reject them. That points to a disconnect between supply and demand: a market failure. So once again we see a historic opportunity for an alliance of companies to found an alternative ecosystem — one that returns us to the earlier promise of the digital age as an era of empowerment and the democratization of knowledge.

This interview has been edited for clarity and length.