360 In-Depth: The Gatekeepers of Google

In 2006, Thailand announced it was blocking access to YouTube for anyone with a Thai I.P address, and then identified 20 offensive videos for Google to remove as a condition of unblocking the site.

'If your whole game is to increase market share,' says Lawrence Lessig, a Stanford professor and lawyer speaking of Google, 'it's hard to . . . gather data in ways that don't raise privacy concerns or in ways that might help repressive governments to block controversial content.'

In March of last year, Nicole Wong, the deputy general counsel of Google, was notified that there had been a precipitous drop in activity on YouTube in Turkey, and that the press was reporting that the Turkish government was blocking access to YouTube for virtually all Turkish Internet users. Apparently unaware that Google owns YouTube, Turkish officials didn't tell Google about the situation: a Turkish judge had ordered the nation's telecom providers to block access to the site in response to videos that insulted the founder of modern Turkey, Mustafa Kemal Ataturk, which is a crime under Turkish law. Wong scrambled to figure out which videos provoked the court order and made the first in a series of tense telephone calls to Google's counsel in London and Turkey, as angry protesters gathered in Istanbul. Eventually, Wong and several colleagues concluded that the video that sparked the controversy was a parody news broadcast that declared, "Today's news: Kamal Ataturk was gay!" The clip was posted by Greek football fans looking to taunt their Turkish rivals.

Wong and her colleagues asked the Turkish authorities to reconsider their decision, pointing out that the original offending video had already been voluntarily removed by YouTube users. But after the video was taken down, Turkish prosecutors objected to dozens of other YouTube videos that they claimed insulted either Ataturk or "Turkishness." These clips ranged from Kurdish-militia recruitment videos and Kurdish morality plays to additional videos speculating about the sexual orientation of Ataturk, including one superimposing his image on characters from "Queer Eye for the Straight Guy." "I remember one night, I was looking at 67 different Turkish videos at home," Wong told me recently.

After having many of the videos translated into English, Wong and her colleagues set out to determine which ones were, in fact, illegal in Turkey; which violated YouTube's terms of service prohibiting hate speech but allowing political speech; and which constituted expression that Google and YouTube would try to protect. There was a vigorous internal debate among Wong and her colleagues at the top of Google's legal pyramid. Andrew McLaughlin, Google's director of global public policy, took an aggressive civil-libertarian position, arguing that the company should protect as much speech as possible. Kent Walker, Google's general counsel, took a more pragmatic approach, expressing concern for the safety of the dozen or so employees at Google's Turkish office. The responsibility for balancing these and other competing concerns about the controversial content fell to Wong, whose colleagues jokingly call her "the Decider," after George W. Bush's folksy self-description.

Wong decided that Google, by using a technique called I.P. blocking, would prevent access to videos that clearly violated Turkish law, but only in Turkey. For a time, her solution seemed to satisfy the Turkish judges, who restored YouTube access. But last June, as part of a campaign against threats to symbols of Turkish secularism, a Turkish prosecutor made a sweeping demand: that Google block access to the offending videos throughout the world, to protect the rights and sensitivities of Turks living outside the country. Google refused, arguing that one nation's government shouldn't be able to set the limits of speech for Internet users worldwide. Unmoved, the Turkish government today continues to block access to YouTube in Turkey.

THE ONGOING DISPUTE between Google and Turkey reminds us that, throughout history, the development of new media technologies has always altered the way we think about threats to free speech. At the beginning of the 20th century, civil libertarians in America worried most about the danger of the government silencing political speech: think of Eugene V. Debs, the Socialist candidate for President, who was imprisoned in 1919 for publicly protesting American involvement during World War I. But by the late 1960s, after the Supreme Court started to protect unpopular speakers more consistently, some critics worried that free speech in America was threatened less by government suppression than by editorial decisions made by the handful of private mass-media corporations like NBC and CBS that disproportionately controlled public discourse. One legal scholar, Jerome Barron, even argued at the time that the courts should give unorthodox speakers a mandatory right of access to media outlets controlled by giant corporations.

Today the Web might seem like a free-speech panacea: it has given anyone with Internet access the potential to reach a global audience. But though technology enthusiasts often celebrate the raucous explosion of Web speech, there is less focus on how the Internet is actually regulated, and by whom. As more and more speech migrates online, to blogs and social-networking sites and the like, the ultimate power to decide who has an opportunity to be heard, and what we may say, lies increasingly with Internet service providers, search engines and other Internet companies like Google, Yahoo, AOL, Facebook and even eBay.

The most powerful and protean of these Internet gatekeepers is, of course, Google. With control of 63 percent of the world's Internet searches, as well as ownership of YouTube, Google has enormous influence over who can find an audience on the Web around the world. As an acknowledgment of its power, Google has given Nicole Wong a central role in the company's decision-making process about what controversial user-generated content goes down or stays up on YouTube and other applications owned by Google, including Blogger, the blog site; Picasa, the photo-sharing site; and Orkut, the social networking site. Wong and her colleagues also oversee Google's search engine: they decide what controversial material does and doesn't appear on the local search engines that Google maintains in many countries in the world, as well as on Google.com. As a result, Wong and her colleagues arguably have more influence over the contours of online expression than anyone else on the planet.

In response to the rise of online gatekeepers like Wong, some House Democrats and Republicans have introduced a bipartisan bill called the Global Online Freedom Act, which would require that Internet companies disclose to a newly created office in the State Department all material filtered in response to demands by foreign governments. Google and other leading Internet companies have sought modifications to the bill, arguing that, without the flexibility to negotiate (as Wong did with Turkey), they can't protect the safety of local employees and that they may get kicked out of repressive countries, where they believe even a restricted version of their services does more good than harm. For the past two years, Google, Yahoo and Microsoft, along with other international Internet companies, have been meeting regularly with human rights and civil-liberties advocacy groups to agree on voluntary standards for resisting worldwide censorship requests. At the end of last month, the Internet companies and the advocacy groups announced the Global Network Initiative, a series of principles for protecting global free expression and privacy.

Voluntary self-regulation means that, for the foreseeable future, Wong and her colleagues will continue to exercise extraordinary power over global speech online. Which raises a perennial but increasingly urgent question: Can we trust a corporation to be good - even a corporation whose informal motto is "Don't be evil"?

"To love Google, you have to be a little bit of a monarchist, you have to have faith in the way people traditionally felt about the king," Tim Wu, a Columbia law professor and a former scholar in residence at Google, told me recently. "One reason they're good at the moment is they live and die on trust, and as soon as you lose trust in Google, it's over for them." Google's claim on our trust is a fragile thing. After all, it's hard to be a company whose mission is to give people all the information they want and to insist at the same time on deciding what information they get.

THE HEADQUARTERS OF YOUTUBE are in a former Gap building in San Bruno, Calif., just a few miles from the San Francisco International Airport. In the lobby, looming over massage chairs, giant plasma-screen TVs show popular videos and scroll news stories related to YouTube. The day I arrived to interview the YouTube management about how the site regulates controversial speech, most of the headlines, as it happens, had to do with precisely that topic. Two teenagers who posted a video of themselves throwing a soft drink at a Taco Bell employee were ordered by a Florida judge to post an apology on YouTube. The British culture secretary had just called on YouTube to carry warnings on clips that contain foul language.

The volume of videos posted on YouTube is formidable - Google estimates that something like 13 hours of content are uploaded every minute. YouTube users can flag a video if they think it violates YouTube's community guidelines, which prohibit sexually explicit videos, graphic violence and hate speech. Once flagged, a video is vetted by YouTube's internal reviewers at facilities around the world who decide whether to take it down, leave it up or send it up the YouTube hierarchy for more specialized review. When I spoke with Micah Schaffer, a YouTube policy analyst, he refused to say how many reviewers the company employs. But I was allowed to walk around the office to see if I could spot any of them. I passed one 20-something YouTube employee after another - all sitting in cubicles and wearing the same unofficial uniform of T-shirt and jeans. The internal reviewers were identifiable, I was told, only by the snippets of porn flickering on their laptops.

The idea of a 20-something with a laptop in San Bruno (or anywhere else, for that matter) interpreting community guidelines for tens of millions of users might not instill faith in YouTube's vetting process. But the most controversial user flags or requests from foreign governments make their way up the chain of command to the headquarters of Google, in Mountain View, Calif., where they may ultimately be reviewed by Wong, McLaughlin and Walker.

Walker, the general counsel, wrote for The Harvard Crimson as an undergraduate and considered becoming a journalist before going into law; McLaughlin, the head of global public policy, became a fellow at Harvard's Berkman Center for Internet and Society after working on the successful Supreme Court challenge to part of the federal Communications Decency Act. And Wong, a soft-spoken and extremely well organized woman, has a joint degree in law and journalism from Berkeley and told me she aspired to be a journalist as a child because of her aunt, a reporter for The Los Angeles Times.

Wong gives the best analogy for her role at Google. Is she acting like a judge? An editor? "I don't think it's either of those," she said. "I definitely am not trying to pass judgment on anything. I'm taking my best guess at what will allow our products to move forward in a country, and that's not a judge role, more an enabling role." She stressed the importance for Google of bringing its own open culture to foreign countries while still taking into account local laws, customs and attitudes. "What is the mandate? It's 'Be everywhere, get arrested nowhere and thrive in as many places as possible.' " So far, no Google employees have been arrested on Wong's watch, though some have been detained.

When Google was founded, 10 years ago, it wasn't at all obvious whether the proprietors of search engines would obey the local laws of the countries in which they did business - and whether they would remove links from search results in response to requests from foreign governments. This began to change in 2000, when a French Jew surfed a Yahoo auction site to look for collections of Nazi memorabilia, which violated a French law banning the sale and display of anything that incites racism. After a French judge determined that it was feasible for Yahoo to identify 90 percent of its French users by analyzing their I.P. addresses and to screen the material from the users, he ordered Yahoo to make reasonable efforts to block French users from accessing the prohibited content or else to face fines and the seizure of income from Yahoo's French subsidiary. In January 2001, Yahoo banned the sale of Nazi memorabilia on its Web sites.

The Yahoo case was a landmark. It made clear that search engines like Google and Yahoo could be held liable outside the United States for indexing or directing users to content after having been notified that it was illegal in a foreign country. In the United States, by contrast, Internet service providers are protected from most lawsuits involving having hosted or linked to illegal user-generated content. As a consequence of these differing standards, Google has considerably less flexibility overseas than it does in the United States about content on its sites, and its "information must be free" ethos is being tested abroad.

For example, on the German and French default Google search engines, Google.de and Google.fr, you can't find Holocaust-denial sites that can be found on Google.com, because Holocaust denial is illegal in Germany and France. In the wake of the Yahoo decision, Google decided to comply with governmental requests to take down links on its national search engines to material that clearly violates national laws. (In the interest of disclosure, however, Google has agreed to report all the links it takes down in response to government demands to chillingeffects.com, a Web site run by Harvard's Berkman Center that keeps a record of censored online materials.)

Of course, not every overseas case presents a clear violation of national law. In 2006, for example, protesters at a Google office in India demanded the removal of content on Orkut, the social networking site, that criticized Shiv Sena, a hard-line Hindu political party popular in Mumbai. Wong eventually decided to take down an Orkut group dedicated to attacking Shivaji, revered as a deity by the Shiv Sena Party, because it violated Orkut terms of service by criticizing a religion, but she decided not to take down another group because it merely criticized a political party. "If stuff is clearly illegal, we take that down, but if it's on the edge, you might push a country a little bit," Wong told me. "Free-speech law is always built on the edge, and in each country, the question is: Can you define what the edge is?"

INITIALLY, GOOGLE'S POLICY of removing links to clearly illegal material on its foreign search engines seemed to work. But things changed significantly after Google bought and expanded YouTube in 2006. Once YouTube was available in more than 20 countries and in 14 languages, users began flagging hundreds of videos that they saw as violations of local community standards, and governments around the globe demanded that certain videos be blocked for violating their laws. Google's solution was similar to the one the French judge urged on Yahoo: it agreed to block users in a particular country from accessing videos that were clearly illegal under local law. But that policy still left complicated judgment calls in murkier cases.

In late 2006, for example, Wong and her colleagues debated what to do about a series of videos that insulted the king of Thailand, where a lêse-majesté law makes criticisms of the king a criminal offense. Wong recalls hearing from an employee in Asia that the Thai government had announced that it was blocking access to YouTube for anyone with a Thai I.P. address. Soon after, a Thai government official sent Wong a list of the U.R.L.'s of 20 offensive videos that he demanded Google remove as a condition of unblocking the site. Some of the videos were sexually explicit or involved hate speech and thus clearly violated the YouTube terms of service. Some ridiculed the king - by depicting him with his feet on his head, for example - and were clearly illegal under Thai law but not U.S. law. And others - criticizing the Thai lêse-majesté law itself - weren't illegal in Thailand but offended the government.

After an extensive debate with McLaughlin and Walker, Wong concluded that since the lêse-majesté law had broad democratic support in Thailand, it would be better to remove the videos that obviously violated Thai law while refusing to remove the videos that offended the government but didn't seem to be illegal. All three told me they were reassured by the fact that Google could accommodate the Thai government by blocking just the videos that were clearly illegal in Thailand (and blocking those for Thai users only), leaving them free to exercise their independent judgment about videos closer to the line. The Thai government was apparently able to live with this solution.

Over the past couple of years, Google and its various applications have been blocked, to different degrees, by 24 countries. Blogger is blocked in Pakistan, for example, and Orkut in Saudi Arabia. Meanwhile, governments are increasingly pressuring telecom companies like Comcast and Verizon to block controversial speech at the network level. Europe and the U.S. recently agreed to require Internet service providers to identify and block child pornography, and in Europe there are growing demands for network-wide blocking of terrorist-incitement videos. As a result, Wong and her colleagues said they worried that Google's ability to make case-by-case decisions about what links and videos are accessible through Google's sites may be slowly circumvented, as countries are requiring the companies that give us access to the Internet to build top-down censorship into the network pipes.

IT'S NOT ONLY FOREIGN COUNTRIES that are eager to restrict speech on Google and YouTube. Last May, Senator Joseph Lieberman's staff contacted Google and demanded that the company remove from YouTube dozens of what he described as jihadist videos. (Around the same time, Google was under pressure from "Operation YouTube Smackdown," a grass-roots Web campaign by conservative bloggers and advocates to flag videos and ask YouTube to remove them.) After viewing the videos one by one, Wong and her colleagues removed some of the videos but refused to remove those that they decided didn't violate YouTube guidelines. Lieberman wasn't satisfied. In an angry follow-up letter to Eric Schmidt, the C.E.O. of Google, Lieberman demanded that all content he characterized as being "produced by Islamist terrorist organizations" be immediately removed from YouTube as a matter of corporate judgment - even videos that didn't feature hate speech or violent content or violate U.S. law. Wong and her colleagues responded by saying, "YouTube encourages free speech and defends everyone's right to express unpopular points of view." In September, Google and YouTube announced new guidelines prohibiting videos "intended to incite violence."

In addition to Lieberman, another outspoken critic of supposed liberal bias at YouTube and Google is Michelle Malkin, the conservative columnist and blogger. Malkin became something of a cause célèbre among YouTube critics in 2006, when she created a two-minute movie called "First, They Came" in the wake of the violent response to the Danish anti-Muhammad cartoons. After showing pictures of the victims of jihadist violence (like the Dutch filmmaker Theo Van Gogh) and signs declaring "Behead Those Who Insult Islam," the video asks, "Who's next?" and displays the dates of terrorist attacks in America, London, Madrid and Bali.

Nearly seven months after she posted the video, Malkin told me she was "flabbergasted" to receive an e-mail message from YouTube saying the video had been removed for its "inappropriate content." When Malkin asked why the video was removed, she received no response, and when she posted a video appealing to YouTube to reinstate it, that video, too, was deleted with what she calls the "false claim" that it had been removed at her request. Malkin remains dissatisfied with YouTube's response. "I'm completely flummoxed about what their standards are," she said. "The standards need to be clear, they need to be consistent and they need to be more responsive."

I watched the "First, They Came" video, which struck me as powerful political commentary that contains neither hate speech nor graphic violence, and I asked why it was taken down. According to a YouTube spokesman, the takedown was a routine one that hadn't been reviewed by higher-ups. The spokesman said he couldn't comment on particular cases, but he forwarded a link to Malkin's current YouTube channel, noting that it contains 55 anti-jihadist videos similar to "First, They Came," none of which have been taken down. ("First, They Came" can now be found on Malkin's YouTube channel, too.)

The removal of Malkin's video may have been an innocent mistake. But it serves as a reminder that one person's principled political protest is another person's hate speech, and distinguishing between the two in hard cases is a lot to ask of a low-level YouTube reviewer. In addition, the publicity that attended the removal of Malkin's video only underscores the fact that in the vast majority of cases in which material is taken down, the decision to do so is never explained or contested. The video goes down, and that's the end of it.

Yet even in everyday cases, it's often no easier to determine whether the content of a video is actually objectionable. When I visited YouTube, the management showed me a flagged French video of a man doubled over. Was he coughing? Or in pain? Or playacting? It was hard to say. The YouTube managers said they might send the item to a team of French-language reviewers for further inspection, but if the team decided to take down the video, its reasons would most likely never become public.

AS THE LAW PROFESSOR TIM WU TOLD ME, to trust Google, you have to be something of a monarchist, willing to trust the near-sovereign discretion of Wong and her colleagues. That's especially true in light of the Global Network Initiative, the set of voluntary principles for protecting free expression and privacy endorsed last month by leading Internet companies like Google and leading human rights and online-advocacy groups like the Center for Democracy and Technology. Google and other companies say they hope that by acting collectively, they can be more effective in resisting censorship requests from repressive governments and, when that isn't possible, create a trail of accountability.

Google is indeed more friendly to free speech than the governments of most of the countries in which it operates. But even many of those who are impressed by Wong and her colleagues say the Google "Decider" model is impractical in the long run, because, as broadband use expands rapidly, it will be unrealistic to expect such a small group of people to make ad hoc decisions about permissible speech for the entire world. "It's a 24-hour potential problem, every moment of the day, and because of what the foreign governments can do, like put people in jail, it creates a series of issues that are very, very difficult to deal with," Ambassador David Gross, the U.S. coordinator for International Communications and Information Policy at the State Department, told me. I asked Wong whether she thought the Decider model was feasible in the long term, and to my surprise, she said no. "I think the Decider model is an inconsistent model because the Internet is big and Google isn't the only one making the decisions," she told me.

Wong and her colleagues said they would be happiest, of course, if more countries would adopt U.S.-style free-speech protections. Knowing that that is unlikely, they said they would prefer that countries around the world set up accountable bodies that provide direct guidance about what controversial content to restrict. As an example of his preferred alternative, Andrew McLaughlin pointed to Germany, which has established a state agency that gathers the U.R.L.'s of sites hosting Nazi and violent content illegal under German law and gives the list to an industry body, which then passes it on to Google so that it can block the material on its German site. (Whenever Google blocks material there or on its other foreign sites, it indicates in the search results that it has done so.)

It is striking - and revealing - that Wong and her colleagues would prefer to put themselves out of business. But it is worth noting that even if Google's suggestion were adopted, and governments around the world began to set up national review boards that told Google what content to remove, then those review boards might protect far less free speech than Google's lawyers have. When this concern was raised, McLaughlin said he hoped that the growing trends to censor speech, at the network level and elsewhere, would be resisted by millions of individual users who would agitate against censorship as they experienced the benefits of free speech.

There's much to be said for McLaughlin's optimism about online free-speech activism. Consider recent experiences in Turkey, where a grass-roots "censuring the censors" movement led more than 400 Turkish bloggers to shutter their Web sites in solidarity with mainstream sites that were banned for carrying content that, among other things, insulted Turkey's founding father. In America, and around the world, the boundaries of free speech have always been shaped more by political activism than by judicial decisions or laws. But what is left out of McLaughlin's vision is uncertainty about one question: the future ethics and behavior of gatekeepers like Google itself.

"Right now, we're trusting Google because it's good, but of course, we run the risk that the day will come when Google goes bad," Wu said. In his view, that day might come when Google allowed its automated Web crawlers, or search bots, to be used for law-enforcement and national-security purposes. "Under pressure to fight terrorism or to pacify repressive governments, Google could track everything we've searched for, everything we're writing on gmail, everything we're writing on Google docs, to figure out who we are and what we do," he said. "It would make the Internet a much scarier place for free expression." The question of free speech online isn't just about what a company like Google lets us read or see; it's also about what it does with what we write, search and view.

WU'S FEARS THAT violations of privacy could chill free speech are grounded in recent history: in China in 2004, Yahoo turned over to the Chinese government important account information connected to the e-mail address of Shi Tao, a Chinese dissident who was imprisoned as a result. Yahoo has since come to realize that the best way of resisting subpoenas from repressive governments is to ensure that private data can't be turned over, even if a government demands it. In some countries, according to Michael Samway, who heads Yahoo's human rights efforts, Yahoo is now able to store communications data and search queries offshore and limits access of local employees, so Yahoo can't be forced to turn over this information even if it is ordered to do so.

Isolating, or better still, purging data is the best way of protecting privacy and free expression in the Internet age: it's the only way of guaranteeing that government officials can't force companies like Google and Yahoo to turn over information that allows individuals to be identified. Google, which refused to discuss its data-purging policies on the record, has raised the suspicion of advocacy groups like Privacy International. Google announced in September that it would anonymize all the I.P. addresses on its server logs after nine months. Until that time, however, it will continue to store a wealth of personal information about our search results and viewing habits - in part to improve its targeted advertising and therefore its profits. As Wu suggests, it would be a catastrophe for privacy and free speech if this information fell into the wrong hands.

"The idea that the user is sovereign has transformed the meaning of free speech," Wu said enthusiastically about the Internet age. But Google is not just a neutral platform for sovereign users; it is also a company in the advertising and media business. In the future, Wu said, it might slant its search results to favor its own media applications or to bury its competitors. If Google allowed its search results to be biased for economic reasons, it would transform the way we think about Google as a neutral free-speech tool. The only editor is supposed to be a neutral algorithm. But that would make it all the more insidious if the search algorithm were to become biased.

"During the heyday of Microsoft, people feared that the owners of the operating systems could leverage their monopolies to protect their own products against competitors," says the Internet scholar Lawrence Lessig of Stanford Law School. "That dynamic is tiny compared to what people fear about Google. They have enormous control over a platform of all the world's data, and everything they do is designed to improve their control of the underlying data. If your whole game is to increase market share, it's hard to do good, and to gather data in ways that don't raise privacy concerns or that might help repressive governments to block controversial content."

Given their clashing and sometimes self-contradictory missions - to obey local laws, repressive or not, and to ensure that information knows no bounds; to do no evil and to be everywhere in a sometimes evil world - Wong and her colleagues at Google seem to be working impressively to put the company's long-term commitment to free expression above its short-term financial interests. But they won't be at Google forever, and if history is any guide, they may eventually be replaced with lawyers who are more concerned about corporate profits than about free expression. "We're at the dawn of a new technology," Walker told me, referring not simply to Google but also to the many different ways we now interact online. "And when people try to come up with the best metaphors to describe it, all the metaphors run out. We've built this spaceship, but we really don't know where it will take us."

Will