The Filter Bubble: Algorithm vs. Curator & the Value of Serendipity

How the web gives us what we want to see, and that’s not necessarily a good thing.

Most of us are aware that our web experience is somewhat customized by our browsing history, social graph and other factors. But this sort of information-tailoring takes place on a much more sophisticated, deeper and far-reaching level than we dare suspect. (Did you know that Google takes into account 57 individual data points before serving you the results you searched for?) That’s exactly what Eli Pariser, founder of public policy advocacy group MoveOn.org, explores in his fascinating and, depending on where you fall on the privacy spectrum, potentially unsettling new book, The Filter Bubble — a compelling deep-dive into the invisible algorithmic editing on the web, a world where we’re being shown more of what algorithms think we want to see and less of what we should see.

I met Eli in March at TED, where he introduced the concepts from the book in one of this year’s best TED talks. Today, I sit down with him to chat about what exactly “the filter bubble” is, how much we should worry about Google, and what our responsibility is as content consumers and curators — exclusive Q&A follows his excellent TED talk:

The primary purpose of an editor [is] to extend the horizon of what people are interested in and what people know. Giving people what they think they want is easy, but it’s also not very satisfying: the same stuff, over and over again. Great editors are like great matchmakers: they introduce people to whole new ways of thinking, and they fall in love.” ~ Eli Pariser

What, exactly, is “the filter bubble”?

EP: Your filter bubble is the personal universe of information that you live in online — unique and constructed just for you by the array of personalized filters that now power the web. Facebook contributes things to read and friends’ status updates, Google personally tailors your search queries, and Yahoo News and Google News tailor your news. It’s a comfortable place, the filter bubble — by definition, it’s populated by the things that most compel you to click. But it’s also a real problem: the set of things we’re likely to click on (sex, gossip, things that are highly personally relevant) isn’t the same as the set of things we need to know.

How did you first get the idea of investigating this?

EP: I came across a Google blog post declaring that search was personalized for everyone, and it blew my mind. I had no idea that Google was tailoring its search results on an individual basis at all — the last I’d heard, it was showing everyone the same “authoritative” results. I got out my computer and tried it with a friend, and the results were almost entirely different. And then I discovered that Google was far from the only company that was doing this. In fact, nearly every major website is, in one way or another. (Wikipedia is a notable exception.)

In an age of information overload, algorithms certainly finding the most relevant information about what we’re already interested in more efficiently. But it’s human curators who point us to the kinds of things we didn’t know we were interested in until, well, until we are. How does the human element fit into the filter bubble and what do you see as the future of striking this balance between algorithmic efficiency and curatorial serendipity?

EP: The great thing about algorithms is that, once you’ve got them rolling, they’re very cheap. Facebook doesn’t have to pay many people to edit the News Feed. But the News Feed also lacks any real curatorial values — what you’re willing to Like is a poor proxy for what you’d actually like to see or especially what you need to see. Human curators are way better at that, for now — knowing that even though we don’t click on Afghanistan much we need to hear about it because, well, there’s a war on. The sweet spot, at least for the near future, is probably a mix of both.

One interesting place this comes up is at Netflix — the basic math behind the Netflix code tends to be conservative. Netflix uses an algorithm called Root Mean Squared Error (RMSE, to geeks), which basically calculates the “distance” between different movies. The problem with RMSE is that while it’s very good at predicting what movies you’ll like — generally it’s under one star off — it’s conservative. It would rather be right and show you a movie that you’ll rate a four, than show you a movie that has a 50% chance of being a five and a 50% chance of being a one. Human curators are often more likely to take these kinds of risks.

How much does Google really know about us, in practical terms, and — more importantly — how much should we care?

EP: That depends on how much you use Google — about me, it knows an awful lot. Just think: it’s got all of my email, so it not only has everything I’ve written to friends, it has a good sense of who I’m connected to. It knows everything I’ve searched for in the last few years, and probably how long I lingered between searching for something and clicking the link. There are 57 signals that Google tracks about each user, one engineer told me, even if you’re not logged in.

Most of the time, this doesn’t have much practical consequence. But one of the problems with this kind of massive consolidation is that what Google knows, any government that is friends with Google can know, too. And companies like Yahoo have turned over massive amounts of data to the US government without so much as a subpoena.

Companies like Yahoo have turned over massive amounts of data to the US government without so much as a subpoena.” ~ Eli Pariser

I’d also argue there’s a basic problem with a system in which Google makes billions off of the data we give it without giving us much control over how it’s used or even what it is.

Do you think that we, as editors and curators, have a certain civic responsibility to expose audiences to viewpoints and information outside their comfort zones in an effort to counteract this algorithmically-driven confirmation bias, or are people better left unburdened by conflicting data points?

EP: In some ways, I think that’s the primary purpose of an editor — to extend the horizon of what people are interested in and what people know. Giving people what they think they want is easy, but it’s also not very satisfying: the same stuff, over and over again. Great editors are like great matchmakers: they introduce people to whole new ways of thinking, and they fall in love.

Is it possible to reconcile personalization and privacy? What are some things we could do in our digital lives to strike an optimal balance?

EP: Well, personalization is sort of privacy turned inside out: it’s not the problem of controlling what the world knows about you, it’s the problem of what you get to see of the world. We ought to have more control over that — one of the most pernicious things about the filter bubble is that mostly it’s happening invisibly — and we should demand it of the companies we use. (They tend to argue that consumers don’t care — we should let them know we do.)

On an individual level, I think it comes down to varying your information pathways. There was a great This American Life episode which included an interview with the guy who looks at new mousetrap designs at the biggest mousetrap supply company. As it turns out, there’s not much need for a better mousetrap, because the standard trap does incredibly well, killing mice 90% of the time.

The reason is simple: Mice always run the same route, often several times a day. Put a trap along that route, and it’s very likely that the mouse will find it and become ensnared.

So, the moral here is: don’t be a mouse. Vary your online routine, rather than returning to the same sites every day. It’s not just that experiencing different perspectives and ideas and views is better for you — serendipity can be a shortcut to joy.

Ed. note: The Filter Bubble is out today and one of the timeliest, most thought-provoking books I’ve read in a long time — required reading as we embrace our role as informed and empowered civic agents in the world of web citizenship.

Brain Pickings has a free weekly newsletter and people say it’s cool. It comes out on Sundays and offers the week’s best articles. Here’s an example. Like? Sign up.

Kurt Vonnegut’s Fictional Interviews with Luminaries

What near-death experiences have to do with Shakespeare, Jesus and Isaac Asimov.

In 1997, iconic writer Kurt Vonnegut pitched an idea to New York public radio station WNYC: He would conduct fictional interview with dead cultural luminaries and ordinary people through controlled near-death experiences courtesy of real-life physician-assisted suicide proponent Dr. Jack Kevorkian, allowing the author to access heaven, converse with his subjects, and leave before it’s too late. The producers loved the idea and Vonnegut churned out a number of 90-second segments “interviewing” anyone from Jesus to Hitler to Isaac Asimov. The interviews — funny, poignant, illuminating, timeless, profoundly human — are collected in God Bless You, Dr. Kevorkian, a fantastic anthology playing on the title of Vonnegut’s 1965 novel, God Bless You, Mr. Rosewater, some of the best cultural satire of the past century.

During my most recently controlled near-death experience, I got to interview William Shakespeare. We did not hit it off. He said the dialect I spoke was the ugliest English he had ever heard, ‘fit to split the ears of groundlings.’ He asked if it had a name, and I said ‘Indianapolis.’” ~ Kurt Vonnegut

Thanks to the wonderful Letters of Note — which you should be reading, or else you’re seriously missing out — here’s Vonnegut’s original pitch to WNYC:

Image courtesy of WNYC via Letters of Note

The interviews offer a priceless blend of cultural commentary and existential human preoccupations by way of comedy, from politics to the meaning of life, in what’s perhaps best-described as TED meets SNL.

I asked this heroic pet lover how it felt to have died for a schnauzer named Teddy. Salvador Biagiani was philosophical. He said it sure beat dying for absolutely nothing in the Viet Nam War.” ~ Kurt Vonnegut

Relentlessly entertaining and (un)surprisingly insightful, God Bless You, Dr. Kevorkian is absolutely fantastic and a rare fiction treat even for those of us with a strong general proclivity for nonfiction.

Brain Pickings has a free weekly newsletter and people say it’s cool. It comes out on Sundays and offers the week’s best articles. Here’s an example. Like? Sign up.

Out of Character: The Psychology of Good and Evil

What Aristotle has to do with Tiger Woods and the story of the world.

The dichotomy of good and evil is as old as the story of the world, and timeless in its relevance to just about everything we do in life, from our political and spiritual views to our taste in music, art and literature to how we think about our simple dietary choices. But while most of us recognize that these concepts of good and bad aren’t always black-and-white categories, we never cease to be surprised when someone or something we’ve perceived as “good” does or becomes something we perceive as “bad,” from an esteemed politician’s transgression to a beloved celebrity’s slip into addiction or scientology or otherwise socially undesirable behavior.

In Out of Character: Surprising Truths About the Liar, Cheat, Sinner (and Saint) Lurking in All of Us, researchers David DeSteno and Piercarlo Valdesolo explore this curious disconnect through the rigorous lens of science. Drawing on their research at the Social Emotions Lab at Northeastern University, the authors offer a fascinating yet highly readable perspective on the psychology of the hero/villain spectrum of human character, inviting us to reconceive personality, both our own and that of others, with a more balanced moral view that reflects the fluidity of human psychology.

The derivation of the word ‘character’ comes from an ancient Greek term referring to the indelible marks stamped on coins. Once character was pressed into your mind or soul, people assumed it was fixed. But what modern science repeatedly shows is that this just isn’t the case. As we discuss in our book, everyone’s moral behavior is much more variable than any of us would have initially predicted.” ~ David DeSteno

In this excellent talk from Northeastern’s Insights series, DeSteno reveals some of the fascinating research behind the book and the illuminating insights that came from it.

The analogy of color is an interesting way to think about [character]. Most of us think that colors are very discrete things — something’s red, it’s got redness; something’s blue, it’s got blueness. But we are creating these categories. They’re not natural kinds, they’re not given in ways that represent fundamentally distinct things. Ultimately, what determines what colors we see are the frequencies of light waves entering our eyes, so it’s along a continuum. It’s kind of the same with character. Things blend. We assume that if someone is good, that we’ve characterized them as good, that’s a discrete category, they can’t be bad. And when they are, our categories shatter. That’s because we have this illusory, arbitrary idea of what vice and virtue mean” ~ David DeSteno

Ultimately, Out of Character: Surprising Truths About the Liar, Cheat, Sinner (and Saint) Lurking in All of Us makes a compelling case for seeing human character as a grayscale continuum, not a black-and-white dichotomy of good and bad, enlisting neuroscience and cognitive psychology to reaffirm the age-old Aristotelian view of virtue and vice as fluid, interlaced existential capacities.

Brain Pickings has a free weekly newsletter and people say it’s cool. It comes out on Sundays and offers the week’s best articles. Here’s an example. Like? Sign up.