Star Wars: the Rise of Online Review Culture

In the days before the Internet, eating at an unknown restaurant meant relying on a clutch of quick and dirty heuristics. The presence of many truck drivers or cops at a lonely diner supposedly vouchsafed its quality (though it may simply have been the only option around). For “ethnic” food, there was the classic benchmark: “We were the only non-[insert ethnicity] people in there.” Or you could spend anxious minutes on the sidewalk, under the watchful gaze of the host, reading curling, yellowed reviews, wondering if what held in 1987 was still true today. In an information-poor environment, you sometimes simply went with your gut (and left clutching it).

Today, via Yelp (or TripAdvisor or Amazon, or any Web site teeming with “user-generated content”), you are often troubled by the reverse problem: too much information. As I navigate a Yelp entry to simply determine whether a place is worth my money, I find myself battered between polar extremes of experience: One meal was “to die for,” another “pretty lame.” Drifting into narrow currents of individual proclivity (writing about a curry joint where I had recently lunched, one reviewer noted that “the place had really good energy, very Spiritual [sic], which is very important to me”), I eventually capsize in a sea of confusion. I either quit the place altogether or, by the time I arrive, am weighed down by a certain exhaustion of expectation, as if I had already consumed the experience and was now simply going through the motions.

What I find most striking is that, having begun the process of looking for reviews of the restaurant, I find myself reviewing the reviewers. The use of the word “awesome” — a term whose original connotation is so denuded that I suspect it will ultimately come to exclusively signify its ironic, air-quote-marked opposite — is a red flag. So are the words “anniversary” or “honeymoon,” often written by people with inflated expectations for their special night; their complaint with any perceived failure on the part of the restaurant or hotel to rise to this momentous occasion is not necessarily mine. I reflexively downgrade reviewers writing in the sort of syrupy dross picked up from hotel brochures (“it was a vision of perfection”).

In one respect, there is nothing new in reviewing the reviewer; our choices in pre-Internet days were informed either by friends we trusted or critics whose voices seemed to carry authority. But suddenly, the door has been opened to a multitude of voices, each bearing no preexisting authority or social trust. It is no longer merely enough to read that someone thought the vegetarian food was bad (you need to know if she is a vegetarian), or the hotel in Iowa City was the best they have ever seen (just how many hotels have they seen?), or a foreign film was terrible (wait, they admit they don’t like subtitles?). Critics have always had to be interrogated this way (what dendritic history of logrolling lay behind the rave about that book?), but with the Web, a thousand critics have bloomed. The messy, complicated, often hidden dynamics of taste and preference, and the battles over it, are suddenly laid out right in front of us.

In The German Ideology, Karl Marx famously ruminated on the predicted collapse of the division of labor in a communist society, where he would be free to “do one thing today and another tomorrow, to hunt in the morning, fish in the afternoon, rear cattle in the evening, criticize after dinner, just as I have a mind, without ever becoming hunter, fisherman, shepherd, or critic.” It may not be communism, but the Internet has enabled the fruition of at least one of these activities: criticizing after dinner — particularly if the object of criticism is dinner itself.

The rise of this crowd-sourced aggregate of amateur reviewers, a reserve army of critical labor, is generally seen as an egalitarian blossoming, freeing consumers from the tyranny of individual mandarins, each harboring his or her own agendas and tastes. “The excising of the expert reviewer is happening right across the board,” writes Suzanne Moore in The Guardian. “Who needs expertise when every Tom, Dick, and Harriet reviews everything for free anyway. Isn’t this truly democratic? The nature of criticism is changing, so this hierarchy of expertise is crumbling.”

One can almost hear the anticipatory echoes of something like Yelp in the context of José Ortega y Gasset’s The Revolt of the Masses (1930). The multitude, he wrote, once “scattered about the world in small groups,” now appears “as an agglomeration.” It has “suddenly become visible, installing itself in the preferential positions in society. Before, if it existed, it passed unnoticed, occupying the background of the social stage; now it has advanced to the footlights and is the principal character.” The disgruntled diner, now able to make or break a restaurant through sheer collective will. Against this leveling of critical power, the old guard fulminates. Ruth Reichl, the former editor of Gourmet, recently harrumphed that “anybody who believes Yelp is an idiot. Most people on Yelp have no idea what they’re talking about.”

If the God Criticism — in the sense of experts telling the anxious middle what to read, what to see, and how to be — now lays on its side, an Enver Hoxha statue in a Tirana back alley, what’s left? A new utopia of fisherman-critics who are free to make up their own minds and influence others? A glorious world of transparency and objectivity? A radical rewriting of the canon?

Perhaps. But there are complications with this idea that the Internet has obviated the need for experts and for critical authority. One question is what is happening to criticism itself when the evaluative architecture on a site such as Amazon is the same for leaf blowers as it is literature, when everything seems to be quantifying one’s hedonic response to a consumption activity; when we are forced into a ruthless dyad of thumbing up or thumbing down, or channeled into expressing a simple “liking” for something when the actual response may be more complex.

If the Internet was supposed to wrest criticism from elites, a good deal of the reviewing energy on Yelp (and other sites) is precisely an effort to establish one’s bona fides. In the reviews for a new seafood restaurant in my neighborhood, a number of the writers tout themselves as “New Englanders,” thus implying that they implicitly know of what they speak. A reviewer for an Indian restaurant in midtown Manhattan lays down a sort of tripartite claim on authority: “I am a foodie and my love for Indian food (as an Indian) is tough to match. I eat at this restaurant at least once a week. Really innovative mix of ingredients, and yet extremely authentic.” Not only is he a foodie, he is an Indian foodie who, like all true food critics, has eaten here more than once—thus no need to unpack that thorny word “authentic.”

Yelp is filled with this sort of signaling, as economists call it — making discreet references affirming one’s authority in an effort to rise above the masses of similar reviewers (“I knew the chef from his previous stint at ...”; “of all the Henan cuisine places I’ve eaten, this is one of the ...”). And even as it aggregates its democratic horde — after filtering out reviews for various reasons, including those suspected of being fraudulent — Yelp itself strives to reintroduce hierarchy, by designating a class of “elite” reviewers (identified by special badges), picked by a team known as The Council. “We don’t share how it’s done,” a Yelp spokesperson said, as if describing the shadowy process by which Michelin inspectors are hired.

What further complicates this picture of the masses liberating the objects of criticism from the tyranny of critics is that so many reviewers seem to turn toward petty despotism. Reading Yelp reviews, particularly of the one-star variety, one quickly senses the particular ax being ground — the hostess who shot the “wrong” look at the “girls’ night” group; a greeting that is too effusive, or insufficiently so; the waiter deemed “too uneasy with being a waiter”; or any number of episodes (each example has been taken from Yelp) that have little to do with food.

As Paul Myerscough, an editor at The London Review of Books, has written of the “affective labor” that is now such a prominent feature of the service economy (and is drilled into workers at chains such as Pret A Manger through quasi-Stakhanovite uplift campaigns), “Work increasingly isn’t, or isn’t only, a matter of producing things, but of supplying your energies, physical and emotional, in the service of others.” For consumers-turned-overseers who feel they did not receive the right kind of emotional energy, Yelp becomes a place to catalog these litanies of complaint.

When I was in college, around the time critical theory was in full bloom on the American campus, a favorite professor of mine kicked off a seminar by saying we were to going to do criticism. “I’m not talking about the sort of gonadal, ‘thumbs up/thumbs down’ kind of criticism,” she said. Rather, we would analyze texts, films — any kind of cultural product — assisted by an array of high-powered lenses: deconstructionism, semiotics, structuralism, reader-response, Jacques Lacan, Roland Barthes. (My writing career may actually have launched with a Mythologies-style analysis of the “Not Your Father’s Oldsmobile” ad campaign.) Whether we liked something or not was irrelevant; our job was to think about what the work said, what the work did, and what we brought to it.

There may be no field besides criticism more open, or indeed appropriate, to perpetual, almost Maoist self-examination; I do not imagine that accountants or podiatrists much concern themselves with impassioned forums in which the “role of the accountant” or the “future of podiatry” is debated. Criticism itself is meant to be criticized — an idea nicely captured in the title of an H. L. Mencken essay, “Criticism of Criticism of Criticism.” Wrote Mencken of the critic: “He makes the work of art live for the spectator; he makes the spectator live for the work of art. Out of the process comes understanding, appreciation, intelligent enjoyment.” Nearly a century later, despite all the hand-wringing, some version of this definition is extant. In his critics’ “manifesto,” Daniel Mendelsohn, author of the recent Waiting for the Barbarians, argues that the “critic is someone who, when his knowledge, operated on by his taste in the presence of some new example of the genre he’s interested in — a new TV series, a movie, an opera or ballet or book — hungers to make sense of that new thing, to analyze it, interpret it, make it mean something.”

Most online reviewing, Mendelsohn notes, “isn’t criticism proper” — it’s full of heat, yes, but lacks light. And before you cry elitism, he notes that academics and other “expert” reviewers often fall prey to the reverse condition. For a comparison of criticism and online reviewing, let us first turn to Mendelsohn himself, writing about Mad Men, the cable television series chronicling mid-20th-century social upheaval:

Worst of all — in a drama with aspirations to treating social and historical “issues” — the show is melodramatic rather than dramatic. By this I mean that it proceeds, for the most part, like a soap opera, serially (and often unbelievably) generating, and then resolving, successive personal crises (adulteries, abortions, premarital pregnancies, interracial affairs, alcoholism and drug addiction, etc.), rather than exploring, by means of believable conflicts between personality and situation, the contemporary social and cultural phenomena it regards with such fascination: sexism, misogyny, social hypocrisy, racism, the counterculture, and so forth.

This is a critic at the top of his game, a few deft strokes forcing you to reassess your own judgment — and not in a purely “like” or “dislike” sense — of a show, one that comes bedecked with awards and other tokens of critics’ adoration.

And now let us turn to Netflix. Here is the review of Mad Men deemed “most helpful” (whatever that means) by an impressive 393 out of 394 viewers, as of the time of writing:

I am in the middle of Season 3 and I cannot stop watching, this show is incredible! If my math is correct Sally Drapper [sic] was born in 54. In 1962 she is 8. I was born in 1954. I had an Aunt, Cousin, and family friend that were ‘working girls’ in the 60s and for me the show depicts this world perfectly.

It goes on, in a similar vein. Netflix formerly featured quotes from working critics on the title pages of films and shows, but as a Netflix employee who counts himself a committed cineaste explained to me with a certain chagrin, by his reckoning, only 15 percent or so of Netflix users were interested in professional critics’ opinions.

Yet freed from the yoke of expert opinion, what are we left with? Hundreds of individual reviews, each written by people who, like critics, come bearing their own agendas and biases. You may not “like” A. O. Scott’s taste, but at least you know who he is and what he stands for. And so we look for new forms of authority and trust, new ways to filter. We ourselves are invited to review the reviewers (if only in that “gonadal” good/bad sort of way) by “liking” their comments or rating their helpfulness. Data points pile upon data points.

It is precisely in this vast range of online activity where the value and interest lie for researchers investigating what is not actually known as “criticism” but, rather, “electronic word of mouth.” The trove of data generated from online reviews, the thinking goes, may offer quantitative insight into a perpetually elusive dynamic: the formation of judgments, the expression of preferences, the mechanics of taste. The results, filled with subtle biases and conformity effects, are not always pretty.

While any one review is essentially useless — the low transaction cost, as the Columbia Business School’s Ray Fisman has noted, tags it with the “cheap talk” problem — the aggregate level is where, through sheer numbers, the noise can be filtered, the outliers marginalized, and statistical consensus achieved.

For example, Yelp compiles key lines that more or less repeat in reviews (“the homemade whipped cream on the side hit the spot”) and posts these “highlights” near the top of the page. The shift to smartphones and smaller screens has made the idea of reading through dozens of reviews even less palatable, encouraging greater compression by opinion-mining projects such as the University of Washington’s RevMiner. It rates descriptive words by their “strength”; for example, “exquisite” is stronger than “good.” A person searching for something like “good dim sum,” the researchers note, does not really mean good dim sum, but “dim sum that others have described as ‘great’ or ‘amazing.’” Good is no longer good enough. You need to be awesome.

Customers deem this feedback desirable, and it can move cultural markets. The Harvard Business School’s Michael Luca has found, for example, that a one-star uptick in a Yelp review can lead to a nine percent improvement in revenues for independently owned restaurants. Other studies have shown a similar impact for independent hotels — and for books.

In a discussion on the future of literary criticism, Jen Doll noted in The Atlantic Wire that “book reviews are not science.” And yet, on Amazon they approach it. One can discover that the average rating for goods sold on the site is approximately 4.3 stars out of 5, as it is elsewhere on the Internet. Why so high? There is, undoubtedly, a raft of selection biases going on — an author’s fans are more likely to weigh in positively; an existing review’s rating wields an influence on later reviewers; the mere fact of having purchased something may make someone less likely to issue a negative review. But curiously, as more ratings trickle in, a study by business professors David Godes and Jose Silva has found, the average rating begins to decline. “The more reviews there are,” Godes and Silva suggest, “the lower the quality of the information available”; later reviewers tend to be either less serious or less disposed to like the book, or to respond to other reviewers rather than to the book itself. While one might think a five-star review would summon more passion than a four-star review, one study found that four-star reviews were, on average, longer.

What consumers make of reviewers is also a fertile field of study. A team of Cornell University and Google researchers, for example, found that a review’s “helpfulness” rating falls as the review’s star rating deviates from that of the average review — as if it were being punished for straying. As the team noted, defining “helpfulness” is itself tricky: Did the review help people make a purchase, or were they rewarding it for conforming with what others were saying? There are a number of feedback effects: Early reviews tend to draw more helpfulness votes, simply because they’ve appeared online longer. The more votes a review has, the more its “default authority,” and the more votes it tends to attract.

One thing the Internet reviewing culture makes clear is that, at least with “experience goods” — things such as books or music — we often seem to react more strongly to someone else’s opinion than to the work itself. As Temple University’s Susan Mudambi and David Schuff found, people tend to rate longer reviews for “search goods” — such as cameras or printers — more positively than those for “experience goods.” A strong negative review for a camera might reflect some discrete product failure (pictures were blurry), but a strong negative review for a book might simply be another person’s taste getting in the way.

Indeed, one might protest that reviewing restaurants at Yelp and books at Amazon and films at Netflix are all different enterprises, but I would argue that there is a sort of metalogic to online reviewing that subsumes all categories. What people are doing, after all, is generally not situating a work in its historical context — or performing some other kind of critical heavy lifting — but reflecting upon their own consumption experience. This logic has become so ingrained, so expected, that one occasionally spies a flummoxed “review” of a simple product such as paper clips: “What can I say? They’re paper clips!” Four stars!

That a site such as Amazon sells virtually everything under the sun offers a chance for these varying groupings of products — “search goods,” “experience goods” — to blur and flatten, as do the lines of authority; what does the competent paper clip critic have to say about French symbolist poetry?

Trawling through the reviews of a book I recently purchased, The Old Ways, an elegy to life on foot by the travel writer Robert Macfarlane, I was struck by the sole one-star review: “I too use walking as a way of thinking. But without maps of his walks this book is seriously incomplete. I wonder why there are no maps.” One might argue that this is to have missed much of the point of Macfarlane’s work. But the ground on which this person was engaging the book — a narrow quibble over a functional attribute of the book itself having nothing to do with the writing — was not unlike the assessment of any other consumer good on Amazon, such as a conferral of one star on the iPad for not having a USB port.

Yet all may not be lost. That one-star review of The Old Ways has received not a single “helpful” vote. What’s more, two readers felt compelled to weigh in on the review itself. “It appears you had one criterion, and only one, for your rating of this book,” wrote one. “Did anything make you think this book would include maps?” asked another. The rise of online reviewing may be toppling the singular critical voice from its pedestal, and with its fall, taste has shattered into a thousand fragments. We are every day sifting through those shards, trying to make meaning of everyone else’s attempt to say what something meant to them.

—

Tom Vanderbilt is the author of Traffic: Why We Drive the Way We Do (and What It Says About Us), a columnist for Slate, and a contributing editor of Outside and Print. He lives in Brooklyn, New York.

Photo courtesy of Shutterstock