How numbers rule our lives

Worried that your personal information is being collected and manipulated? Wonder why your favorite websites know your taste in books and movies better than your own mother does? In Super Crunchers, Ian Ayres answers these questions and more as he tells a numerical tale both exciting and ­cautionary.

In the past several years, advances in computer storage capacity have made possible massive databases that are changing the ways government agencies, market research firms, and universities run their ­operations—­and our lives. Ayres, an economist and law professor at Yale, sets out to explain how these databases, some of which contain thousands of times the information stored in the Library of Congress, can be quickly analyzed to shape ­real-world decisions.

Super Crunchers is not the dry econometrics textbook you couldn’t get through in college. Ayres relies on baseball scouts, wine critics, entrepreneurs, and doctors to illustrate his argument, and goes into the gory mathematical details only in the last chapter. He illustrates ­regression—­a statistical procedure that exploits databases to estimate how var­ious factors influence a single outcome—­by describing how electronic matchmaking sites such as eHarmony and True.com put people together. These companies propose matches by collecting clues to personality traits and social skills with detailed questionnaires. The clues become data, which are then crunched to calculate how compatible one person might be with others who have also submitted information. Sometimes, opposites do attract.

The basic idea underlying super crunching is that using regression techniques produces far more accurate results than the intuition and experience of the traditional expert. The debate over how decisions should be made is nowhere clearer than in Ayres’s chapter on ­“evidence-­based medicine.” No human brain, even that of a good doctor, can actively recall all 11,000 human diseases and their symptoms. Enter the Isabel database, which serves as the “Google of medical diagnosis.” The doctor logs a patient’s symptoms, and Isabel generates a list of possible diagnoses. In about 10 percent of cases, Isabel points doctors toward a diagnosis they didn’t consider but should ­have.

Though Ayres initially titled his book The End of Intuition, he does leave a role for the gut. People are bad at weighing the relative importance of factors that affect a given outcome, but they have good hunches about which variables should be considered in the first place. Databases alone do not always yield definitive results; some have been so manipulated that they resemble “prisoners who will tell you anything you want to know.” Ayres advises all crunchers to check their assumptions carefully and, if possible, to allow results to be independently verified.

He offers the cautionary tale of John Lott, a scholar who, in 2000, created and analyzed a crime database and found that when citizens are allowed to carry concealed weapons, ­would-­be attackers are discouraged and crime decreases. Lott’s conclusions were instrumental in the passage of at least nine state laws permitting concealed weapons. When he shared his database with Ayres and others, however, they discovered that slight changes to the regression equations negated the more guns/less crime relationship, and that after coding errors were corrected, the data set suggested that con­cealed weapons laws are, if anything, likely to increase crime ­rates.

A couple of years ago, Malcolm Gladwell made the case for intuition. In Blink, he argued that instantaneous, subconscious decisions based on accumulated knowledge and experience are often superior to those derived from “more deliberate and exhaustive ways of thinking.”

What Ayres and Gladwell do agree on is that the brain does not make good decisions when flooded with information. We are biased by preconceived ideas, prone to rationalization, and easily influenced. We think we are right far more often than we are. After reading Ayres’s book, some may be ready to throw intuition to the wind. The more cautious among us will take comfort in Gladwell’s conviction that the human mind may still yield insights not readily supplanted by those of a computer.

* * *

Reviewed: Super Crunchers: Why ­Thinking-­by-­Numbers Is the New Way to Be ­Smart. By Ian Ayres. Bantam. 260 pp.

Photo courtesy of Wikimedia Commons