The Wisdom of Crowds by James Surowiecki
My rating: 5 of 5 stars

Bill Buckley famously quipped that he would rather be lead by “the first 400 people listed in the Boston telephone directory than the faculty of Harvard,” a populist observation which still brings a smile to our faces. This book, which I’ve wished to read for some time, finally explains the wisdom of Buckley’s insight. It also answers a nagging question – for me at least – on why so many otherwise intelligent politicians, especially on the left, say and do such stupid things.

The simple answer is that diversity of opinion (which the author calls “cognitive diversity”) is the only diversity that matters. Get enough unconnected adults together in a room, with a mix of opinions and experiences, and their deliberations will consistently produce far better results than “going to the experts.” This is counterintuitive in the extreme, as we have been increasingly taught that one should trust a few, heavy-breathing experts in any given area (public health, social policy, you name it) to decide what is best for us plebs. “With most things, the average is mediocrity.” the author explains. “With collective intelligence, it’s excellence.” [p.11] And later: “Heretical or not, it’s the truth: the value of expertise is, in many contexts, overrated.” [p.32] You can imagine that this book was not well received by the Harvard faculty or liberal lions in Boston or elsewhere.

Amazingly the author, James Surowiecki, was a journalist until a few years ago at The New Yorker, where he wrote the Financial Page column. I say amazing as Surowiecki arrived at the venerable institution after editor Tina Brown had worked hard to make the magazine trendy, edgy and, by injecting politics, dumber and dumber. It was actually editor David Remnick who took a chance on him, and for many years he remained a voice of reason within its increasingly politicized pages.

His refreshing thesis is stated clearly in the introduction: “under the right circumstances, groups are remarkably intelligent, and are often smarter than the smartest people in them.” [p.xiii] Ponder that for a moment. The smartest person, even several of the most expert, will regularly underperform the group’s collective wisdom. Surowiecki is wonderfully categorical in saying this insight – “that under the right conditions, imperfect humans can produce near-perfect results – has not been challenged.” [p.106]

The original experiment, which underpins the Wisdom of Crowds (not of Mobs, which the author admits demonstrate the opposite), took place over a century ago at a county fair in England. There, a British scientist named Galton stumbled on an ox weight-judging competition, where 800 fair-goers paid a small fee to make a guess to earn rewards. Obviously, a county fair attracts many farmers and ranchers who are experienced with ox-tending, as well as many fair goers who aren’t. As Surowiecki explains:

“Galton undoubtedly thought that the average guess of the group would be way off the mark. After all, mix a few very smart people with some mediocre people and a lot of dumb people, and it seems likely you’d end up with a dumb answer. But Galton was wrong.” [p.xiii]

The average guess was 1,197 pounds, essentially a perfect guess as the correct one was 1,198. (And notice that Galton’s presumed attitude eerily echoes that of many of today’s intellectuals.)

How could that be? Luck? Anti-populists and elites the world over tremble at the answer…

Yet guessing the weight of an animal is a rather simple assessment of a clear activity: weighing something. The introduction ends with a more recent example involving much more complicated calculations. In 1968, the U.S. submarine Scorpion disappeared on its return from a tour of duty in the North Atlantic. Only the sub’s last reported location was known, which drew a potential search circle of twenty miles wide and many thousands of feet deep – a potentially hopeless task. Instead of gathering a small group of submarine experts, the naval officer in charge “assembled a team of men with a wide range of knowledge, including mathematicians, submarine specialists, and salvage men. Instead of asking them to consult with each other to come up with an answer, he asked each of them to offer his best guess.” Each participant was asked to rate the likelihood of various scenarios (what went wrong, speed at the time, steepness of descent), which were collected and, via a process called Bayes’s theorem, produced a best- or collective guess of the sub’s location. The end result? Only 220 yards from the actual location at the bottom of the sea.

Even so, both these examples come from but one area of the problem-solving arena, which Surowiecki calls simple “cognition” problems (mostly guessing something that can be known definitively). Other areas where the startling wisdom of crowds also manifest include “coordination” problems (what is a fair price for buyers and sellers? how to drive safely in heavy traffic?) and cooperation ones (getting distrustful people to work together even against their self-interest, including paying taxes and dealing with pollution).

“[T]here are times – think of a riot, or a stock market bubble – when aggregating individual decisions produces a collective decision that is utterly irrational.” [p.xix] But these are exceptions which tend to prove the rule, and often lack critical elements for good decision making.

The key to good group decisions, he discovers, is cognitive diversity and independence of thought “because the best collective decisions are the product of disagreement and contest, not consensus or compromise…. Paradoxically the best way for a group to be smart is for each person in it to think and act as independently as possible.” [p. xix]

The author fleshes out the ramifications of these startling observations in capitalist markets, corporate decision making, and even democratic governance, where the skeptical (and eugenicist) Galton realized after his county fair epiphany, “‘The result seems more creditable to the trustworthiness of a democratic judgement than might have been expected.’” [p.xiii]

But what about group think, one wonders? Surowiecki confirms “when decision makers are too much alike – in worldview and mind-set – they easily fall prey to groupthink” which “includ[es] a conviction that dissent is not useful.” “Deliberation in a groupthink setting has the disturbing effect not of opening people’s minds but of closing them.” [pp.25,38]

Allan Bloom’s The Closing of the American Mind anyone? Yet, since its publishing in 1987, have we paid heed to its warnings?

Which brings me to my contention of the startlingly poor decisions that many politicians, especially of the left, make.

Not to pick on New York, one of the bluest of blue states, yet both the Governor and city Mayor made disastrous decisions during this pandemic. Were they the result of liberal groupthink? The Governor mandated nursing homes to open their doors to former and non-resident Covid patients to free up hospital beds, and forbade that the nursing homes test them for Covid. This tragic order was disguised, denied, and only reluctantly reversed after a month of cascading nursing home deaths. The Mayor ordered the public transport schedule (subways and buses) to be cut in half in the condescending belief that if he allowed full schedules it would encourage more non-essential workers to use them. Instead, it forced essential workers into many tighter spaces, likely spreading the disease more easily. (To be fair, the Mayor of London was similarly arrogant and misled.) Do these leaders’ inner circle lack diversity of opinion? Do they only interact with like-minded people? Likely.

Worse, we can now surmise how the Administrative state, birthed by the ultra-progressive President Wilson over a century ago, has transformed into the ideologically-bent and powerful “swamp” of today. As Surowiecki explains,

“And trusting an insulated, unelected elite to make the right decisions is a foolish strategy, given all we now know about small group dynamics, groupthink, and the failure of [cognitive] diversity.” [p.267]

What a recipe for lack of accountability amid a plethora of poor decisions.

Sadly, the recent examples are only the tip of the iceberg of disastrous public policy decisions spanning decades in Democrat-controlled cities, counties, and states. Due to the left’s extreme disdain for opposing views (think of the New York Times’s recent firing an editorial page editor for publishing a well-considered opposing view), many of these places – and almost all universities – have been turned into laboratories and breeding grounds for leftist ideology and groupthink.

Notice that Surowiecki’s keys to good collective decision making, cognitive diversity and independence of thought, are about true and productive diversity – in thinking! The tragic emphasis on cultural diversity and skin color these days (contrary to Martin Luther King Jr.’s vision for America) is in fact producing a monolithic conformity, instead of true diversity, in every policy area it touches.

Why do I suspect that Democrats are particularly susceptible? Besides the rigid PC conformity enforced at almost all places of learning, it is heart-wrenching to watch the Fourth Estate surrender its traditional role of providing balanced and unbiased information, and letting the consumer decide, and increasingly turning into an advocacy monolith for leftist thinking. As Surowiecki warns,

“…independence of opinion is both a crucial ingredient in collectively wise decisions and one of the hardest things to keep intact.” [p.39]

Given that rightist politicians and policy makers swim in a sea of leftist media and educational ideology, and live in a society where leftist culture reigns supreme, they may be less susceptible to groupthink – but not immune. That Republican governors tend to be better managers and make better decisions than their Democrat counterparts may stem from their frequent need to interact with big-city leftist mayors. On the other hand, blue states tend to be uniformly blue, with disastrous outcomes – such as the spiraling pension debts overwhelming Illinois, California, and New York finances.

Surowiecki wrote his book prior to the great populist revolt against elites, unaccountable administrative states, and PC-pushers, and the dripping condescension they evidence towards the average Joe.

While he doesn’t go as far as saying the wisdom of crowds manifests itself in election results (which sometimes more resemble markets manipulated by corruption and purveyors of bias and erroneous information, including many in the media), he does second Churchill’s wisdom that our democratic republic system is the least bad among the options.

In sum, his thesis is a clarion call of warning about the destructive lack of wisdom of almost all of our reigning elites, the Harvard faculty included.

Yet how, finally, to interpret these startling results from the higher, spiritual realm? The author briefly opines on such wisdom in moral matters by quoting a founding father:

“Thomas Jefferson, for one, thought it likely that they [experts] might be worse. ‘State a moral case to a ploughman and a professor’ he wrote. ‘The former will decide it as well as and often better than the latter because he has not been led astray by artificial rules.’” [p.267]

I suspect Surowiecki intimates a better answer when discussing the hubris of experts:

“…there’s little correlation between an expert’s confidence in his judgement and the accuracy of it. In other words, experts don’t know when they don’t know something.” [p.278]

Even those with only a casual understanding of the bible know that the primordial sin, the cause of Lucifer’s fall, was pride. Many of today’s experts (and politicians), untethered by a sense of humility in today’s fashionably anti-Christian culture, are so pride-filled as to dismiss in the name of “progress” the wisdom of their fellow man – as well as the wisdom of our collective history.

The Wisdom of Crowds, James Surowiecki (2005), Anchor Books, New York

View all my reviews