Author: Philip E. Tetlock and Dan Gardner · Published: 2015 · Genre: Psychology, Decision Making
Superforecasting shows that some ordinary people can predict world events far better than chance, and better than many experts, and explains the habits that make them good. Its biggest strength is its evidence: the lessons come from a large, multi-year, government-funded forecasting tournament, not from anecdotes. Its biggest limitation is scope, since the method works best on clear questions that resolve within a year or so, not on the rare, world-changing events that matter most. It’s worth reading for anyone whose job involves judgment under uncertainty.
The core ideas
- Most expert predictions are barely better than guessing. Tetlock’s earlier research, published as Expert Political Judgment (2005), found that the average expert’s long-range forecasts did little better than random guesses, the famous dart-throwing chimpanzee. Experts wedded to one big idea, the hedgehogs, did worse than foxes, who drew on many small ideas.
- Forecasting can be measured and improved. In the Good Judgment Project, Tetlock and Barbara Mellers recruited thousands of volunteers for a tournament run by IARPA, a US intelligence research agency, from 2011 to 2015. Accuracy was scored with Brier scores, which punish confident wrong answers. The top 2 percent, the superforecasters, mostly stayed on top year after year, and even a short training module improved accuracy.
- How they do it. Superforecasters start with the outside view (how often do things like this usually happen?) before adding specifics. They break big questions into smaller estimable parts, as the physicist Enrico Fermi did, use precise probabilities rather than vague words like likely, and update often in small steps as news arrives.
- Mindset beats raw brainpower. They are smart but not geniuses. What sets them apart is being actively open-minded, treating beliefs as hypotheses to test, and keeping score. Teams of superforecasters did better still.
What’s great
The book is built on data and is candid about it. Tetlock reports that his forecasters beat a control group by a wide margin and, by a widely reported figure, outperformed intelligence analysts with access to classified information by roughly 30 percent. Those are unusual claims for a popular book, and they rest on scored, checkable forecasts.
Gardner, a journalist, keeps the prose readable, and the profiles of individual forecasters, ordinary volunteers rather than credentialed pundits, make the lessons concrete. The advice is practical: you can start scoring your own predictions tomorrow, and the closing list of ten commandments for aspiring superforecasters is a handy summary.
What’s weak
The biggest limitation is one Tetlock admits. Tournament questions had to resolve within months or a year or so, and the book concedes that accuracy fades toward chance as horizons stretch out to several years. Nassim Nicholas Taleb argues that what matters most is the rare, extreme event, and that scoring many small probability calls tells you little about exposure to catastrophe. The book engages this critique seriously but doesn’t fully resolve it.
Some readers also find the later chapters, on leadership and military command, a stretch beyond the evidence, and the book is longer than its core message needs. The superforecasters were self-selected volunteers who enjoyed the work, so how well the findings transfer to busy organizations remains an open question.
Who should read it
Analysts, investors, managers and policy people should read it, as should anyone who consumes predictions from pundits. Pair it with Daniel Kahneman’s Thinking, Fast and Slow for the psychology behind the errors and Taleb’s The Black Swan for the strongest counterargument.
Rating: 4.5/5
Image credits — In-article images: Image by NOAA via Wikimedia Commons, Public Domain.
