Fascinating article about Bruce Bueno de Mesquita’s highly controvertial (and highly successful) game theoretic prediction models.

The New York University political science professor has developed a computerized game theory model that predicts the future of many business and political negotiations and also figures out ways to influence the outcome. Two independent evaluations, one by academics and one by the U.S. Central Intelligence Agency, have both shown that about 90 percent of his predictions have been accurate. Most recently, he has used his mathematical tools to offer approaches for handling the growing nuclear crisis with Iran.

An article from Good goes into more detail regarding the derivation of that 90% accuracy rate:

To verify the accuracy of his model, the CIA set up a kind of forecasting face-off that pit predictions from his model against those of Langley’s more traditional in-house intelligence analysts and area specialists. “We tested Bueno de Mesquita’s model on scores of issues that were conducted in real time—that is, the forecasts were made before the events actually happened,” says Stanley Feder, a former high-level CIA analyst. “We found the model to be accurate 90 percent of the time,” he wrote. Another study evaluating Bueno de Mesquita’s real-time forecasts of 21 policy decisions in the European community concluded that “the probability that the predicted outcome was what indeed occurred was an astounding 97 percent.” What’s more, Bueno de Mesquita’s forecasts were much more detailed than those of the more traditional analysts. “The real issue is the specificity of the accuracy,” says Feder. “We found that DI (Directorate of National Intelligence) analyses, even when they were right, were vague compared to the model’s forecasts. To use an archery metaphor, if you hit the target, that’s great. But if you hit the bull’s eye—that’s amazing.”

A key element of his models process is the careful interviewing process used to construct the computer model. Without any further reading, my hypothesis is that it is Bueno de Mesquita’s skill in conducting these interviews that truly sets his approach apart from others. After all, the established fields of decision analysis and facilitation have demonstrated the benefit of augmenting expert assessments with intelligent questioning. At its best, this is what consulting can offer.

What effect could such an approach have on larger policymaking?

[Bueno de Mesquita] has just launched and is the director of NYU’s Alexander Hamilton Center. “The mission for the center is the application of logic and evidence to solving fundamental policy problems. Not to a bipartisan solution, but to a nonpartisan solution.”

I definitely have to learn more about Bueno de Mesquita. If any readers have further reading to augment the below references, they’d be most welcome.

Additional reading:
-Bueno de Mesquita, B. 1997. A decision making model: Its structure and form. International Interactions 23:233-251.
-McGurn, W. 1996. We warned you. Far Eastern Economic Review 159 (June 13):68.