There is a moment, in the quarterly review of a decision that worked, where the room agrees it was a good call. Heads nod. The slide turns. Nobody asks the question that would have made the meeting worth holding.
The question is: was it actually a good decision, or did we get lucky?
We have, between us, sat through several hundred of these reviews. The pattern is steady. When the outcome was good, the room concludes the decision was good. When the outcome was bad, the room concludes the decision was bad. The reasoning runs entirely on the back end. The decision itself — the analysis, the framing, the alternatives the team considered and rejected, the assumptions baked into the case — is barely re-examined.
This sounds harmless. It is not. It is the most expensive cognitive shortcut a leadership team can take, because it ensures that the company does not learn from its decisions. It learns from its outcomes. And the two are different things.
What a decision quality actually means
A decision is a choice between options under uncertainty. The quality of that choice can only be assessed against what was known, and knowable, at the time it was made. The outcome belongs to the future, and the future contains noise — competitors that move, regulations that change, customers that surprise, hires that pan out or don't.
Howard Raiffa made the point in the 1960s. Annie Duke has restated it well in the last decade. The framing is by now uncontroversial in decision science. It is, in our experience, almost entirely absent in business practice.
Consider two cases, both real, both anonymized.
In the first, a regional CEO commits twelve million dollars to a new manufacturing line on the strength of a single channel forecast, no scenario work, and an offhand assurance from the supplier that delivery times are "manageable." The line ships on time. The launch goes well. The CEO is promoted. The decision is filed as a win.
In the second, a different regional CEO commissions a tight reference-class study, runs a premortem with the operating team, identifies three failure modes and writes kill criteria against each, then commits eight million dollars to a smaller version of the same bet. One of the failure modes (a supplier insolvency) materializes. The launch slips eight months. The numbers are ugly. The CEO is quietly counselled to manage smaller projects.
The first decision was bad and got lucky. The second was good and got unlucky. The company learned the wrong lesson from both. Five years later, the same pattern would repeat — except this time the bad-but-lucky CEO is the one in the bigger chair.
Outcomes are evidence, not verdicts. A leadership team that judges its decisions by outcomes is a team that has outsourced its self-correction to noise.
Why outcome bias is the default
Outcome bias is the default for the same reason most cognitive defaults are: it is cognitively cheap and socially convenient. The outcome is observable. The decision quality is not — it has to be reconstructed, deliberately, against the knowledge state at the time. That reconstruction is uncomfortable. It often surfaces that the people in the room could have done better, given what they had.
Boards are particularly prone to this. Board members typically join late and leave often. They see the outcome. They do not see the seven months of pre-deal work, the alternatives considered and dropped, the assumptions privately disagreed about, the kill criteria that were almost written and weren't. The deck they read is the deck that justifies the decision, not the decision itself. The outcome, when it lands, is the only data they have to work with — so they use it.
The result is a system that selects for executives who appear to make good decisions, which in practice means executives who are lucky, or who tell stories about their decisions skillfully enough that the lucky-versus-skilled question never arises.
What outcome bias costs
The cost is most visible in three places.
The first is learning. A company that judges decisions by outcomes cannot learn from a near-miss. The near-miss looks like a win. The factors that almost killed the project go unexamined. The next project, run by the same team, repeats them — and the next, until one of them happens to coincide with bad luck and the whole pattern detonates at once. Most "sudden" corporate failures, in our experience, are the cumulative settlement of decision debt that previous good luck had been hiding.
The second is promotion. Promote on outcomes and you promote, on average, the executives who took bigger bets. Some of those bets were calibrated; some weren't. Over time the population of senior leaders skews toward people who happened to be on the lucky side of large variances. Those people will, statistically, regress. The middle of their next decade is full of large public failures that the talent process did not have the language to predict.
The third — quietly the worst — is culture. When the company rewards outcomes, the rational executive stops reasoning explicitly about uncertainty. The deck gets sharper. The hedging gets smoothed. The internal disagreement disappears from the documents, which means it disappears from the institutional memory. Three years in, nobody can reconstruct why the company entered the Asian market, or what would have had to be true for the bet to have failed. They cannot learn from a record they did not write.
A five-question test for the next big call
In our engagements, we treat the question of decision quality operationally. Before any commitment north of a meaningful threshold (the threshold varies by company; we usually advise pegging it to a fraction of operating profit), the team is asked five questions. None are exotic. The discipline is in answering them honestly.
- What are we choosing between, and what are we choosing against? A decision is a comparison. If the deck contains one option, the deck is not yet a decision document.
- What would have to be true for this to be the wrong call? Not a list of "risks." A list of worlds, each one specific enough that you could tell, six months in, whether you were living in it.
- What is our confidence in the part that actually matters? Not blanket confidence in the deck. Confidence in the specific assumption — usually one or two — that the rest of the case hangs on. Most decks have one assumption that, if wrong, ends the project. Find it. Stress it.
- What evidence between now and the next review would update us? A decision with no anticipated update is not a bet. It is a vow. Vows are for weddings, not allocations.
- Who in the room would tell us this is wrong, and what would they say? If nobody can answer, the room has not done the work. If everyone names the same person, that person was not actually invited.
A board that asks these questions, in front of the executive presenting, restructures the meeting. The deck gets thinner. The conversation gets sharper. The decision the company makes is a different decision, and — averaged over the next ten years — a meaningfully better one.
How we run decision audits
The compounding form of this discipline is the decision audit. Once a year, a leadership team takes its twenty largest decisions of the prior twelve months, scores each on what was knowable at the time versus what was actually weighed, and writes a one-page assessment of decision-process quality independent of outcome. The audit takes about two days. It produces, reliably, more institutional learning than any planning offsite we have ever sat through.
The first time a team does this, the reaction is uncomfortable. Decisions that were celebrated come back marked "lucky." Decisions that failed come back marked "well-framed; bad draw." Promotions look different in the light of the audit than they did when the outcomes were fresh. Some teams find this too uncomfortable to repeat. Those teams will, in due course, repeat their mistakes.
The teams that lean into it — that institute the audit annually, share the findings widely, and adjust promotion and compensation against decision quality rather than only outcome quality — are the teams whose forecasts get tighter, whose strategy becomes more honest, whose bench gets deeper. The audit is not the only thing that matters. It is, in our judgment, the highest-leverage thing a leadership team can do that nobody else in their industry is doing.
We help executives and boards install this discipline before it has to be installed reactively, after a large failure has made the case for them. If your last twelve months produced one or two outcomes that surprised you — in either direction — that is a conversation worth having. The surprise is usually the audit's first finding.
Writes about decision quality at Bayeseon. Reach the team at hello@bayeseon.com.