Planning Fallacy: Why We Underestimate and How to Fix It

What is the planning fallacy?

The planning fallacy is the tendency to underestimate how long a task will take, how much it will cost, and how well it will go, despite knowing that comparable past efforts ran late and over budget. Kahneman and Tversky introduced the term in 1979 while describing how people make intuitive predictions.

The key word is “despite.” This is not ignorance. Most people can tell you that their last four reports took longer than planned. They still predict the fifth will be quick, because the prediction is built from a mental simulation of the specific task rather than from the record of similar ones.

Two things get bundled under the label, and separating them matters:

• Genuine bias: you believe the optimistic estimate. This is the psychology problem.
• Strategic misrepresentation: you know the estimate is optimistic and say it anyway, because a realistic number would not get approved. This is a political problem.

Flyvbjerg’s work on megaprojects argues the second is at least as common as the first. The remedies differ: bias needs a better procedure, while misrepresentation needs decision rules that make optimistic numbers expensive to submit.

What did the classic thesis study actually find?

Thirty-seven honours students predicted when they would submit their thesis. The 33 whose completion dates were recorded predicted 33.9 days on average and took 55.5 days — a 64% underestimate. Only 29.7% finished by their predicted date.

The most instructive part is the worst-case condition. Asked to predict a date assuming “everything went as poorly as it possibly could,” the same students averaged 48.6 days. Only 48.7% finished even by that pessimistic date. Their worst case was still optimistic.

That table is the argument for reference class forecasting in miniature. If a group of intelligent people cannot produce a worst case that holds half the time, asking them to “add a bit of buffer” is not a control. It is a hope.

The same paper found the effect is asymmetric: people underestimate their own completion times but not other people’s. Observers, if anything, overestimate how long others will take. That asymmetry is why a colleague can often spot an unrealistic plan faster than its author — and why plan reviews work better when someone outside the work does the reviewing.

Why does the inside view beat the outside view?

Because plans are built by simulating the specific task, and simulation is a poor forecaster. When you imagine your project, you run the version where the dependency arrives on time, the reviewer responds in a day, and nobody gets sick. The simulation is coherent, detailed, and almost never includes the base rate.

Kahneman and Tversky described two perspectives:

• The inside view focuses on this task: the plan, the people, the steps, the obstacles you can foresee.
• The outside view ignores the specifics and asks a simpler question: how long did this class of task take the last twenty times anyone did it?

The inside view feels more rigorous. It is the one that produces a Gantt chart. But it systematically omits the class of events that dominate real durations — interruptions, reworks, approvals, and dependency delays that are individually unlikely and collectively certain.

Four mechanisms keep the bias in place:

Attribution errors. A past project ran late because the supplier was unusually slow or the client changed their mind. Each explanation is locally reasonable, and together they let you conclude this time will be different.

Selective memory. Failures fade faster than successes. Without records, the estimate is rebuilt from a handful of memorable wins rather than the full distribution.

Anchoring on the desired date. Once a deadline exists, the estimate drifts toward it. The plan stops being a forecast and becomes a justification.

Social pressure. Optimistic estimates get approved; cautious ones get challenged. Where the estimate is negotiated rather than measured, the incentive runs one way.

> Important note: The planning fallacy is not the same as optimism bias. Optimism bias is a general belief that good things are more likely to happen to you. The planning fallacy is specifically about predictions of task duration, cost, and outcome — and it appears even in people who describe themselves as pessimists.

How large are the errors in real projects?

Large enough that entire industries plan around them. Flyvbjerg’s database of more than 16,000 projects, measured against the budget approved at decision-to-build, finds that only 0.5% come in on budget, on time, and with the promised benefits.

Other reference points from the same body of work: Olympic Games cost overruns have averaged around 157% in real terms, IT projects around 73%, and dams around 96%. These are not estimates from a bad year; they are the central tendency across decades of projects run by competent organisations.

An IT overrun of 73% is worth pausing on, because most readers here work in software-adjacent projects. It does not mean every sprint runs 73% long; it means that when you class projects by type and measure final cost against the approved business case, the middle of the distribution sits far above the plan.

How does reference class forecasting work?

Reference class forecasting replaces the inside view with a base rate and then applies an explicit uplift. It has three steps: build the reference class, derive the probability distribution of outcomes, and adjust your estimate to the level of risk you are willing to accept.

The method was formalised by Flyvbjerg for the UK Treasury’s Green Book and applied to transport, energy, and IT portfolios. The logic is simple: if comparable projects overran by a median of 34%, a new project in that class should carry at least that much contingency unless there is a specific reason to expect better.

Here is a worked example for a data migration.

A retail company estimates a 14-week, $300,000 migration. It assembles a reference class of 12 comparable migrations from its own history and from two peer organisations: ERP cutovers of similar data volume, similar team size, and similar legacy complexity.

Sorted, the median (average of the 6th and 7th values) is (36 + 41) ÷ 2 = 38.5%. The 80th percentile sits between the 9th and 10th values, around 55%.

• P50 budget: $300,000 × 1.385 = $415,500
• P80 budget: $300,000 × 1.55 = $465,000
• P50 schedule: 14 weeks × 1.385 ≈ 19.4 weeks
• P80 schedule: 14 weeks × 1.55 ≈ 21.7 weeks

That is a very different conversation from “fourteen weeks, $300,000.” The P50 number means half of comparable projects finished within it. The P80 means four in five did. Which one you present depends on what a miss costs — and that is a decision for the sponsor, not the estimator.

Three cautions. The reference class must be narrow enough to be comparable and broad enough to be statistical: twelve projects is thin, forty is better. The data must be actual outcomes against the original baseline, not re-baselined figures that hide the drift. And the uplift must survive approval — a contingency removed in negotiation is not a contingency.

What if you have no reference class?

Use decomposition and structured group estimation. Break the work into units small enough to have their own history, estimate each in ranges, and aggregate; the errors partly cancel, and the aggregate is usually better calibrated than a single top-down number.

Four techniques work in practice:

Decompose to task level. A “website rebuild” has no base rate. “Migrate 40 pages, 3 integrations, and a CMS theme” has components you have done before. Estimate where history exists.

Ask for a range, not a point. A three-point estimate — optimistic, most likely, pessimistic — makes uncertainty visible and gives you a distribution to sum rather than a single number to defend.

Run a premortem. Spend ten minutes writing the story of how the project failed. This surfaces the omissions the inside view edited out, far cheaper than discovering them in week nine.

Estimate a class, then adjust. Ask “what usually happens to projects like this?” before “what will happen to this project?” The order matters; the first question sets the anchor.

How much buffer should you actually add?

Add buffer once, at the project level, sized to the risk you are carrying — not on every task. Task-level padding compounds quietly and gets consumed anyway.

Consider a five-task sequence, each estimated at three days, for a total of 15 working days. Two approaches:

The totals are identical; the behaviour is not. Distributed padding hides the contingency inside each task, so nobody can see how much protection remains. A single buffer is visible, can be tracked as it burns down, and creates a real conversation when it drops below half.

The percentage should come from data, not a rule of thumb. If your last ten projects overran by a median of 25%, a 25% project buffer is not pessimism; it is the base rate. Without data, 30% on novel work and 10–15% on repetitive work is defensible — then measure, because the point is to replace the guess with your own distribution.

> Common mistake: Adding a buffer and then announcing the padded date as a commitment. The buffer gets consumed by Parkinson’s law, and the next estimate starts from an already-inflated baseline. Keep the buffer visible as a separate line, and report against the unpadded plan.

Why does experience alone not fix it?

Because experience is reinterpreted rather than recorded. Buehler, Griffin and Ross found that people explain away past misses as external, temporary, and specific — which strips the past of its predictive value.

The exception is instructive: when participants were instructed to connect relevant past experiences to their prediction, the optimistic bias was eliminated. Not reduced. Eliminated. The outside view works when it is forced, and it does not happen by default.

That is the entire case for keeping records. An organisation that tracks estimates against actuals has a reference class. An organisation that relies on its most experienced manager has vivid anecdotes, filtered for the ones that flatter.

How do you fix estimation on a team?

Measure estimates against actuals every cycle, review the gap without blame, and use the resulting distribution for the next estimate. That loop, run for three months, beats any estimation framework.

A workable routine:

1. Record the estimate at commitment. The original number and the date it was made, before work starts.
2. Record the actual at completion. Duration, effort, and any scope change, in the same units.
3. Report the ratio, not the miss. “This quarter we ran at 1.4× estimate” is a calibration fact. “We missed three deadlines” is a blame fact.
4. Adjust the next estimate with the ratio. If the team consistently runs at 1.4×, multiply the next bottom-up estimate by 1.4 before publishing.
5. Review outliers individually. A 3× miss usually has a specific cause worth learning; 1.2× is noise.

Velocity data does the same job for agile teams, provided it is real velocity — completed and accepted work — not story points assigned at planning. A team stable at 34 points per sprint, facing a 120-point backlog, is looking at 3.5 sprints. Saying “two sprints” is arithmetic everyone will watch fail.

When is underestimation rational?

Sometimes an optimistic estimate is a deliberate bet: a startup bidding for a contract may quote tight because the alternative is no contract, and a team may commit to a stretch date to force prioritisation.

The honest approach is to separate the plan from the commitment: keep a realistic internal forecast and a communicated commitment, and record the gap. If the gap closes, fine. If it does not, you have the data to explain why.

What you must not do is confuse the two and then treat the commitment as a forecast. That is how a portfolio ends up always “80% done” and never finishing.

How do you track estimates without heavy process?

You need three fields per task: original estimate, actual, and the estimate date. A spreadsheet handles this for a team of ten; a project platform handles it when estimates, actuals, and status live in different places and reconciliation takes a day.

Doitify includes time and effort tracking alongside tasks, sub-tasks, and milestones, plus work and performance reports showing planned versus actual by project — the loop above without an export step. Transparency: Doitify is our product, so treat that as a disclosure rather than an independent review. A disciplined spreadsheet and a monthly review will outperform a tool nobody updates.

The real constraint is behavioural. Someone has to say the number out loud without turning it into a performance review. Teams that can do that improve within a quarter. Teams that cannot will keep discovering the same nine-day task every three days.

Conclusion

The planning fallacy is not a character flaw; it is a predictable feature of how people forecast. The fix is not to try harder or to be more pessimistic. It is to change the question from “how will this go?” to “how did this go the last twelve times?”

Three habits carry most of the benefit. Write the estimate down before the work starts. Record the actual when it ends. Use the ratio to adjust the next estimate, and use a reference class when the work is unfamiliar.

Do that for a quarter and your estimates will still be wrong. They will be wrong in both directions, by smaller amounts, with a number behind them — which is as close to accurate as planning gets.

If this post on planning fallacy was helpful, you might also enjoy Milestone vs Deliverable: What’s the Real Difference? and Free Gantt Chart Template (Excel & Google Sheets).