Why Your Delivery Average Doesn't Tell the Whole Story
Last night, I ran a small experiment on my Monte Carlo forecaster. I asked it to simulate the delivery date for 79 tasks. In the first run, I gave it a perfectly stable historical throughput: [4, 4, 4, 4, 4]. In the second run, I used a more chaotic but mathematically identical input: [5, 3, 4, 3, 5]. The average number of tasks per sprint was the same, but the results were radically different.
The first simulation returned a single, confident date: September 8th. The second showed a wide distribution of dates, where September 8th was merely the median (a 50% chance). The project was more likely to be delivered in the next sprint, with a pessimistic-but-possible scenario stretching into November. This result made me happy. It proved that my forecaster correctly distinguishes between a simple average, which is just a number, and the complex reality it’s supposed to represent.
What This Experiment Really Shows
The arithmetic mean is seductive. It’s easy to calculate and easy to understand, but it’s a fiction. In the real world, an “average” delivery rate doesn’t exist; only the actual number of tasks completed in a given period exists.
Even though both scenarios averaged four tasks per sprint, the reality of that work can vary immensely. Tasks can differ in size and complexity. The team might be weakened by illness. A specific task might require niche knowledge of a library or algorithm. A multitude of factors influences a team’s pace, and a simple average hides all of this crucial context and variability.
Looking at the second forecast, we might see an 85% probability of finishing by September 22nd. But it also tells us there is a 15% risk that the project could slip into the second half of November. That’s a huge delay. The consequences could be severe, and we cannot afford to ignore that risk, even if the probability seems small.
So, What Should We Do?
If averages are so misleading, how can we approach forecasting? My recommendation is to adopt a more mature, pragmatic mindset based on three principles:
Stop Relying on Single-Point Averages. Acknowledge that an average is a dangerous oversimplification. It hides risk and creates a false sense of certainty. Your first step is to stop using it as your primary tool for planning and making commitments.
Re-forecast Often. With modern tools like a Monte Carlo simulator, forecasting is no longer a time-consuming, one-off event. It’s cheap and fast. You should re-run your forecasts regularly—for instance, after every sprint—feeding the model with the newest data to get an up-to-date picture of the future.
Think and Communicate in Probabilities, Not Certainties. Instead of asking, What’s the date?, start asking, What is the probability we will make it by this date?. Present forecasts as a range of dates with associated probabilities (e.g., We have an 85% chance of finishing between April 15th and May 5th), not as a single, false promise. This builds trust and enables a mature, risk-based conversation with business stakeholders.
Moving from the deceptive simplicity of averages to the valuable clarity of probabilistic forecasting is a key step in data-driven leadership.
If you’re facing challenges with metrics, forecasting, or measuring the true impact of AI in your engineering organization, let’s have a no-commitment conversation.
