Newsletter Sign-up

There is a saying that goes something like, "The only certain thing about forecasts is that they are always wrong."

It is amazing how many people subscribe to this "wisdom" and just shrug off the entire concept. As humans, most of us hate being wrong. So why would we ever play a game (or do an activity) where we will surely be wrong? How can we overcome this phobia?

How much more accurate do you think you will be on the second shot? Third shot? Fourth?

Does this sound familiar? Does it possibly sound like an analogy for a revenue forecasting process, where the target for the coming year is decided at the end of October?

But how can we learn to change our forecasting processes to improve both our comfort and accuracy? Simple. The same way an aspiring basketball player might get better at taking free throws.

USE A METHODICAL AND DISCIPLINED APPROACH

How would you train someone that has never played basketball before to make free throws? Would you start them on a regulation basketball hoop that is 10-feet tall and only 18 inches in diameter (or about an inch less than twice the diameter of a men's basketball). Probably not the wisest choice.

Probably you would start them with a hoop that is shorter and wider in diameter, or with a smaller ball--all of which would decrease the margin of error involved and increase the odds of the player making shots and gaining confidence.

Would you get an experienced player to teach them the technique of holding a ball, allowing them to observe the elements that might affect the shot (i.e., distance from the hoop, weight of the ball, headwinds and tailwinds) and mentally prepare for the shot beforehand? Probably a wise idea.

To translate this simple analogy to our organizations:

First, we define a clear approach for forecasting and set expectations for following the approach. The approach should include expectations for:

-- clearly defining key variables (both controllable and uncontrollable) that could affect the actual value in question

-- use of internal and external information to inform estimates

-- use of modeling techniques to inform and derive estimates

-- use of other experts (both internal and external) in challenging and validating estimates

-- documenting forecasts and associated rationale and support.

Next, we make it OK for our people to make wider guesses--woops, I mean forecasts. Instead of asking them to estimate what revenue (or customer demand) for widget X will be for 2011 ("give me one number"), we could ask them to tell us what they believe the full range of possible outcomes could be. Then we could ask them what they believe the odds of achieving each value within the range of outcomes to be and why.

Simple, right? Relatively, however, it does take some patience and discipline to do right; no different than learning good technique for shooting a free throw.

PRACTICE OFTEN

Take a lot of shots, constantly compare actual outcomes to estimates, and then adjust your forecasting technique for things we learn.

If people are only making forecasts once a year, then their year-over-year improvement will be limited, at best. What would happen if we asked them each month to forecast the next month's revenues, the revenue over the next three months, as well as the revenue over the next 12 months, following the methodical and disciplined approach described above, while we compared the actual values to the previous period's estimates each month?

First, people would get accustomed to the approach, and it would feel less and less awkward and more and more natural.

Second, they would get better at more clearly articulating their views as a variable and associating those views with probabilities.

Third, by more frequently observing the actual outcomes to their estimates, they would begin to adjust their technique and judgments based on these learnings.

Finally, they would gradually make better and better forecasts, meaning that the actual outcomes would more consistently fall within the estimated distributions. Wherever they fell within the distribution would be consistent with the "quartile explanation" provided by the expert during the forecasting discussion. (Note that I did not say anything about the actual value falling closer to the expert's base case.)

REWARD AND INCENTIVIZE GOOD FORECASTING TECHNIQUE AND OUTCOMES

It is very possible for someone to make a great forecast, do all the right things and still end up with an outcome in the lower quartile of outcomes. It is also very possible for someone to make a bad forecast, do all the wrong things and still end up with an outcome in the higher quartile of outcomes. Therefore, just rewarding good outcomes is not a great approach. Rather, we should:

First, reward people for using good technique in making their forecast. More specifically, if they clearly translate and articulate their expert views on a topic into a robust and reliable form that can be used for business planning and prioritization purposes.

Second, reward them further if they do all the things that they said that they would do.

Third, reward them further still if the actual outcomes turnout to be consistent with their estimated distribution and quartile explanations

Fourth, reward them if the actual outcomes turnout to be in the higher quartiles of estimated outcomes.

This type of incentive structure is even more critical in areas where the actual outcome is largely determined by factors outside of the organization's and employees' control.

DAVID M. WONG is director of enterprise risk management at CME Group, the world's largest and most diverse derivatives exchange.

(Editor's note: Read Part 1 and Part 2 of his column, "Forecast Says 50 Degrees for Next Year" here.)

October 22, 2010

Copyright 2010© LRP Publications