“[Assuming] ‘what you see is all there is’ facilitates the achievement of coherence and of the cognitive ease that causes us to accept a statement as true… and explains a long and diverse list of biases of judgment… including overconfidence, framing effects, and base rate neglect.” -Daniel Kahneman, Thinking Fast and Slow.
In Part I of this series, we looked at how investors as a group have done a terrible job at forecasting growth, while we also recognized that, like it or not, all investors must implicitly or explicitly make forecasts about the future. In this post, we’ll look at the concept of base rates in making forecasts and how they provide guardrails that help avoid forecasting mistakes.
For us, this comic does a great job of illustrating the challenge at hand.
(Source: Natalie Fratto)
Let’s assume that the illustration shows an investor trying to understand the future of a particular company. A traditional value investor stands on the shore and declares they won’t be tempted into the abyss. They are going to stay on the shore, thank you very much, and leave the reckless exploration for growth investors.
As illustrated by Ed Yardeni’s data shown in Part I, growth investors look at the deep abyss in front of them and cry out in excitement. “The water is crystal clear!” They shout as they peer down to what they think is the bottom. But what they don’t appreciate is they can only see a very small portion of the future, while the vast majority of what lies beneath is hidden from view.
So how then should investors behave? Faced with the comforting, but naïve choice to simply stay on the shore, vs the reckless option to dive into the abyss headfirst, the answer is not immediately clear.
What would you do if faced with a need to explore an uncharted and mostly not visible body of water? If you’re a professional explorer, you plunge in, but only with a significant set of safety measures in place. Tools that you can use to keep yourself from falling off the deep end.
At Ensemble, we look out over the abyss with excitement and the healthy dose of humility that comes from having invested in growing companies for two decades and built firsthand knowledge of the dangers that lie out of sight.
Just as Odysseus had his crew tie him to the mast as they sailed past the Sirens so that he could hear their song but not throw himself to his death, we forecast growth while metaphorically tying ourselves to the shore. The length and types of ropes we use vary over time and between companies, but at its core, our process is built on trying our best to understand the future that is visible, while at the same time accounting for the fact that most of the future is not visible.
One of the most fundamental tools we use are “base rates.” Let’s say you are asked to guess whether it is more likely to rain in Seattle or in Las Vegas on a random day in the future, but you are given no other information to assist you in making your forecast. Most every person who has even the most general sense of American weather patterns would say that it is more likely to rain in Seattle.
This is because while you may not know the exact historical probabilities, you know that Seattle is a city where it rains frequently while Las Vegas is a city that gets little rain. So, the “base rate” of the frequency of rain is higher in Seattle than in Las Vegas. If you take the time to look up this base rate, you’ll find that it rains in Seattle on about 42% of all days, while it rains in Las Vegas about 7% of the time.
The base rate describes the probability of something happening based on the historical record. The “reference class” describes the set of data you are using. Are you using the frequency of rain from last year or the last hundred years? Are you including only the city proper or the entire metro area when measuring rain frequency?
Now let’s say that you are asked what the probability is that it will rain tomorrow in each city, and you are shown a current photo of the sky in each city. If the sky was full of dark clouds in Las Vegas, but clear and blue in Seattle, which city would you say is more likely to get rain tomorrow?
This isn’t an easy question to answer. We know that dark clouds fill the sky before it rains, and that rain doesn’t fall from clear blue skies. But we also know that Seattle gets much more rain in general while Las Vegas gets little. So, it is common sense to try to combine these two sources of data.
In forecasting, the “inside view” refers to the information you have about the specific forecast being made. In this case, the inside view is the information about the conditions of the sky in each city. The “outside view” refers to the base rate data about the historical experience of a relevant reference class, or in this case, the frequency with which it rains in each city.
Most people neglect or underweight outside view information. The dark cloudy skies are such a compelling and tangible indicator of rain, that we discount the base rate probability of rain in Las Vegas being rare. And so most people, when faced with a conflict between inside view and outside view information, go with what the inside view information suggests. After all, they are trying to forecast the likelihood of rain tomorrow, not just the general probability. And look at those dark clouds!
But as Nobel prize winning psychologist Daniel Kahneman explains in the quote opening this post, while inside view information is extremely compelling in helping us get comfortable with a forecast, overweighting this information and neglecting the outside view information of base rates is a primary reason most people are so bad at forecasting.
It is base rate neglect that is the root of Paul Samuelson’s quip made in 1966 that, “Declines in the US stock market have correctly predicted nine of the last five recessions.” Investors take note of inside view information about a potential coming recession but forget that the base rate of recession frequency is relatively low. We explored this concept during the recession scare in the fourth quarter of 2018 in our post Strong Evidence of Rare Events: False Positives in a Sea of Noise.
In the post, we explored the lessons of Bayes Theorem as explained in Nate Silver’s book, The Signal & The Noise: Why So Many[…]