Part 2-5: Overcoming the Heuristics

As we’ve been discussing, the general solution to overcoming statistical heuristics is by estimating the base probability, then making adjustments based on new data. Let’s work through an example.

Julie is currently a senior in a state university. She read fluently when she was four years old. What is her grade point average (GPA)?

People often compute this using intensity matching and representativeness, like so:

• Reading fluently at 4 puts her at, say, the 90th percentile of all kids.
• The 90th percentile GPA is somewhere around a 3.9.
• Thus Julie likely has a 3.9 GPA.

Notice how misguided this line of thinking is! People are predicting someone’s academic performance 2 decades later based on how they behaved at 4. System 1 pieces together a coherent story about a smart kid becoming a smart adult.

The proper way to answer questions like these is as follows:

• Start by estimating the average GPA - this is the base data if you had no information about the student whatsoever. Say this is 3.0.
• Determine the GPA that matches your impression of the evidence. In this case, you might think someone who was fluent when she was 4 years old would have a GPA of 3.9.
• Estimate the correlation between your evidence and GPA. Do you think fluent reading at 4 years old is 100% correlated with academic success in college? Likely not—there are factors that affect reading age that aren’t relevant to college GPA, and vice versa. Is it 10% correlated? 50%?
• FInally, depending on the strength of your correlation, move from the average GPA from the first step to your estimated GPA in the second step. If you believe the correlation is just 10%, then you would move 10% from 3.0 to 3.9, ending up at around 3.1.

This methodical approach is generalizable to any similar prediction task. It avoids overly extreme results from intuition, instead using base rates and assessing the quality of information. It allows for regression toward the mean (eg replace “average GPA and student’s GPA” with “day 1 golf score and day 2 golf score”).

Are Smart Predictions Always Good?

Kahneman notes that absence of bias is not always what matters most. Relying too much on statistics would avoid the prediction of rare or extreme events on shaky information.

For example, venture capitalists make their money by correctly predicting the few companies that will make it big. They lose only 1x their money on a bad investment but make 1000x their money on a Google, so it’s important not to miss out on the next Google. However, using the type of quantitative analysis above might paralyze some investors, if they start with the baseline failure rate of startups (which is very high) and have to adjust upward from that anchor. For some people prone to paralysis, having distorted overoptimistic evidence might be better.

Similarly, sometimes the evidence is against you but your choice feels right, like when you know the high divorce rate as you’re about to get married. In these cases, you might be happier deluding yourself with extreme predictions—”our marriage is going to defy the odds.” Listening to our intuitions is more pleasant and less hard work than acting consciously against them. But at the least, be aware of what assumptions you are making and understand how much you know.