Behavioral Economics Guide 2022, edited by Alain Samson, begins with an essay by Dan Goldstein that offers a chilling reminder of research that only compares some potential outcomes, not the whole range (“The Rise of Applied Behavioral Economics”). It sets the scene:
You are sitting in a workshop in a hotel somewhere in the world. You know the one with the U-shaped table and a dozen people and a bottle of sparkling water for each person. 10 am, someone’s presentation, and you are productively daydreaming. You’re inspired and you know that since it’s 10am, you’ll have the best idea of the day. You have heard something about probability weighting, that is, how people overestimate small probabilities when they are read (as in the gambling studies on which prospect theory was built), but underestimate small probabilities when they experience them (Hertwig et al., 2004). You start thinking about conveying probabilities with visual stimuli. You think that if people see a visualization of probabilities, it will be different from reading about them and experiencing them. Since frequency representations help people with other tasks (e.g., Walker et al., 2022), it is possible that people who view probability visualizations as frequencies will not cause them to either overestimate or underestimate the probabilities they represent. Do you think that if you can find a way to visually display probabilities as arrays of frequency-based icons without using a language or simulation, this could have many applications and improve decision making in other problems such as mortgages, gambling, or investments .
So the idea goes something like this. Show people a grid like this. Ask them to rate the number of black squares, which can be thought of as a way of representing probability (in this case, 24 out of 100, or 24%). It’s not clear what will happen. Will people follow the generally accepted pattern of underestimating smaller probabilities and overestimating larger ones? Or will they be accurate on average in their predictions?
As Goldstein’s story tells, you select multiple values to test this theory, and your friend selects multiple values to test the theory. But when you get together to discuss it, you will find that you have the opposite results! How can this happen? The problem comes from the fact that you and your friend looked at just a few results, not the entire range of possibilities from 0 to 100. When Goldstein et al did a full range study, here’s what they found. Case count estimates were pretty good at low levels below 10; slightly high at around 20; significantly underestimated from about 35 to 55; significantly overestimated from about 65 to 80; slightly underestimated around 90; and then fairly accurate for high levels above 95.
In a hypothetical Goldstein story, imagine that you tried out just a few of the values shown in the black boxes, while your friend tried out just a few of the values in the orange boxes. Each of you will be missing a big piece of the puzzle. Clearly, looking at only a few values can be misleading; Only by looking at all the possible outcomes can a conclusion be drawn. Goldstein writes:
When you test all values from 0 to 100, you get this very strange but very reliable pattern – up, down, up, down, up. I believe it was first discovered by Shuford in 1961 (see also
Hollands and Dire, 2000). Because you tested low values below 20 and high values around 50 and 90, you saw overshoot at low values and underestimate at high values. However, since your friend tested low values around 30 and high values around 70, he saw the opposite, namely underestimation at low values and overestimation at high values. The moral of the story is that looking at the world through the keyholes of a two-tier design can give you a very misleading picture.
Any study that proposes multiple selections from a wider range will run into this potential problem. As a real-life example, consider the problem of a program that aims to encourage people to save for retirement. You want to describe to people the benefits of saving. Wouldn’t it be better to focus people’s attention on the total value of their savings or how much they can get in a month in retirement benefits? The following pattern emerges:
If you ask people what they think is more acceptable for retirement, a lump sum of money or an equivalent annuity, they will often answer that a lump sum sounds more satisfying. For example, people tend to say that a lump sum of $100,000 seems more satisfying than $500 a month for life… After hearing this, people might say, “What’s new? Everyone knows that splitting large amounts into monthly amounts makes them
seem smaller. This is why companies advertise their monthly prices and not yearly prices! That’s why charities are asking you to donate pennies a day!”
However, when you ask about large amounts of money, people find the lump sum less, not more, adequate. For example, $8,000 a month for life sounds better than a lump sum of $1.6 million. What happened to the popular belief that monthly amounts seem smaller? Where is the penny a day effect that everyone knows about?
Again, a study that offers only a few options can be misleading.
The volume includes descriptions of a number of areas of recent research in behavioral economics, a 42-page glossary of behavioral economics terms from to “Action Bias: to the Zero Cost Effect” (with links!), Nd pages of advertisements for graduate programs. in behavioral economics. As an example, I’ll just mention one of the other scholarly discussions on the “FRESH framework,” which applies to whether people exercise self-control to achieve long-term goals, by Kathleen D. Vos and Avni M. Shah. : From the abstract:
[W]We summarized the latest findings and put forward a set of guiding principles called the FRESH framework: Fatigue, Reminders, Ease, Social Influence and Habits. Examples of outcomes considered include doctors writing more opioid prescriptions at the end of the workday than before (fatigue); using digital reminders to encourage people to return to goals, such as for personal savings they may have given up (reminders); visual displays that provide people with data about their behavioral patterns to provide feedback and active monitoring (lightness); the importance of geographically-local peers in changing behavior such as residential water use (social impact); as well as digital and other tools that help people break the link between environmental aspects and problem behaviors (habits).