This is the third in a series on cognitive bias and rational irrationality. Read Part I here and Part II here.
In his 1997 Book How the Mind Works, Harvard University psychologist Steven Pinker1 recalls a memory from his teenage years. He was on vacation with his family and his dad remarked that they seemed to be due for some good weather. After all, it had been raining for days. A teenaged Steve rebuked him: didn’t he know, that was the Gambler’s Fallacy! Probability has no memory. Just because you’ve had a streak of bad days doesn’t mean you’re due for a good one.
Except in this case, the elder Pinker was in the right, and the younger, in the wrong. “Cold fronts aren’t raked off the earth at day’s end and replaced with new ones the next morning,” Pinker writes. “A cloud cover must have some average size, speed, and direction, and it would not surprise me (now) if a week of clouds really did predict that the trailing edge was near and the sun was about to be unmasked, just as the hundredth railroad car on a passing train portends the caboose with greater likelihood than the third car.” In other words, in real life, many (if not most) events are not, in fact, independent occurrences. In statistical terms, they are governed by the hazard function: what’s the likelihood that something will happen in a given time interval given that it has not happened already?
In our day to day decision making, Pinker concludes, “an astute observer should commit the gambler’s fallacy and try to predict the next occurrence of an event from its history so far, a kind of statistics called time-series analysis.”
Since I’ve been getting into the weeds on when a fallacy is not a fallacy2, it felt opportune to tackle the one cognitive shortcut that is most likely to lead to horrific decision making at the poker table. So here we are, extolling the virtues of the Gambling Fallacy.
Or are we? A deck of cards has no memory, caring not if you just received a bad beat or not. A coin has no memory of whether it just landed on heads or tails. Outside the realm of gambling, a multiple choice exam has no memory: every answer is independent of the one before. Just because a series of answers happened to be “A” doesn’t mean “A” is any less likely to be the next answer. Of course, an exam is made by a human and humans don’t love streaks, so maybe a prevalence of one answer does mean something about its likelihood on the next question (indeed, some research clearly supports this)—but these days, it’s quite easy to have a computer truly randomize responses.
In fact, one of the first demonstrations of the Gambler’s Fallacy in action was on an exam of sorts: in 1951, UCLA psychiatrist and psychopharmacologist Murray Jarvik ran a study where the stated goal was to have students predict the future. When he said now!, they were to write down either a checkmark or a plus sign. Then, he would say “check” or “plus,” so that they could see if they were correct. The real goal, of course, was not to prove some sort of ESP. It was instead to demonstrate how poorly we do with streaks in data. And that’s exactly what Jarvik found.
Even when, probabilistically speaking, it became clear that “check” should be guessed more often than “plus” (it was always the prevailing answer, with a frequency of 60, 67, or 75%), whenever more than two “checks” appeared in a row, students became less likely to guess “check” on a subsequent round. And as the streak increased, the guessing tendency (and hence, accuracy) decreased apace. Here’s what Jarvik observed; “The interference of the negative recency effect with the overall trend of probability learning is so great after three to four ‘checks’ … that all gains are temporarily obliterated,”—that is, gains in probabilistic learning—“and after four or five ‘checks’ the preponderance of anticipations is in the opposite direction, that is, of plusses to come.” (I, too, had an evil professor who made a True/False section comprised entirely of “True” responses. And, yes, many students changed answers because they thought it couldn’t possibly be right.)
So what does that tell us? With independent phenomena, the Gambler’s Fallacy is fallacious. With continuous phenomena, like the weather patterns observed by Steven Pinker’s dad, it isn’t. Our goal is to figure out which we are dealing with—and the true fallacy is mistaking the former for the latter, as in Jarvik’s crafty example.
And then, of course, even if me do make the correct determination, we are faced with the fact that randomness and uncertainty exist in all elements of life. Even in the realm of truly continuous events, where past behavior actually matters, probability does not behave according to a normal distribution when you’re trying to predict one specific case. On average, sure. But right now? Well, you could be in for a statistical anomaly. Maybe, a week of clouds really does mean the trailing edge is near, given your location and the time of year and the historical data and the like. Usually. But you might also be in a freak weather event where it rains for weeks on end with nary a sight of the sun—and in the absence of additional meteorological data, your past intuitions may not be correct this time around.
The hundredth car of a train is more likely to predict the caboose, yes. But what of a train delay? Do you use the same frame of logic to posit that the next train is surely more likely to come now that you’ve been waiting for so long—or do you conclude that you should leave rather than compound the time already spent waiting (sunk cost fallacy), since there might be a more significant malfunction in play? I’ve certainly been on both sides of the decision, erroneously choosing to remain only to hear the fateful announcement of a line suspension – and leaving a station just to hear my train pull up to the platform.
Which is all to say, probabilistic thinking is a bitch. And when you add to it the bitchiness of everyday existence, that our lives truly do not care about behaving in any predictable patterns, you can see why fallacies may be fallacious or not—or fallacious, but not quite in the way you thought. Good luck to all of us as we try to navigate the constant maze. I hope the rain goes away and the sun is just around the corner.
Disclaimer: he was one my senior thesis advisors at Harvard.