Survivorship Fallacy

(also known as: survivorship bias)

Description: This is best summed up as "dead men don't tell tales." In its general form, the survivorship fallacy is basing a conclusion on a limited number of "winner" testimonies due to the fact we cannot or do not hear the testimonies of the losers. This is based on the cognitive bias called the survivorship bias.

Logical Form:

There are X winners and Y losers.

We only hear the testimonies of the winners.

Therefore, our conclusion is based on X winners.

Example #1:

Let's use the example of automobile accidents since we have relatively good data on these. According to the National Highway Traffic Safety Administration, roughly 32,000 people die each year on the roads in the United States. The number of people involved in fatal accidents is roughly double that amount, meaning that about half of the people survive, and half die. Let's say that 80% of people involved in a fatal accident reach out for supernatural help. Given an even distribution among survivors and non survivors this would mean that 25,600 people who reach out for supernatural help die and 25,600 who reach out for supernatural help live to tell about it. Now we need to add in the 20% of survivors who did not reach out for supernatural help that lived: 6,400. So what we end up with is a group of 32,000 survivors, 80% of whom appear to have been saved by supernatural intervention. Of course, dead men don't tell tales, so we forget about the 80% of those who died and reached out for supernatural help and didn't get it. Because of the survivorship bias, we have a radically biased sample that leads to a fallacious conclusion.

Example #2:

The survivorship bias is used by scammers and con artists who take advantage of the "statistically ignorant" public. One common scam is something I call the "prophetic investor." The scammer will send an e-mail to a very large group of people (say 10 million) with a claim that they have a perfect track record for picking winning investments. But they tell you not to take their word for it, let them prove it you by picking a stock a day for 7 days in a row that increases in value. Then, they say when you are convinced, call them and invest with them. Here's how the scam works:

Day 1: Five different stocks are chosen, and each stock is sent to 2 million people as the winning pick. Let's say three of those stocks make money, and two don't. The 4 million people who received the stock pick that lost money are removed from the list (kind of like dying).

Day 2: Another 5 stocks are chosen. This time, each stock is only sent to the "survivors," about 1.2 million people each get a new pick. Out of that group, perhaps just two stocks are winners. That means that 2.4 million people got the winning stock: two days in a row! You can see where this is headed...

...

Day 7: After the final day of sending, about 100,000 "survivors" remain on the list. These are people who have been sent the winning picks 7 days in a row and are convinced that the "investor" must be legitimate. After all, what are the chances that anyone would pick 7 winning stocks in a row with that much confidence?

Tip: Whether someone is selling you investment services or a religion, think about the survivorship bias and how you might be jumping to an inaccurate conclusion.

References: