Post-Designation

(also known as: fishing for data)

Description: Drawing a conclusion from correlations observed in a given sample, but only after the sample has already been drawn, and without declaring in advance what correlations the experimenter was expecting to find.

Logical Form:

A sample from a data set is drawn.
A correlation is found that was not looked at nor is it statistically surprising.
The correlation is seen as being meaningful.

Example #1:

In looking at the records of my students, I have found that 9 out of 10 are an only child.  Therefore, society is moving towards one-child families.

Explanation: When you start looking at data with no expectations, anything goes, and any data due to random, statistical anomalies will stand out as “odd”.  In this case, the fact that 9 out of 10 kids don’t have siblings is outside of the norm, but that is the nature of probability.  If you were hypothesizing that most kids don’t have siblings, and you found this data, then it would provide more of a reason to do further research in making a more justified conclusion. 

Example #2:

In looking at the difference between 100 Christians and 100 atheists, we found that Christians were significantly more likely to eat tuna fish.

Explanation: When you fish for data, you are bound to catch something -- in this case, tuna.  Notice that because we were looking for anything, we are bound to find it.

Exception: At times, truth is revealed in data whether we look for it or not, but we need to realize that meaningless statistical anomalies are to be expected when looking at data.

Fun Fact: Post-designation appears to be just a different name for the multiple comparisons fallacy. I have kept both entries because I have expanded the multiple comparisons fallacy to include multiple comparisons between studies, which doesn’t seem to fit the post-designation fallacy’s definition.