The Problem With Correcting for Multiple Comparisons
The Problem With Correcting for Multiple Comparisons It has been widely considered within the scientific community that the threshold for statistical significance of a value depends on the amount of comparisons being calculated in a dataset. The most popular way to correct for multiple comparisons is the Bonferroni correction, in which the threshold for statistical significance of each comparison is divided by the amount of comparisons being (e.g., the common threshold for statistical significance is p=0.05, but if making 5 comparisons, each value would have to be significant at p=0.01 or less for it to be considered significant at the original threshold). However, correcting for multiple comparisons is problematic, which I will demonstrate. Imagine a study were conducted that calculated the correlation between two 1000 item scales, in a sample size of 100, where the correlation was computed to be 0.35. The p value would far surpass the threshold for statistical significance, at a p va