maximtrp
commented
1 year ago Hello! Thank you for this, but no bug here. This is due to the specifics of these corrections. I suggest spending some time on delving into the code and mathematics of p adjustments, if you are interested.
You can also use R to double-check:
p.adjust(c(0.97972236, 0.00210181, 0.00192951), method="holm")
[1] 0.97972236 0.00578853 0.00578853
I'm not sure if it's a bug, but while learning statistics I noticed that, for example, in the Dunn test, the Holm-Bonferroni correction causes incorrect final p values. In the code below, the result p for comparing B with C is mistakenly copied to compare A with C. Removing p_adjust='holm' shows the correct result (without correction, of course). Other adjustments showing a doubling of the p-value are : "bonferroni","holm", "holm-sidak", "simes-hochberg", "fdr_bh", 'fdr_by', 'fdr_tsbh', 'fdr_tsbky'.
Dataset `import pandas as pd from scipy.stats import kruskal from scikit_posthocs import posthoc_dunn
data = { 'ID': ['A1', 'A2', 'A3', 'A4', 'A5', 'A6', 'A7', 'A8', 'A9', 'A10', 'A11', 'A12', 'A13', 'A14', 'A15', 'A16', 'A17', 'A18', 'A19', 'A20', 'A21', 'A22', 'A23', 'A24', 'A25', 'A26', 'A27', 'A28', 'A29', 'A30'], 'Parameter': [12, 14, 46, 18, 20, 52, 24, 26, 38, 19, 29, 32, 31, 30, 62, 24, 23, 11, 29, 31, 48, 21, 10, 50, 14, 13, 52, 20, 15, 41], 'Group': ['A', 'B', 'C', 'A', 'B', 'C', 'A', 'B', 'C', 'A', 'B', 'C', 'A', 'B', 'C', 'A', 'B', 'C', 'A', 'B', 'C', 'A', 'B', 'C', 'A', 'B', 'C', 'A', 'B', 'C'] }
df = pd.DataFrame(data)
statistic, p_value = kruskal(df[df['Group'] == 'A']['Parameter'], df[df['Group'] == 'B']['Parameter'], df[df['Group'] == 'C']['Parameter'])
print(f"Kruskal-Wallis p-value: {p_value}")
posthoc_results = posthoc_dunn(df, val_col='Parameter', group_col='Group', p_adjust='holm')
print("\nDunna's Test Results:") print(posthoc_results)`
To Reproduce Steps to reproduce the behavior:
Result Dunna's Test Results: A B C A 1.000000 0.979722 0.005789 <- this value is incorrect B 0.979722 1.000000 0.005789 <- this value is correct C 0.005789 0.005789 1.000000
after removing correction <- this is correnct A B C A 1.000000 0.979722 0.002102 B 0.979722 1.000000 0.001930 C 0.002102 0.001930 1.000000
Expected behavior Dunna's Test Results: A B C A 1.000000 0.979722 0.004204 B 0.979722 1.000000 0.005789 C 0.004204 0.005789 1.000000
System and package information (please complete the following information):