Nan in corrected pvalue when conducted pairwise_tests post hoc for 2 way repeated measure ANOVA

[hql471]

Hello there,

I am not sure if this a bug or a feature. I am conducting post hoc analysis for 2 way repeated ANOVA and I noticed that I receive nan pvalue if I use p value adjustment. I noticed that on the example in pingouin.pairwise_tests nan p-corr and p-adjust also showed up. (row 3 in table bellow) I am wondering if this is a bug or feature of post hoc pair wise analysis ? What would be the cause and solution for this ?

Thank you so much for your time. Cheers.

[hql471]

After diving into the source code, I figure out where the problem lies in: Minimal reproducible code:

import pandas as pd import pingouin as pg pd.set_option('display.expand_frame_repr', False) pd.set_option('display.max_columns', 20) df = pg.read_dataset('mixed_anova.csv') posthocs = pg.pairwise_tests(dv='Scores', within='Time', subject='Subject', between='Group', padjust='bonf', data=df)

In library file pairwise.py:

If testing between just 2 group, than p-value won't be corrected. If pairwise testing between (August, January, June) then p-value will be corrected, as in row 0, 1, 2 in question table. If pairwise testing between just 2 groups Control and Meditation (row 3 in question the table), then p-value won't be corrected, hence p-corr and p-adjust is NaN.

I believe method for correcting p-value suppose to take into account the total amount of test (for our example should be 7), at least that what I know about Bonferroni method. For my question, i think the solution shoud applying multicomp function right after pg.pairwise_tests(data = df ....) to obtain corrected p-value.

Thank you so much.

[raphaelvallat]

Hi @hql471,

Apologies about the slow reply. Yes, you are correct. The correction is applied separately for each contrast ("Time", "Group" and interaction "Time * Group"). Since you only have two groups in the "Group" contrast, Pingouin does not apply a correction. Now, if you prefer to correct for all the tests at once, then you will indeed need to apply the multicomp function on the uncorrected p-values.

Closing this issue but please feel free to reopen.

Thanks, Raphael