search

tramarobin / fctSnPM

Using spm1d package (v.0.4.3), compute anova and post-hoc tests from anova1 to anova3rm, with a non-parametric approach (permutation tests)
GNU General Public License v3.0
2 stars 2 forks source link

Changed Value of interaction section #17

Closed jinhee615 closed 3 years ago

jinhee615 commented 3 years ago

Hello!

I have 3x3 two-way ANOVA design, And I wonder why interaction section value changed when I run same code twice.

With spm1d code (spm1d.stats.anova2onerm(Y1, Group, condition, subj)) Interaction clusters were found in 0-14%, 18-28%, 85-88%, 96-100% Like Image below. image

When I run fctSPM, Interaction section was different. and it changed over trials. image Upper one is the first trial, and Under one is second trial with exactly same code and same condition. But the result was changed.

My code was spmAnalysis2=fctSPM(Y3,independantEffects, repeatedMeasuresEffects, 'effectsNames', effectNames) for above graphs. Which one is more reliable? And why those differences were occurred?

+ I tried other code spmAnalysis2=fctSPM(Y3,independantEffects, repeatedMeasuresEffects, 'effectsNames', effectNames, 'multiiT', 500 , 'maximalIT', 20000); three times and 3 results were different. image

tramarobin commented 3 years ago

Hi,

It happens when the number of iterations is to small. You should (must) increase the number of iterations with the input 'multiIT' as it defines the number of permutations as multiIT/alpha. Default is 10, corresponds to 200 iterations for 5% risk.

When there are more iterations (the more is the better) the analysis is more reproductible and gives the same results.

By default the number of maximal iterations is 10 000. But you can increase it with the input 'maximalIT'.

It really depends on your computer but in one dimension, 20 000 iterations run quite smoothly.

jinhee615 commented 3 years ago

Even I increase 20 000 iterations, SPM1d and fctSPM exhibits different % section of interaction (I uploaded 3 images at the end of above questions). How can I handle it?

Or do I have to use only fctSPM without SPM1d? if so, can you tell me how to see interaction clusters? (to find exact point of interaction)

tramarobin commented 3 years ago

Oh sorry didn't watch the end of yourmessage.

Classic SPM analysis is parametric while a non parametric approach is used in this function, that can explain the discrepancies between the methods. As the t-value is very close to the threshold and that post-hoc with Bonferonni correction is only approximate, I would interpret the results with caution.

Maybe try with even more iterations but i doubt the results will be really different.

jinhee615 commented 3 years ago

If I want to use only fctSPM, is there any function to check where significant main effect or interaction location? ( exact % point of significance ) and its p-value?

I ve been run Classic spm analysis and fctSPM both. I took main effect and interaction clusterds by SPM, and took post hoc and graphs. But if those differencee appears again, maybe i must choose between one of them, which one is more reliable to me. But I really like fctSPM graphs, I want to use it if it s possible.

And You told me fctSPM use non parametric method, But my data is following timeseries, continuous. Is it appropriate to use fctSPM ?

tramarobin commented 3 years ago

If I want to use only fctSPM, is there any function to check where significant main effect or interaction location? ( exact % point of significance ) and its p-value?

For the ANOVA, you can find location and p-value in spmAnalysis.anova

For the post hoc, you can find location and p-value in spmAnalysis.posthoc

For posthoc, the clusters are not corrected with the result of the ANOVA (only the t-test is considered).

And You told me fctSPM use non parametric method, But my data is following timeseries, continuous. Is it appropriate to use fctSPM ?

The non parametric approach of SPM uses permutation tests, which allow to calculate the threshold (F and t) based on the distribution of the statistical test. If you haven't yet, you can read Nichols TE, Holmes AP (2002). Nonparametric permutation tests for functional neuroimaging: a primer with examples. Human Brain Mapping 15(1), 1–25.

Even if your data are normally distributed, the permutation will calculate and define a threshold close to the parametric approach. However, it can still have differences. This method is appropriate and more flexible than the parametric approach.

I ve been run Classic spm analysis and fctSPM both. I took main effect and interaction clusterds by SPM, and took post hoc and graphs. But if those differencee appears again, maybe i must choose between one of them, which one is more reliable to me. But I really like fctSPM graphs, I want to use it if it s possible.

You should use one methodology and not mix between them. It is the same for every statistical tests, how reporting the "close to significant" is often not easy.

jinhee615 commented 3 years ago

Thank you so much for your kind answer! :)