search

0todd0000 / spm1d

One-Dimensional Statistical Parametric Mapping in Python
GNU General Public License v3.0
63 stars 21 forks source link

spm1d feature requests #45

Open 0todd0000 opened 8 years ago

0todd0000 commented 8 years ago

The following functionality is being developed for future versions of spm1d:

STATISTICS

GENERAL

COMPLETED

The following features (previously listed above) are available in spm1d 0.4 (released 01 October 2016):

Please reply to this post or create a new issue if you have any additional feature requests.

bernard-liew commented 8 years ago

Hi Todd,

I was wondering if Bayesian Inference will be part of future versions?

Regards, Bernard

0todd0000 commented 8 years ago

Hi Bernard, Yes, absolutely. I'll add it to the list above. Todd

bernard-liew commented 8 years ago

Hi Todd,

Not sure if you are still considering other functions, totally random but came to my mind when doing a review. If more and more studies are reporting waveforms and if data are made publicly available, then meta-analytical methods of waveforms should be something for the future? And test of things like heterogeneity, meta-regression, sub-group analysis etc would be something that I need is useful?

Many thanks, Bernard

0todd0000 commented 8 years ago

Hi Bernard, thanks very much for the suggestion! I'll add it to the list above. If enough public 1D data are available meta-analysis would indeed be possible. It might take a few studies to develop a general software tool, but it could definitely useful. Thanks!

Todd

romainmartinez commented 7 years ago

Hi Todd, Post hoc tests and factorial MANOVA are my main my expectations ; also, I was wondering if other implemented p-value corrections are planned, such as Holm–Bonferroni, Šidák or FDR ?

0todd0000 commented 7 years ago

Hi, thank you for the feedback! Yes, a variety of corrections including the Holm–Bonferroni and Šidák corrections will be considered along with the listed "post hoc tests" as a high priority for development.

FDR procedures are already implemented, but require a bit of clean-up prior to public release; "FDR" wasn't previously included in the list above, but it's now there, thank you for the suggestion!

Factorial MANOVA will indeed also become available, but I'm not yet sure how comprehensive the post hoc procedures will be for MANOVA.

jabou356 commented 7 years ago

Hi Todd,

I was wondering if a N-way repeated measure ANOVA with muscles as one of the repeated factors was equivalent to a N-1 way MANOVA. If it is not equivalent, do you think it is a good alternative as factorial MANOVA are not available yet in your toolbox?

Thanks Jason

0todd0000 commented 7 years ago

Hi Jason,

They're similar but not equivalent. RM ANOVA makes more assumptions than MANOVA makes, and there are correspondingly fewer parameters estimated in RM ANOVA, implying that it's more powerful than MANOVA. There's a discussion on this topic here: https://stats.stackexchange.com/questions/13197/differences-between-manova-and-repeated-measures-anova and an internet search for "manova vs. repeated measures anova" should yield a number of other relevant discussions.

Two-way MANOVA is in-development for spm1d but will likely not be available until 2018 or later. If you're after N-way MANOVA functionality (N > 1), then spm1d can only really be used for approximate post hoc analysis.

Cheers, Todd

jabou356 commented 7 years ago

Thanks!

kmshort commented 6 years ago

Hi Todd,

I'd just like to +1 to the ANOVA unbalanced designs. I've posted in stack overflow about nested ANOVA, and having an unbalanced nested ANOVA would be great for the work I do where the observations are inherently unequal. John McDonald talks about the caveats for an unbalanced nested ANOVA here: http://www.biostathandbook.com/nestedanova.html but also offers an "excel" spreadsheet for the implementation of a "Satterthwaite approximation" for changing the means squares at each level in an unbalanced situation.

cheers, Kieran

0todd0000 commented 6 years ago

Hi Kieran,

Thank you for voting up this feature. I've moved unbalanced designs up toward the top of the list to prioritize its development for future versions of spm1d.

The Satterthwaite approach is indeed useful, and is relatively straightforward to implement for 0D data analysis. For 1D data analysis the necessary correction is more complex, and I believe that the preferred approach in the SPM literature is Parametric Empirical Bayes (PEB) as described here: Chapter 12: Random Effects Analysis I previously tried to implement PEB but encountered a few difficulties, and I'm not sure how long it will take to get a valid, stable version.

If you need to run unbalanced design analysis now please consider the following two options:

  1. Use SPM12, which supports unbalanced designs using PEB

  2. Use a non-parametric procedure in spm1d.stats.nonparam. The F statistic will not be correct for this analysis, because as you point out the MS terms are generally not correct without a correction. Regardless, in non-parametric permutation analysis it's generally valid to use any test statistic you like, so the F statistic as computed should be fine. If you try to use the code as is a similar unbalanced error will be generated as a warning that the F statistic is incorrect, but it should be possible to modify the source code as needed.

Todd

thomas-neitmann commented 6 years ago

It is mentiod above that FDR is already implemented but "requires a bit of clean-up prior to public release". Is it ready for usage by now? I'd like to use it for adjusting p-values for scalar field post hoc test after conducting a vector field test.

0todd0000 commented 6 years ago

The Python version of FDR is 95% complete, but the MATLAB version had not yet been started. Both should be ready for release by summer 2019, when a related paper is ready for submission. Alternatively, if you're interested in writing your own code, note that FDR routines don't require very many commands, and the two approaches below can be readily adapted to 1D data:

Chumbley JR, Friston KJ. False discovery rate revisited: FDR and topological inference using Gaussian random fields. Neuroimage. 2009 Jan 1;44(1):62-70.

Genovese CR, Lazar NA, Nichols T. Thresholding of statistical maps in functional neuroimaging using the false discovery rate. Neuroimage. 2002 Apr 1;15(4):870-8.

thomas-neitmann commented 6 years ago

@0todd0000 Thank you for the references! I finally had a chance to read them. Am I right that in order to implement the FDR one would need to extract the p-Values of all clusters (including non-significant ones) and then procede as "usual" in the case of multiple testing in the scalar case?

0todd0000 commented 6 years ago

I don't think FDR can be used with clusters, because p values are undefined when there are no clusters. Instead try assembling uncorrected p values for each continuum node. If the data array shape is (J,Q) where J is sample size and Q is the number of continuum nodes, then there will be Q separate p values, which are produced by separately conducting common 0D tests at each node.