Suggestion: Write classes for classification and statistics

[thht]

hi, i am currently doing my first analysis in MNE. i got aware that at a lot of steps in the workflow, data is conveniently represented in classes (raw, epochs, average, TFR, etc....). this seems to be lacking for classification and statistics.

as i am going to wrap these two things in classes anyhow to make my life easier, i thought it would be a good idea to ask for your input first because if it is deemed useful by you, i would be happy to contribute the code to MNE.

my current idea is basically to make statistics results a subclass of Average (for evoked data, haven't thought about tfr data yet...). it would need some additional fields but it could basically reuse a lot of functionality, because it is quite similar.

classification seems a bit less straight forward because the "main" result is the scores and one would need to take care of folds, weights and projections.

so, i am curious what you think!

[agramfort]

can you give more details for the first use case. What would the class container actually contain?

[jona-sassenhagen]

I think a lot of this would be covered by #4859

We also have a few functions that store beta values, p values, t values, ... in evoked objects. And we have the new masking functionality in evoked.plot_image.

[thht]

@jona-sassenhagen thanks for pointing out the #4859. i was not aware and i think it would be great to have a more sophisticated and flexible API for stats.

however, #4859 seems to focus on how to define the statistical tests and models. it does have a part that argues for returning/creating objects, providing save and plot methods but as far as i understood, this does not seem the priority.

and i think, it should be easy enough to separate the two things:

1. improve the API of defining the stats.
2. wrapping the stat functions / future functions and their results in classes.

the classes should be designed in a way to either mimic or be agnostic to the API of the underlying statistics. the write/read and viz part would be hardly affected if i am not mistaken.

so, to make a long story short: i would me more than happy to collaborate on this. if somebody is already working on this (i.e. the class abstraction specifically), or has code lying around that he/she would like to share, i would be glad to get in touch with that person.

i will also monitor #4859 and see how i can stay as close as possible when designing my classes.

as i said, i need to do classification and statistics with MNE data in the coming weeks. and i will start working on a personal solution because i hate having to deal with raw arrays of data in my scripts. ;-)

@jona-sassenhagen if you think, i can be of any help, please let me know!

[jona-sassenhagen]

@thht also consider https://github.com/mne-tools/mne-python/pull/5071

[jona-sassenhagen]

https://7545-1301584-gh.circle-artifacts.com/0/html/auto_tutorials/plot_stats_cluster_erp.html

[thht]

@agramfort if it has not become clear so far, i basically want to be able do something like this:

# let's assume: evoked_visual = [average_vis_sub1, average_vis_sub2, ...]
# and: evoked_auditory = [average_aud_sub1, average_aud_sub2, ...]

stats = EvokedStatistics(
                                      {
                                           'visual': evoked_visual,
                                           'auditory': evoked_auditory,
                                       },
                                       stat_function = mne.stats.permutation_cluster_test,
                                       stat_function_kwargs = {'n_permutations=10000'}
    )

stats.plot()

[thht]

@jona-sassenhagen thanks for the links! it has some great examples how to calculate the stats and plot them. i would like to put that logic into classes so users can conveniently use them and scripts look cleaner.

[jona-sassenhagen]

@agramfort will probably suggest that you set up an external repo in mne-sandbox first.

My suggestion is: I think the evoked objects are quite sufficient for plotting purposes. What we need is a smoother way for efficiently getting data into our stats functions, and getting the results back in the right format.

[thht]

sure, as i said i will start developing it as an independent project first and once i think it is good enough to present here, i will do so.

i basically wanted to see whether there is any interest in this kind of code and who might be working on similar things so there can be some coordination and to get some feedback and the idea.

if it is ok with you and the rest of the community, i will just use this issue communicate when i have something and to keep coordinating a bit.

and as i already said, if someone is interested in collaborating, getting the code or whatever, i am open for that!

[jona-sassenhagen]

Sure.

@cbrnr @mmagnuski

[agramfort]

I would suggest to create a repo and add all the interested people here as admin so they can help