GAT Grand average

[jona-sassenhagen]

Analogous to #2523, for plotting purposes, a GAT grand average would be useful.

[kingjr]

what do you want to average? multiple gat objects or the ypred of a single gat ?

[jona-sassenhagen]

Multiple GAT objects, analogous to evoked.grand_average and #2537

[dengemann]

What's the gain here? Do we really need this? Insert convincing arguments, we should not add this only to increase the API symmetry.

[kingjr]

For now I would direct users to use the scores_ attributes. Averaging a GAT object will create many difficulties: shall we average the coef or the y_pred of the estimators, if so, how.

However it's true that we don't document how to deal with multiple subjects.

[jona-sassenhagen]

Right now, as-is, I feel as if everything is very fine on a single-subject level, and you can play around with single subjects and get very fascinating results, but once you actually want to do a whole study, you're kind of on your own. (In the end, you DO want a study-wide GAT plot, right?)

This goes for TFR, statistics, ..., and for GAT. And it's not inherently bad to expect users of a new and complex technique to understand it well enough to deal with this, but I also think it could be much more user friendly.

[kingjr]

Right now, as-is, I feel as if everything is very fine on a single-subject level, and you can play around with single subjects and get very fascinating results, but once you actually want to do a whole study, you're kind of on your own.

I agree with you in principle. I would first start with an example/tutorial and then move on to make an MNE class / script.

(In the end, you DO want a study-wide GAT plot, right?)

Absolutely. Especially because it's quite tricky to do statistical inference at the single subject level (see https://github.com/mne-tools/mne-python/issues/1848).

In my opinion, your point actually applies to the whole package which is well integrated for within subjects analyses, but still has some room for improvement regarding multisubjects handling, stats and plots.

[jona-sassenhagen]

In my opinion, your point actually applies to the whole package Indeed, I was thinking of the complete package.

I was talking with @agramfort about an mne.grand_average function in #2537

[kingjr]

ok. I'm wondering whether the best wouldn't be to have a function that transform the GAT object into an epoch and/or an evoked. For linear estimators making unidimensional predictions (most use cases), each decoder can be thought of as a projection/component/set of sources. In this view, gat.y_pred_ would become epochs, and gat.scores_ could be an evoked.

WDYT?

[jona-sassenhagen]

In principle, I think that's a cool idea. But that would mean- we need to come up with a grand average for epchs, like eeglab has, or you still couldn't plot the gat matrix for multiple  subjects - the api would suggest doing strange things like time frequency decomposition of gats - We'd have single-channel evokeda and epochs, which feels a bit odd, and we'd need a special channel type too

[teonbrooks]

glad that this is being discussed. my 2c, I think that the focus of 0.11 should be multi-subject analyses and updating/improving documentation.

[kingjr]

In principle, I think that's a cool idea. But that would mean- we need to come up with a grand average for epchs, like eeglab has, or you still couldn't plot the gat matrix for multiple subjects - the api would suggest doing strange things like time frequency decomposition of gats - We'd have single-channel evokeda and epochs, which feels a bit odd, and we'd need a special channel type too

You're right, this was definitely a bad idea; we do need a separate function.

should be multi-subject analyses and updating/improving documentation.

+∞

[jona-sassenhagen]

my 2c, I think that the focus of 0.11 should be multi-subject analyses and updating/improving documentation.

I like that. Note that corrmap is about multi subject stuff (taking lists of Ica objects). 

[kingjr]

Is there already a generic API format for intersubjects functions?

[agramfort]

Is there already a generic API format for intersubjects functions?

not really.

[jona-sassenhagen]

So maybe it WOULD be better to change #2537 into a generic thing instead of merging now ...

[jona-sassenhagen]

Not to swamp everyone with issues, but should I open a new one for @teonlamont s proposal, to discuss intersubject stuff/list everything we want?

[agramfort]

So maybe it WOULD be better to change #2537 https://github.com/mne-tools/mne-python/pull/2537 into a generic thing instead of merging now ...

+1

[jona-sassenhagen]

2569 is now merged :)

[dengemann]

Right now, as-is, I feel as if everything is very fine on a single-subject level, and you can play around with single subjects and get very fascinating results, but once you actually want to do a whole study, you're kind of on your own. (In the end, you DO want a study-wide GAT plot, right?)

I'm still not convinced this effort is a necessary investment. Why don't you simply aeverage your scores and plot them with a few lines of matplotlib? Mind the API overkill.

[jona-sassenhagen]

That is indeed what I'm doing, but don't you think we should, as with most other things, have a convenient method for newcomers? It'd be more DRY too, as there already is a good function to plot GATs. And there are some plans to improve plotting functions for GATs.

The API overkill would be fairly marginal I hope - as there is a generic grand_average now, all that would be needed is a combine_gat function (that could as well be made private, and never needs to be called by the user for this).

[dengemann]

Ok with me I just want to avoid "geometrical reasoning", deductive motivation of necessities for API dev because we have already X. If we can come up with one easy to learn way to do common things and it simplifies things for newcomers and it will really be used I'm fine with this.

[jona-sassenhagen]

Ah no. For me, it's for comfort - reimporting/rewriting custom GAT average/GAT plot scripts etc is a time and space waster, and makes it harder for my colleagues who're even worse at Python than I am.

I'd implement it myself, but I'm not sure what we can and can't easily average (scores, y_preds) ... Though I guess fundamentally, we should just average what we plot, as we don't do statistics on grand average objects anyways.

[dengemann]

I guess fundamentally, we should just average what we plot, as we don't do statistics on grand average objects anyways.

but then you break the semantics of the object, you would need to introduce another sort of class that makes clear what you can expect to get.

[jona-sassenhagen]

Hm, good point ... @kingjr ?

[agramfort]

I am not convinced we need such a function either.

I prefer to have users that understand numpy / matplotlib than users that need to understand too much custom MNE API code.

[dengemann]

+100

On 10 Nov 2015, at 13:18, Alexandre Gramfort notifications@github.com wrote:

I am not convinced we need such a function either.

I prefer to have users that understand numpy / matplotlib than users that need to understand too much custom MNE API code. — Reply to this email directly or view it on GitHub.

[kingjr]

I would rely on @agramfort and @dengemann's judgements regarding the API.

What I have been missing so far, is more a general set of guidelines on how to structure a group analysis (not just averaging and stats, but also data saving, nice plotting etc) than a new class.

[agramfort]

something I have in mind it open a fullly proprecessed study with full scripts to go up to group analysis.

I'm thinking of doing this with Dan's openfmri data. I "just" need to find the time to clean up my script and make them public...

[kingjr]

I'm (trying to) write up a python notebook with MEG on our orientation decoding, not sure whether that could also serve as a basis.

[agramfort]

any shared material is good

I am also thinking of a notebook based blog with a weekly post...

[kingjr]

weekly post

Ha! You're obviously more optimistic than I am. But yeah, small and regular posts are definitely more useful to the community.

[jona-sassenhagen]

We're not going to do this any time soon are we?