Store ICA matrixes in fiff file

[dengemann]

Our current ICA class allows to save sources in a fiff file. Also saving the ICA matrixes in a fiff file would be very nice for different reasons, e.g. saving the entire ICA session / sharing between programs but also sharing with colleagues and finally toggling between mixed and unmixed in regular sensor space raw object. We already started discussing that earlier but I think our discussion didn't converge satisfyingly. As we now have epochs.save for some time this one came back to mind. I would like to create a new tag, e.g. FIFF_MNE_MISC_DATA : 123 and then save the matrixes using wirite_float_matrix.

Wdyt?

[agramfort]

you should write in more like a noise cov that stores also to what channels the ICA mixing matrix applies. Otherwise +1. See mne/fiff/cov.py

[dengemann]

we the could init an ica session from saved ica matrixes. wdyt?

On 08.12.2012, at 14:40, Alexandre Gramfort notifications@github.com wrote:

you should write in more like a noise cov that stores also to what channels the ICA mixing matrix applies. Otherwise +1. See mne/fiff/cov.py — Reply to this email directly or view it on GitHub.

[agramfort]

we the could init an ica session from saved ica matrixes. wdyt?

if it solves a relevant problem yes :)

[dengemann]

e. g. a staticmethod ICA.load() that takes fname as argument and returns an instance of ICA

wdyt?

On 08.12.2012, at 15:28, Alexandre Gramfort notifications@github.com wrote:

we the could init an ica session from saved ica matrixes. wdyt?

if it solves a relevant problem yes :) — Reply to this email directly or view it on GitHub.

[agramfort]

read_ica is more consistent with other objects. And static methods are confusing for people.

ica = read_ica(frame) ica.save(fname)

is the convention we have for other objects.

[dengemann]

On 08.12.2012, at 15:38, Alexandre Gramfort notifications@github.com wrote:

read_ica is more consistent with other objects. And static methods are confusing for people.

both true. would then be like read_cov

I like that

ica = read_ica(frame) ica.save(fname)

is the convention we have for other objects. — Reply to this email directly or view it on GitHub.

[dengemann]

so basically i would store all ints and strings and matrixes needed to re-init the session. just that we cannot really restore the internal pca and ica from scikit. i then would bind at least mixing and unmixing and the related pca arrays to self so pca and ica can be discarded after decomposition.

[dengemann]

... i think we even could restore self._pca and self._ica just by also saving their params and the attributes set during decomposition...

[agramfort]

indeed it could be a pain to serialize _ica and pca objects unless you only store the PCA matrices components etc. but that's a bit of work...

[dengemann]

Would storing the _p/ica matrixes require a tag for each matrix? Serializing the rest wouldn't be so hard I gues, we could dump the params in a string like for Epochs using FIFF_DESCRIPTION and then on calling read_ica just initialize new sessions for the parameters and then just setting attributes which aren't created by init. To me it seems part one might be more effortful, but let's see. Definitely worth some effort.

[mluessi]

Would it make sense to add a named matrix tag? It would allow us to easily save custom matrices without having to create new fif tag every time.

[agramfort]

+1

[dengemann]

That would be pretty brilliant!

On Sun, Dec 9, 2012 at 4:35 PM, Martin Luessi notifications@github.comwrote:

Would it make sense to add a named matrix tag? It would allow us to easily save custom matrices without having to create new fif tag every time.

— Reply to this email directly or view it on GitHubhttps://github.com/mne-tools/mne-python/issues/241#issuecomment-11172040.

[dengemann]

What do we have to do to add such a tag, presumably just adding it to mne.FIFF won't be enough. Or is it just that?

On 09.12.2012, at 16:35, Martin Luessi notifications@github.com wrote:

Would it make sense to add a named matrix tag? It would allow us to easily save custom matrices without having to create new fif tag every time.

— Reply to this email directly or view it on GitHub.

[dengemann]

@agramfort 're-ping'! What do we need to create a named tag?

[agramfort]

@agramfort 're-ping'! What do we need to create a named tag?

I would really copy the cov way with a fif block containing FIFF_MNE_ROW_NAMES and/or FIFF_MNE_COL_NAMES + the data.

[dengemann]

Sure, makes sense for now. So for full serialization we would save exaplainedvariance and components_ from _pca and unmixing matrix from _ica. I then would need three new tags or can I separate the data by using different blocks. That's the reason why I'm asking for the named tags...

On Sun, Dec 9, 2012 at 9:37 PM, Alexandre Gramfort <notifications@github.com

wrote:

@agramfort 're-ping'! What do we need to create a named tag?

I would really copy the cov way with a fif block containing FIFF_MNE_ROW_NAMES and/or FIFF_MNE_COL_NAMES + the data.

— Reply to this email directly or view it on GitHubhttps://github.com/mne-tools/mne-python/issues/241#issuecomment-11175610.

[agramfort]

you can create tags starting with FIFF_MNEICA in constants.py

[dengemann]

... ok, great. just looked into cov.py I would dare committing myself to getting this working before 0.5 ;-)

On Sun, Dec 9, 2012 at 10:02 PM, Alexandre Gramfort < notifications@github.com> wrote:

you can create tags starting with FIFF_MNEICA in constants.py

— Reply to this email directly or view it on GitHubhttps://github.com/mne-tools/mne-python/issues/241#issuecomment-11175923.

[agramfort]

... ok, great. just looked into cov.py I would dare committing myself to getting this working before 0.5 ;-)

oh yes. that's not a trivial task to do it right.

[dengemann]

Yeah, but should be doable and would be very nice to have I think. We'll see how it goes.

On Sun, Dec 9, 2012 at 10:07 PM, Alexandre Gramfort < notifications@github.com> wrote:

... ok, great. just looked into cov.py I would dare committing myself to getting this working before 0.5 ;-)

oh yes. that's not a trivial task to do it right.

— Reply to this email directly or view it on GitHubhttps://github.com/mne-tools/mne-python/issues/241#issuecomment-11175976.

[dengemann]

I'm currently designing an ICA.save function and I'm wondering what to do with the pre_whitener which is either a float value or an instance of mne.Covariance; in the latter case I could try to save the covariace into the ica_matix.fif or just keep the file name and hope the covariance matrix will be present. The first feels somehow wrong, because we already have a covariance matrix fif. The second feels wrong because it's just too fidgety. Any thoughts?

[dengemann]

... on the other hand side just trusting the user he / she will take care of the covariance supplied would make saving ICA much easier.

[dengemann]

Ok, froget about it; it's late and the fiff file is somewhat mind-bending. As I just have to save the result of compute_whitener all should be fine.

[dengemann]

wrong issue... deleting.

[dengemann]

Continuing on WIP PR.