search

mne-tools / mne-python

MNE: Magnetoencephalography (MEG) and Electroencephalography (EEG) in Python
https://mne.tools
BSD 3-Clause "New" or "Revised" License
2.7k stars 1.31k forks source link

save epochs to fif #171

Closed agramfort closed 11 years ago

agramfort commented 11 years ago

it would be nice to be able to store epochs in a fif file. there is support for 3d matrices in MNE.

dengemann commented 11 years ago

Yes, that woulde be pretty cool. So does this imply that one can just save 3D matrices to a fiff using the same high-level routines?

On Fri, Nov 2, 2012 at 6:22 PM, Alexandre Gramfort <notifications@github.com

wrote:

it would be nice to be able to store epochs in a fif file. there is support for 3d matrices in MNE.

— Reply to this email directly or view it on GitHubhttps://github.com/mne-tools/mne-python/issues/171.

agramfort commented 11 years ago

Yes, that woulde be pretty cool. So does this imply that one can just save 3D matrices to a fiff using the same high-level routines?

yes. You just need to define a fif tag id (which exists).

dengemann commented 11 years ago

Ok. So what can MNE do with the 3D data. Can mne_browse_raw / men_analye read / display it?

On Fri, Nov 2, 2012 at 7:36 PM, Alexandre Gramfort <notifications@github.com

wrote:

Yes, that woulde be pretty cool. So does this imply that one can just save 3D matrices to a fiff using the same high-level routines?

yes. You just need to define a fif tag id (which exists).

— Reply to this email directly or view it on GitHubhttps://github.com/mne-tools/mne-python/issues/171#issuecomment-10024682.

agramfort commented 11 years ago

Ok. So what can MNE do with the 3D data. Can mne_browse_raw / men_analye read / display it?

no mne_browse_raw and mne_analyze cannot do anything with it currently. but it would just to be able save them to disk. It's usually much smaller than the initial raw data.

larsoner commented 11 years ago

This would be great. Should we just define a new fif tag ID? Looks like the next available integer would be:

FIFF.FIFFB_EPOCHS             = 110

Would this work?

larsoner commented 11 years ago

I was planning on using .mat files to store the data, but this would be cleaner, I think. Especially if we manage to combine these things with the .gz support :)

dengemann commented 11 years ago

Don't see why it shouldn't. I can give it a try.

larsoner commented 11 years ago

Excellent. Maybe I'll give the .gz support a try once you're done with that (and when I'm done with the "science" I have to work on currently).

dengemann commented 11 years ago

I think I'll first finalize the raw_export from ica, which should be fine with the next iteration.

On Tue, Nov 6, 2012 at 6:16 PM, Denis-Alexander Engemann < denis.engemann@gmail.com> wrote:

Don't see why it shouldn't. I can give it a try.

dengemann commented 11 years ago

(and when I'm done with the "science" I have to work on currently).

:) oh yes.

@agramfort : "I'm not doing methods, I'm doing science"

On Tue, Nov 6, 2012 at 6:20 PM, Eric89GXL notifications@github.com wrote:

Excellent. Maybe I'll give the .gz support a try once you're done with that (and when I'm done with the "science" I have to work on currently).

— Reply to this email directly or view it on GitHubhttps://github.com/mne-tools/mne-python/issues/171#issuecomment-10119151.

mluessi commented 11 years ago

I'm looking at the header file from mne C-code, it seems like 110 is already taken, use

 FIFF.FIFFB_EPOCHS             = 122
larsoner commented 11 years ago

I assumed that the next available integer in FIFF.FIFFB would be okay (which would be 110), as opposed to the next available integer across all constants (which would be 122 as you point out), since there are already overlaps between FIFF.FIFFB values and other ones (e.g., FIFFB_EVOKED is the same [104] as FIFF_BLOCK_ID, among other things)...

larsoner commented 11 years ago

But then again I don't completely understand the constants used, I was just trying to infer from the structure of constants.py

mluessi commented 11 years ago

The problem is that the C code has tags defined that were not transferred to Python. I will update constants.py to reflect the C code, so we don't accidentally break things. Yes there, are overlaps, but for the MEG/EEG tags, 110..121 are already used.

larsoner commented 11 years ago

Ahh, I see. That makes sense.

dengemann commented 11 years ago

Btw. as we are discussing persistence topics. I would like to inlcude a save() method for ICA which allows users to save their matrix decomposition and the ICA session. The one could then later intialize an ICA object from the earlier session. This is relevant as decomposing can take some time if you don't do dimension reduction and use the full data set. Shouldn't it be possible to write the mixing / unmixing matrixes in a fit, similar to a covariance matrix?

D

On Tue, Nov 6, 2012 at 7:21 PM, Eric89GXL notifications@github.com wrote:

But then again I don't completely understand the constants used, I was just trying to infer from the structure of constants.py

— Reply to this email directly or view it on GitHubhttps://github.com/mne-tools/mne-python/issues/171#issuecomment-10121398.

mluessi commented 11 years ago

I updated constants.py, see 10edae8e507e19cdae86fad07299c6d84a54ec95 it seems like 113 is available as well.

@dengemann I'm not sure if fiff supports general matrices.. a problem is that unless you define a new tag, you won't know what the matrices actually are.

dengemann commented 11 years ago

Great thanks, it's much more consistent now.

#

What I was thinking was a) either define new tag b) try to 'abuse' exisiting ones c) drop this idea and use e.g. cPickle instead.

On Tue, Nov 6, 2012 at 7:42 PM, Martin Luessi notifications@github.comwrote:

I updated constants.py, see 10edae8https://github.com/mne-tools/mne-python/commit/10edae8e507e19cdae86fad07299c6d84a54ec95it seems like 113 is available as well.

@dengemann https://github.com/dengemann I'm not sure if fiff supports general matrices.. a problem is that unless you define a new tag, you won't know what the matrices actually are.

— Reply to this email directly or view it on GitHubhttps://github.com/mne-tools/mne-python/issues/171#issuecomment-10122235.

mluessi commented 11 years ago

One idea could be to define a tag for pickled python objects.. so we could store python objects in fiff files.. but that would be quite a hack and break comparability with other tools..

dengemann commented 11 years ago

Does anything speak against just pickling the ICA object? Or would you think this is too inconsistent with the mne-python way?

On Tue, Nov 6, 2012 at 8:12 PM, Martin Luessi notifications@github.comwrote:

One idea could be to define a tag for pickled python objects.. so we could store python objects in fiff files.. but that would be quite a hack and break comparability with other tools..

— Reply to this email directly or view it on GitHubhttps://github.com/mne-tools/mne-python/issues/171#issuecomment-10123390.

mluessi commented 11 years ago

The problem with pickling is that if the code changes in the future, you won't be able to unpickle. So unless we are very sure that the code doesn't change, it is not a good idea.

agramfort commented 11 years ago

I am on it:

https://github.com/mne-tools/mne-python/pull/175

at least I started...