EEG data units in .fif file, mne_browse_raw

[haribharadwaj]

So... I ran into an volts/microvolts issue when using the command line mne_browse_raw along with edf data imported using mne.io... The following makes things go off-scale:

(1) Make RawEDF object from .bdf (biosemi file).. The actual numbers seem to be in volts at this stage.. (2) do raw.save() to a .fif file (3) Read .fif file using mne_browse_raw: The units in which things show up in mne_browse_raw are off by a factor of 1e6. I have to set the display scale to 20e-6 microV i.e., 20e-12 for it show up properly.

(4) Manipulate data in .fif using mne_process_raw/mne_browse_raw and save file again... (5) Load re-saved .fif into python using mne.io: At this stage, the data are off from the original RawEDF object by 1e6.

This doesn't seem to be an issue with mne-python or with mne_browse_raw in the sense that they are both separately self-consistent... When you go across the two, however, the units are not consistent. I think mne_browse_raw expects the EEG data to be in microvolts (not sure what the dafault is for .fif format for EEG data), i.e., the numbers are expected to range from something like +400 to -400, whereas the RawEDF object keeps things in volts.. i.e., the numbers actually range from something like +400e-6 to -400e-6.

Is there anything we can/should do to make the two consistent with each other? Example: raw.save multiplies EEG data by 1e6 before saving to .fif and io.Raw(.) constructor multiplies by 1e-6 when loading from .fif file?

Thoughts?

ping: @t3on, @agramfort

[mshamalainen]

The EEG data in the fif file are always in volts (after multiplication with range and cal in the channel info).

Can you put an example somewhere?

• Matti

On Sep 15, 2014, at 9:32 PM, Hari Bharadwaj notifications@github.com wrote:

So... I ran into an volts/microvolts issue when using the command line mne_browse_raw along with edf data imported using mne.io... The following makes things go off-scale:

(1) Make RawEDF object from .bdf (biosemi file).. The actual numbers seem to be in volts at this stage.. (2) do raw.save() to a .fif file (3) Read .fif file using mne_browse_raw: The units in which things show up in mne_browse_raw are off by a factor of 1e6. I have to set the display scale to 20e-6 microV i.e., 20e-12 for it show up properly.

(4) Manipulate data in .fif using mne_process_raw/mne_browse_raw and save file again... (5) Load re-saved .fif into python using mne.io: At this stage, the data are off from the original RawEDF object by 1e6.

This doesn't seem to be an issue with mne-python or with mne_browse_raw in the sense that they are both separately self-consistent... When you go across the two, however, the units are not consistent. I think mne_browse_raw expects the EEG data to be in microvolts (not sure what the dafault is for .fif format for EEG data), i.e., the numbers are expected to range from something like +400 to -400, whereas the RawEDF object keeps things in volts.. i.e., the numbers actually range from something like +400e-6 to -400e-6.

Is there anything we can/should do to make the two consistent with each other? Example: raw.save multiplies EEG data by 1e6 before saving to .fif and io.Raw(.) constructor multiplies by 1e-6 when loading from .fif file?

Thoughts?

ping: @t3on, @agramfort

— Reply to this email directly or view it on GitHub.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

[teonbrooks]

I know that when you load the data, the reader looks in the header for the unit of the recording. If it is marked in microvolts, this information is stored in the info dict in the unit_mul entry as -6. If it is volt, then it is stored as 1.

[agramfort]

@t3on are you on it?

[teonbrooks]

yes, I am. @haribharadwaj if you send me a sample of your edf data, I will check it out.

[haribharadwaj]

@t3on, A sample file is here: http://nmr.mgh.harvard.edu/~hari/I13.bdf [Note: about 200 MB] Try something like below and open the resulting file in mne_browse_raw... Note that you'll need to add the channels A1 to A32 in channel selection:

raw = edf.read_raw_edf('I13.bdf', n_eeg=34, stim_channel='Status', preload=True)
raw.save('I13_fromPython_raw.fif')

The data only shows up properly in mne_browse_raw when you set the scale to 30e-6 microV ... What I noticed is that the raw.info['chs'] field actually has unit_mul of -6, which is correct... So the issue may be that raw.save(.) not using it?

[mshamalainen]

The C code does not really understand the unit_mul field of the channel info. It should be set to zero, and calibration used correctly instead. The readings after multiplying with cal and rang should be in volts. The unit_mul was more like a vision when the fif format was conceived but never utilized.

• Matti

On Sep 16, 2014, at 6:16 PM, Hari Bharadwaj notifications@github.com wrote:

@t3on, A sample file is here: http://nmr.mgh.harvard.edu/~hari/I13.bdf [Note: about 200 MB] Try something like below and open the resulting file in mne_browse_raw... Note that you'll need to add the channels A1 to A32 in channel selection:

raw = edf.read_raw_edf('I13.bdf', n_eeg=34, stim_channel='Status', preload=True) raw.save('I13_fromPython_raw.fif') The data only shows up properly in mne_browse_raw when you set the scale to 30e-6 microV ... What I noticed is that the raw.info['chs'] field actually has unit_mul of -6, which is correct... So the issue may be that raw.save(.) not using it?

— Reply to this email directly or view it on GitHub.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

[agramfort]

Thanks @mshamalainen for clarifying. Now we know what to do. @haribharadwaj do you take a stab at it?

[haribharadwaj]

@agramfort , sure... So the agreed solution would be to actually render the values in the data matrix to micro volts when saving to .fif. Would it make sense to edit the rawEdf class to add a save method that mostly inherits from baseraw.save but does this extra step?

@t3on , do you see any issues with doing the above?

[agramfort]

@agramfort , sure... So the agreed solution would be to actually render the values in the data matrix to micro volts when saving to .fif.

yes I guess so.

Would it make sense to edit the rawEdf class to add a save method that mostly inherits from baseraw.save but does this extra step?

the more you factorize code the better.

[teonbrooks]

I think it should not pose any problem. From what I gathered is that when we are construct channel dict, instead of on line 599 chan_info['unit_mul'] = unit_mul it should be chan_info['unit_mul'] = None and chan_info['cal'] = cal * unit_mul

-teon

Teon Brooks, Pre-hD NSF Fellow, Chateaubriand Fellow PhD Candidate Department of Psychology New York University

On Thu, Sep 18, 2014 at 3:09 PM, Hari Bharadwaj notifications@github.com wrote:

@agramfort https://github.com/agramfort , sure... So the agreed solution would be to actually render the values in the data matrix to micro volts when saving to .fif. Would it make sense to edit the rawEdf class to add a save method that mostly inherits from baseraw.save but does this extra step?

@t3on https://github.com/t3on , do you see any issues with doing the above?

— Reply to this email directly or view it on GitHub https://github.com/mne-tools/mne-python/issues/1560#issuecomment-56035378 .

[teonbrooks]

Would it make sense to edit the rawEdf class to add a save method that mostly inherits from baseraw.save but does this extra step?

Does the cal and range of the channel info compute the properly scaled data on-the-fly each time the fif data is accessed or is the data scaled already when stored? I was under the impression it was the former.

[mshamalainen]

I am not sure whether you are on the right track. The data in fif files is always in basic units, e.g., volts (after multiplying with cal*range). Microvolts are useful for humans but should in my opinion never used inside the actual analysis code.

• Matti

On Sep 18, 2014, at 4:13 PM, Alexandre Gramfort notifications@github.com wrote:

@agramfort , sure... So the agreed solution would be to actually render the values in the data matrix to micro volts when saving to .fif.

yes I guess so.

Would it make sense to edit the rawEdf class to add a save method that mostly inherits from baseraw.save but does this extra step?

the more you factorize code the better. — Reply to this email directly or view it on GitHub.

The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

[teonbrooks]

sorry, let me try to explain it better. my understanding is that the bdf data can be stored in either uV or V. I had been storing the scaling factor in unit_mul but this is not being used by MNE-C. I was wondering if we should multiply the scaling factor 1e-6 to cal to make it compatible with MNE-C.

[agramfort]

I would say so. Or maybe it should be range... I would say both shoudl work

[haribharadwaj]

It should be range... I just issued a PR #1570 ... I tried cal and that didn't work: It was going the other way and also messing up raw.plot().. Not sure if there is a way to specifically write a test that fails before and passes with the patch.. Testing with the data that I linked in this thread seems to work well.

[agramfort]

close in favor #1570