Coordinate frame and Fieldtrip import of Neuromag data

[demitau]

I have an issue with coordinate systems when I import a FieldTrip-processed Neurmag MEG. In general I would like to apply SSS. I have a FT-generated .mat file, that has 'neuromag' coord sys of type 'neuromag306'.

When I import it using mne.io.read_raw_fieldtrip, I get sensors' coordinate system to be FIFFV_COORD_HEAD (and MaxFilter does not like it). However if I later save the resulting raw to a file in .fif format and load it again, the sensor coordinate system is now shown as FIFFV_COORD_DEVICE. Even though the values of the 'loc' don't change at all.

I have also noticed that when importing the fieldtrip file, MNE performs some coordinate transformation.

So I guess there is a bug somewhere here -- either coord system is set by MNE wrong when importing or when exporting.

So to get a MaxFilter-compatible coord system, do I have to perform some coord frame conversion, or should I just change the coord system label in the info structure?

MNE version 0.20.3

[larsoner]

@thht any ideas here?

[thht]

hi, this sounds as if you import without providing an info field from the original raw fiff data. if this is the case, you should receive a warning telling you that this will likely result in unusable sensor position data. so in that case, the behavior you report is expected. if i am wrong, please share the code so i can take a look.

[demitau]

@thht you are correct, I don't have access to the raw data info, unfortunately. So you mean that it is impossible then to get the correct positions or that it is not implemented in MNE? If you mean the latter, could you suggest where to find info how to do it? I was only planning to do MaxFilter in MNE, I did not know it is also affected by this warning (perhaps I could have guessed though).

It is still strange that saving to .fif changes the coord system.

[agramfort]

sensor locations don't move in an MEG system. You should try to find any raw file from this system. It should be enough

[demitau]

@agramfort thanks, I'll try!

[thht]

ok, let me go into some details here: one of the main issues when we developed the FieldTrip reader functions about 2 years ago was that FieldTrip and MNE store the position of the head and the sensors differently. while MNE keeps the information (at least when reading it from FIFF files) as original as possible and in device coordinates, FieldTrip converts them to head coordinates and also applies some more processing to that information. both store a transformation matrix to switch between head and device coordinates. FieldTrip also discards some information that MNE wants/needs/keeps, if i recall correctly.

i was on a good track to come up with a solution but then discovered that there are subtle differences depending on the original acquisition system.

we thus decided that we would guarantee correct sensor information only when that information was read from the original source file. if this information was not present, we decided to keep it in its original state (i.e. as read from the FieldTrip structure) but set any transformation matrices to identity.

[thht]

looking towards a possible solution for this case:

yes, it should be possible to recreate this information. however, i am afraid, i think it will take a considerable amount of work to do so because of the different formats of MNE and FieldTrip.

one thing you should check @demitau is whether your FieldTrip structures contain a hdr.orig field. this should store the original, non-processed header information including sensor positions in device coordinates, HPI information (and thus the transformation matrix MNE needs), the sensor types, hopefully as FIFF constants. you can then read the .mat file into python using the pymatreader.read_mat function and construct the Info object using that information.

[larsoner]

FieldTrip converts them to head coordinates and also applies some more processing to that information. both store a transformation matrix to switch between head and device coordinates. FieldTrip also discards some information that MNE wants/needs/keeps, if i recall correctly.

If the dev_head_t is indeed preserved in the FieldTrip output format, it would be great if we could just restore device coords in info['chs'] using the dev_head_t. But I also seem to recall we might have tried this and it didn't work...

[thht]

well... it kind of worked but as i said FieldTrip does some further processing which also, if i recall correctly, is device dependent. i think we decided that there was no sustainable way to guarantee that this is done properly so we decided to go the "provide the original Info object" way.

i would need to dig through the original PR for specifics, though...

[larsoner]

Okay let's not dig into it too deeply then, we both have the same intuition that it's not going to work, that's good enough for me :)

@thht perhaps we should prevent the round-trip bug though. Maybe we shouldn't allow writing FIF files when MEG sensors are in head coordinates. The other option is to make it actually allow writing MEG sensors with head coordinates, which in theory should work but things down the line might need to be fixed (e.g., plot_alignment, forward code, maybe maxwell_filter?) to make it actually work in this case.

In general we could probably make MNE more coordinate-frame-fluid, though, so maybe it's not a bad idea to go this route (we hit problems with sEEG electrodes because we assume head coords, etc.).

@agramfort any opinion?

[thht]

the round-trip bug is actually not a bug. at least i think so. without having looked at the code i assume what is happening is this:

1. the fiff file gets saved with head coordinates and a transformation matrix set to identity
2. when MNE loads the fiff file, some internal check says: "hey, i would like those coordinates in device space" and converts them using the transformation matrix.
3. but this does nothing because the transmat is an identity matrix.

so, i would suggest these options instead:

1. set the transformation matrix so something invalid. this would imply that the data could still be saved and loaded without errors but as the reader function encounters an invalid transformation matrix, it does not transform coordinates.
2. or tell the reader function not to do any transformation if the transmat is identity

[larsoner]

the fiff file gets saved with head coordinates and a transformation matrix set to identity

but this does nothing because the transmat is an identity matrix.

This equates the MEG device and head coordinate frames. Thinking a little bit about what this implies:

• maxwell_filter(..., origin=..., coord_frame='device') will no longer be subject/recording invariant (for the same MEG system) -- they mean something different run-to-run depending on whether your FIF data was save-loaded in FieldTrip or if the original data were loaded in MNE
• Comparing the head positions (in the MEG helmet) across subjects/runs and understanding where they were in the helmet becomes problematic
• If you want to use the destination argument for maxwell_filter or use coord_frame='meg' in plot_alignment, these things will not work as you would expect, because the reference frame is wrong
• I think things like head position estimation and movement compensation would still work, though (I'd have to think harder about it to be sure)

set the transformation matrix so something invalid.

If we do this and it's acutally correct, we could just as easily apply its inverse to the info['chs'][ii]['loc'] to get them to be in the MEG coordinate frame again. This is the best solution, but as we've discussed, it seemed to be problematic for some reason. Whatever reason it was problematic before should still pose problems for setting info['dev_head_t'] now, it just makes it so that we'd hit those problems later.

Maybe it is time to dig a bit into what the problem was. My guess is that perhaps at the time FieldTrip did not retain the dev_head_t once it was applied...? I'll look in a bit.

[larsoner]

Discussion was roughly here, basically we couldn't work out how to get things transformed back. In theory it's as simple as getting the dev_head_t out of the fieldtrip structure (somehow, if it's there) and applying its inverse to the info['chs'][ii][loc]-as-matrix, but based on the gymnastics you already implemented in practice it might not be so easy :)

[thht]

yes, and that only kind of works for Neuromag. i also had a look at the old discussion and especially orientations were a problem. as i already said, FieldTrip does not keep the original information but applies quite some processing and that creates problems when importing to MNE. one could look for a hdr.orig field in the data structure which contains the original, unprocessed header information. but that would be device specific imformation...

bottom line: i still think there is no reliable, sustainable and maintainable way of implementing this. what we could do would be to provide a few examples how to create an info field from a FieldTrip structure. if you only need to consider one system, a specific version of FieldTrip (remember how this was an issue in #7293 !), this is going to be quite straight forward if you have some background knowledge about coordinate frames and how to go back-and-forth between them...