BIDS-iEEG in MNE-bids

[choldgraf]

Hey @jasmainak - I just chatted with @dorahermes and we'd like to figure out a path forward for incorporating BIDS-iEEG support within MNE-python.

I know you guys have been working on this on the MEG side of things. It looks like this repository is mostly for constructing a BIDS-compatible folder for the purposes of automating the conversion process. Is that correct?

Have you guys also been working on any I/O for MNE-python and BIDS? If so, I can loop in and do some work to get this functioning on the iEEG side of things.

[choldgraf]

cc @teonbrooks as I think he was working on this too!

[jasmainak]

Hey @choldgraf ! That sounds great, we should team up for sure. What do you have in mind? Perhaps you can try to edit the GSoC project that Alex created on the ideas list? We might have a candidate for that.

Yep, this repository has so far only been for constructing BIDS-compatible folders. I guess we could expand it too if you have any interesting ideas. The advantage of working on a separate repository than mne-python is that we can iterate faster on the code without too many long discussions.

As far as I/O is concerned, we should also look into what pybids is doing, because I get the impression that they have already quite sophisticated functionality to query BIDS datasets. But I haven't looked into it closely myself.

[choldgraf]

cool - re: IO, I totally agree we should leverage whatever pybids is doing for parsing the folders etc. What did you have in mind for how people would read BIDS folders into MNE? Just have them navigate themselves to the data folder and load it in the old-fashioned way?

[jasmainak]

I guess you could write dedicated readers that use the sidecar json files to populate the info dictionary. Feel free to propose an API :-)

[choldgraf]

Looking at pybids, I think that it'd be best if we recommended people use grabbids to parse the folder structures, and then MNE-python to do the actual loading etc.

I opened an issue on the pybids repo to see if they're planning on supporting all of the specifications, or just the fMRI spec: https://github.com/INCF/pybids/issues/132

[jasmainak]

Yes I have the feeling that some kind of reader like read_raw_bids might be good enough on the MNE end ...

[choldgraf]

yep - you could just start with a rawarray and build up the info from the metadata as you suggest...maybe a good GSOC sub-project

[agramfort]

@choldgraf do you want to sprint with us in Paris when you're around in march?

[choldgraf]

that'd be great! Only trick is timing, since I'm not sure exactly when I'll be free. Tuesday the 6th I'm at polytechnique all day for the event, Wed and Thurs I'm working with Loic and Min somewhere in Paris, but no idea the amount of time that will be blocked off. Maybe we can find a couple hours to meet up in Paris?

[jasmainak]

propose a date that works for you :) I'm available, and yeah prefer Paris too

[choldgraf]

are @jasmainak and @agramfort available on both wed/thurs? or just part of them? I just realized my flight on Fri isn't until ~6pm so I could meet then as well.

[agramfort]

Friday morning would be perfect for me

In telecom so in paris

On 24 Feb 2018, at 17:46, Chris Holdgraf notifications@github.com wrote:

are @jasmainak and @agramfort available on both wed/thurs? or just part of them? I just realized my flight on Fri isn't until ~6pm so I could meet then as well.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[choldgraf]

sweet - Friday AM sounds good to me @ Telecom --

On Sat, Feb 24, 2018 at 9:00 AM Alexandre Gramfort notifications@github.com wrote:

Friday morning would be perfect for me

In telecom so in paris

On 24 Feb 2018, at 17:46, Chris Holdgraf notifications@github.com wrote:

are @jasmainak and @agramfort available on both wed/thurs? or just part of them? I just realized my flight on Fri isn't until ~6pm so I could meet then as well.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mne-tools/mne-bids/issues/26#issuecomment-368242475, or mute the thread https://github.com/notifications/unsubscribe-auth/ABwSHRAJf2dvT9Lppbo9LHsQkBplITtbks5tYEBAgaJpZM4SRljf .

[jasmainak]

okay great, I marked 9:30 AM on my calendar

[choldgraf]

a quick thought here for whenever we get to this in Paris. I see two main to-dos:

1. Figure out what changes need to be made to duplicate the current functionality but for iEEG datasets (e.g. how much does this assume MEG structure?)
2. Consider adding input functionality (AKA, point MNE-BIDS to a subject's folder and it'll return a Raw object with metadata/channel names/etc filled in.)

[teonbrooks]

Figure out what changes need to be made to duplicate the current functionality but for iEEG datasets (e.g. how much does this assume MEG structure?)

right now, raw_to_bids is serving as a general reader. it's set up for MEG but it's easily expendable. I'm currently adding tests to make sure that it handles the test data for each system, and soon, I want to include the bids-validation from https://github.com/INCF/pybids/blob/master/bids/grabbids/bids_validator.py

this will require having the schemas in place for MEG and iEEG

Consider adding input functionality (AKA, point MNE-BIDS to a subject's folder and it'll return a Raw object with metadata/channel names/etc filled in.)

this is an interesting one to tackle. it works fine when there is one run per folder but it gets complicated when there's multiple runs or sessions to automagically parse since the files won't have a generalized naming convention. would be interested if you have a good solution for that

[choldgraf]

Great - it sounds like having a schema is the easiest way to do the validation then? I can put some time into expanding the raw_to_bids function.

Re: reading in, we can always just require that people give the path to a specific data file, and then the reader just parses the metadata in the surrounding files.

[teonbrooks]

Re: reading in, we can always just require that people give the path to a specific data file, and then the reader just parses the metadata in the surrounding files.

ah ok, I thought you wanted just a directory and it just go from there. yeah, at the moment, you just specify the data file and it does the proper munging. I think after we get the validation in place, we can add an mne report from the current print output

[jasmainak]

@choldgraf I like your agenda for our meeting on Friday. Regarding reading in, I just realized that it's a bit more tricky than simply reading in metadata from surrounding files. BIDS has an inheritance principle due to which it might be necessary to merge the metadata files between hierarchies.

[choldgraf]

@jasmainak all the more reason it'd be useful to have a helper function for input :-)

so, how were you imagining people would read data into MNE from BIDS then? Just read in the raw data, then manually import the TSV files and construct a RawArray from that?

[jasmainak]

totally agreed. Just convincing myself too :-)

so, how were you imagining people would read data into MNE from BIDS then? Just read in the raw data, then manually import the TSV files and construct a RawArray from that?

I was thinking more like a dedicated reader that internally inherits from BaseRaw and constructs a raw object. Of course the TSV filenames are easy to know if there is no inheritance. But I'm not sure if that's the case otherwise.

[jasmainak]

So the updated API would probably read:

from mne.io import read_raw_bids
raw = read_raw_bids(raw_fname, tsv_fnames=None, json_fnames=None)
# None would mean you just use the raw_fname to construct tsv_fname and json_fname

[dorahermes]

Thanks so much for thinking about these things! I am not sure what the MNE structure for electrophysiology data looks like, but here are some thoughts from doing similar things in Matlab, and there, automatically reading a directory will immediately raise the following questions:

• If the electrode positions are saved in two different space (e.g. T1w and MNI), which one to select?
• Import events or not? Import stim/physio files or not? I think that it would already be great if there were basic import functions for the metadata files and assign them to the correct fields. (This keeps the option open to extending this, later, to a more extensive automatic reader, that walks through a directory tree and detects files)

sub-01_ses-01_task-..._ieeg.extension sub-01_ses-01_task-..._ieeg.json sub-01_ses-01_task-..._channels.tsv sub-01ses-01..._electrodes.tsv (there can be two of these with: _space-T1w and _space-MNI) sub-01ses-01..._coordsystem.json (there can be two of these with: _space-T1w and _space-MNI)

Then there are optional: sub-01_ses-01_task-..._events.tsv sub-01_ses-01_task-..._stim.tsv.gz sub-01_ses-01_task-..._stim.json sub-01_ses-01_task-..._physio.tsv.gz sub-01_ses-01_task-..._physio.json

[jasmainak]

If the electrode positions are saved in two different space (e.g. T1w and MNI), which one to select?

umm ... I'm not sure. I would say @agramfort or @teonbrooks know better but usually you have a device to head transformation matrix that is stored in the raw object, so as long as these transformations are available it doesn't matter

Import events or not? Import stim/physio files or not?

I think it would be imported as annotations in raw.annotations as we do it in the EEGLAB reader

[teonbrooks]

If the electrode positions are saved in two different space (e.g. T1w and MNI), which one to select?

I think it is likely that it would be T1w space over MNI, but I could be wrong.

Import events or not? Import stim/physio files or not?

what are the different types of event files here; how are the events differ from the stim and physio?

[dorahermes]

_events are onsets/durations/types for a particular stimulus presentation _stim and _physio are continuous recordings such as movies, audio, heart-rate measurements that are synched to the data. These are less common, but we would like to be able to import these in some way in analysis software (if no automatic tool maybe a how-to-manually-import?).

[choldgraf]

yeah I think the challenge of importing this kind of data is that it is super heterogeneous. I wonder if it'd be valuable to extend section 8.6.1 of the BIDS spec to suggest a .json field called "ImportInstructions" or something like that?

[jasmainak]

should we close this?

[choldgraf]

yeah I think we should close this and, once ready, open up another issue specifically about things like importing, now that #37 and #28 have landed