[API] Writing other data formats directly into BIDS (e.g. Persyst)

[adam2392]

Describe the problem

So right now, I believe that in order to use write_raw_bids, to process data that is in EEG/iEEG format, you need to be one of the acceptable extensions as defined:

allowed_extensions_eeg = ['.vhdr',  # BrainVision, accompanied by .vmrk, .eeg
                          '.edf',  # European Data Format
                          '.bdf',  # Biosemi
                          '.set',  # EEGLAB, potentially accompanied by .fdt
                          ]

allowed_extensions_ieeg = ['.vhdr',  # BrainVision, accompanied by .vmrk, .eeg
                           '.edf',  # European Data Format
                           '.set',  # EEGLAB, potentially accompanied by .fdt
                           '.mef',  # MEF: Multiscale Electrophysiology File
                           '.nwb',  # Neurodata without borders
                           ]

However, this poses an issue because for say files that you can't convert into one of these intermediary extensions. It's pretty buggy to go into EDF files and to my knowledge there isn't an EDF writer that works robustly. So if I have Persyst files (.lay, .dat file extensions), a common EEG software system in clinics, then if I can read it into Raw via https://github.com/mne-tools/mne-python/pull/8176 then ideally I can just write it into BIDS (and convert to brainvision).

Describe your solution

A way to extend how write_raw_bids can function, such that it allows arbitrary extensions as long as you provide an io function,

or

an addition to the accepted extensions in mne-bids

Describe possible alternatives

I suppose an alternative is for me to use raw.save(...) and then re-load it so that way it looks like it "was" a .fif file, but this seems weird to me.

Additional context

See: https://github.com/mne-tools/mne-python/pull/8176#pullrequestreview-481173721

cc @pmyers16

[jasmainak]

@sappelhoff this issue keeps coming up. Has there been an amendment to the specification? I know in MEG we allow manufacturer formats but EEG does not recognize this.

[adam2392]

Actually I believe this issue isn't "technically" a specification issue persay: https://bids-specification.readthedocs.io/en/stable/04-modality-specific-files/04-intracranial-electroencephalography.html

It says that data when stored in BIDS has to be one of the REQUIRED formats, and RECOMMENDED to be BV or EDF for EEG and iEEG data. However, that doesn't preclude us from adding arbitrary readers to the allowed_eeg_extensions and allowed_ieeg_extensions list that converts automatically to BV.

I propose once the PR for MNE reader from Persyst is finished, that I just add that simple addition to the config file.

[agramfort]

+1

[jasmainak]

Ah, I see you want to write to Brainvision than EDF :)

[sappelhoff]

@sappelhoff this issue keeps coming up. Has there been an amendment to the specification? I know in MEG we allow manufacturer formats but EEG does not recognize this.

MRI: some few manufacturers + formats --> use single format MEG: a few more manufactuers + formats --> allow all formats EEG: a ton of manufacturers + myriad of formats --> restrict allowed formats to subset that covers a large portion of what all people use

that's how it was decided and I don't think it'll be changed .. at least I see nobody pushing for it :slightly_smiling_face:

---

however, re: Adam's PR --> I also think that MNE-BIDS could read any format ... and as long as that format is not a BIDS supported format, the conversion to BrainVision (or FIF) is enforced

but if the format is supported, we do not convert, and simply copy over the raw data :man_shrugging:

[adam2392]

Just to keep this thought documented:

however, re: Adam's PR --> I also think that MNE-BIDS could read any format ... and as long as that format is not a BIDS supported format, the conversion to BrainVision (or FIF) is enforced

So the great thing I didn't realize this was already in the codebase at: https://github.com/mne-tools/mne-bids/blob/f28caec7b58cb9ca406cd8b1a3152879f3d85729/mne_bids/write.py#L1104-L1114

Which I took use of and all I had to do was add an element to the dictionary at: https://github.com/mne-tools/mne-bids/blob/f28caec7b58cb9ca406cd8b1a3152879f3d85729/mne_bids/config.py#L31-L35

Perhaps, we could add to the README a documentation stating that not all readers are supported, but you can add arbitrary readers assuming you make a PR successfully into mne-python?

[agramfort]

go for it