MEG: CTF data files should be checked

[sappelhoff]

MEG data in CTF format is saved in a directory ending with .ds.

As @romquentin puts it in a bug report to MNE-BIDS:

any CTF tools from the company require that all the files inside the dataset folder exist, plus these specific files ('.acq', '.eeg', '.hc', '.hist', '.infods', '.infods.bak', '.meg4', '.newds', '.res4') need to have the same name as the directory they are in (the dataset name). If we don't rename these files, it means that these BIDS folder are not compatible anymore with most of IO function for importing CTF data.

currently the bids-validator does not check if the files within the .ds directory for CTF data have the same name as the directory.

@jasmainak suggested to solve this using regexps:

I would suggest adding a rule in the regular expressions of bids-validator that checks if the name of the CTF directory matches the files within.

For reference:

https://github.com/mne-tools/mne-bids/pull/162

[sappelhoff]

the '.acq', '.eeg', '.hc', '.hist', '.infods', '.infods.bak', '.meg4', '.newds', '.res4'

are so-called "misc" files for the BIDS-validator, and as such they pass the validateMisc function at some point during the process:

https://github.com/bids-standard/bids-validator/blob/3bd00eda4e54a92d4a7980dd0da150fdfa428e1c/utils/files/validateMisc.js#L1-L20

What about adding a check in there to make sure that for each file that has one of the extensions named above, the file name is equal to the directory name?

For example, this would be a valid CTF dataset:

|-- sub-01_ses-01_acq-01_coordsystem.json
|-- sub-01_ses-01_task-testing_acq-01_run-01_channels.tsv
|-- sub-01_ses-01_task-testing_acq-01_run-01_meg.ds
|   |-- BadChannels
|   |-- ClassFile.cls
|   |-- bad.segments
|   |-- params.dsc
|   |-- processing.cfg
|   |-- sub-01_ses-01_task-testing_acq-01_run-01_meg.acq
|   |-- sub-01_ses-01_task-testing_acq-01_run-01_meg.hc
|   |-- sub-01_ses-01_task-testing_acq-01_run-01_meg.infods
|   |-- sub-01_ses-01_task-testing_acq-01_run-01_meg.meg4
|   |-- sub-01_ses-01_task-testing_acq-01_run-01_meg.newds
|   `-- sub-01_ses-01_task-testing_acq-01_run-01_meg.res4
`-- sub-01_ses-01_task-testing_acq-01_run-01_meg.json

CTF is organized in a directory, where sub-01_ses-01_task-testing_acq-01_run-01_meg.ds is that main directory ... and you can see the the files in there have the correct names, which does not happen if a user erroneously renames their CTF data.

[jasmainak]

yes, that's good. I think this should be done not only for CTF but also for BTi systems. You can check out some examples here in the specification: https://bids-specification.readthedocs.io/en/latest/99-appendices/06-meg-file-formats.html

[sappelhoff]

erm, but if you look at the specification, it doesn't say anything about having to rename BTi files: Apparently renaming the directory is enough

[jasmainak]

okay fair enough