Designing of the "Translation Rules" File

[yjmantilla]

The "Translation Rules" file is the output of the "DISCOVER RULES" module, which is the one with the heuristics to infer metadata, so this file just is:

the way we represent the rules to infer the metadata

This file should be plain text and human-readable to allow power-user edits.

So what I have thought so far that should be in this file:

• The filepath of the source file usually has information (for example subject id) so a rule should represent this.
• A lot of the technical details are read by mne from the eeg files. So a way to say "this stuff will be inferred by mne upon reading" should exist.
• Similarly mne-inferred stuff should be overwritten, so there should be a way to represent that X property should be "this" instead of what mne thought.
• Maybe support some user-defined rules through python scripting?

To Do

• [x] Characterize what BIDS-info is always and/or optionally inferred with MNE. This could be done by going field by field in the EEG standard and think whether that field is always/optionally inferred from mne or not.
• [x] Design and test the infer-metadata-from-path rule, it is the easiest one in my opinion. So is a good start to get the wheel moving.
• [x] Deal with the minimal fields needed for a valid BIDS dataset. --> See EEG BIDS FIELDS Support in the wiki
• [x] Decide the format of the rules (will it be a dictionary-like , tabular-like) (will it be a yaml file, a json file , a csv file, etc). --> YAML

When the characterization of the fields is done I'm sure we will have more clarity to complete the other task and to even propose more necessary ones.

[yjmantilla]

Oren's proposal

For historical reasons I leave here what @civier initially suggested:

Configuration file format

The file is made of 3 columns:

• Left column gives a “path” to the value that need to be copied to the output BIDS. For example, subject1/eeg/session1/config.csv/1/3 might mean that the information is in the file eeg/session1/config.csv, first row, 3rd column.
• Middle column gives the “path” to the target in the output BIDS file. For example, eeg/eeg_rest_raw/sub-1/ses-rest/eeg/sub-1_ses-rest_coordsystem.json/“Coordinates”/“NAS”/2 might mean that the information should be copied into the file eeg/eeg_rest_raw/sub-1/ses-rest/eeg/sub-1_ses-rest_coordsystem.json in the JSON location “Coordinates” —> “NAS”, 2nd value in array
• Right column is needed if there are several values that need to be combined into one. If might include formulas like $1+$2 which means that the first and second values should be added. If a formula is specified in the 3rd column, the first column should include multiple paths (the first is $1, the second is $2, and so on). The 3rd column can also include Python functions that are required for the calculation.

Of course that the location of files (and location within files) in the BIDS might depend on values in the raw data as well. In that case, also the 2nd column can include $1, $2, etc.

A similar table should also be used in order to indicate where the data files in the raw MEEG format should be copied to, but that is of course simpler, as we assume the files do not need to be combined or altered.

The above is only my suggestion, of course. The notation of the configuration file can be made more efficient (by including global variables) or more sophisticated, as you wish. The important thing is to make it only as sophisticated as necessary. We don’t want to build a universal configuration file, but only one that will serve for MEEG-to-BIDS conversions.

Configuration file format

Example of one row in the table:

Origin location	Target location	Formula
subject1/eeg/session4/config.csv/1/8 subject1/eeg/session4/config.csv/1/9	eeg/eeg_rest_raw/sub-1/ses-rest-ses4/eeg/sub-1_ses-rest_coordsystem.json/“Coordinates”/“NAS”/2	${1} + ${2}

General configuration file format

Example of one row in the table:

Origin location	Target location	Formula
subject1/eeg/session${A}/config.csv/1/8 subject1/eeg/session${A}/config.csv/1/9	eeg/eeg_rest_raw/sub-1/ses-rest-ses${A}/eeg/sub-1_ses-rest_coordsystem.json/“Coordinates”/“NAS”/2	${1} + ${2}

The "Translation Rules" is similar as what @civier imagined as the "General Configuration file".

[yjmantilla]

So, as of now Im thinking yaml > json since:

• it is more readable
• accepts comments which may be useful for configuration files

[yjmantilla]

A validator of the rules file would be nice to implement, so as to assert it is in the correct format.

[yjmantilla]

Closing this, it is done