Segmentation on new trials/raw signals needs to undergo the same preprocessing steps as the trials that participated in training

[mpompolas]

There are two levels of signal datasets in Brainstorm:

1. Link to raw signals
2. Trials

The first one, as the name suggests, is just a link to the raw file that is created by the acquisition system. There are no preprocessing steps on it. This link to the raw file has access to the entire duration of the recording.

Trials on the other hand, are specific segments in time of the recording. They are typically chosen around timings of interest (e.g. like a stimulus presentation). The trials can be created just by chopping the raw recording, or they can be created after the raw files have undergone preprocessing steps (artifact removal, filtering, resampling etc.) Typically, all trials should undergo preprocessing.

Ivadomed training: The training of the Ivadomed model is being done on the trials level. So the preprocessing is already done.

When we have a trained model and we want to segment a new trial, it makes sense that the trials have to undergo the same preprocessing steps for the model to work.

For segmentation of the raw files, since a sliding window is creating new "trials", the same preprocessing needs to be applied.

Sugesstion For all this to work seamlessly with ivadomed, all preprocessing steps will be logged to the config file and then applied to the newly created "trials" that the NIFTIs are based on.

I am already doing something similar by logging the trials filenames, sampling rate of the trials, and the Brainstorm parameters used on the process_ivadomed_create_dataset.m function.

Check:

"brainstorm": {
    "modality": "EEG",
    "event_for_ground_truth": "saw_EST_distant",
    "channel_drop_out": 2,
    "annotations_time_window": [
      -0.1,
      0.3
    ],
    "fs": 100,
    "jitter": 0.5,
    "soft_annotation_threshold": [],
    "bids_folder_creation_mode": "Separate each trial as different subjects",
    "sInputs": [
      "1: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial003_notch_band_resample.mat",
      "2: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial006_notch_band_resample.mat",
      "3: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial018_notch_band_resample.mat",
      "4: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial022_notch_band_resample.mat",
      "5: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial027_notch_band_resample.mat",
      "6: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial038_notch_band_resample.mat",
      "7: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial052_notch_band_resample.mat",
      "8: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial061_notch_band_resample.mat",
      "9: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial066_notch_band_resample.mat",
      "10: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial071_notch_band_resample.mat",
      "11: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial083_notch_band_resample.mat",
      "12: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial111_notch_band_resample.mat",
      "13: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial112_notch_band_resample.mat",
      "14: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial113_notch_band_resample.mat",
      "15: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial123_notch_band_resample.mat",
      "16: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial127_notch_band_resample.mat",
      "17: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial133_notch_band_resample.mat",
      "18: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial137_notch_band_resample.mat",
      "19: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial145_notch_band_resample.mat",
      "20: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial151_notch_band_resample.mat",
      "21: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial157_notch_band_resample.mat",
      "22: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial162_notch_band_resample.mat",
      "23: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial166_notch_band_resample.mat",
      "24: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial174_notch_band_resample.mat",
      "25: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial182_notch_band_resample.mat",
      "26: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial187_notch_band_resample.mat",
      "27: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial207_notch_band_resample.mat",
      "28: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial237_notch_band_resample.mat",
      "29: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial239_notch_band_resample.mat",
      "30: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial244_notch_band_resample.mat",
      "31: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial259_notch_band_resample.mat",
      "32: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial268_notch_band_resample.mat",
      "33: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial271_notch_band_resample.mat",
      "34: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial288_notch_band_resample.mat",
      "35: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial294_notch_band_resample.mat",
      "36: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial303_notch_band_resample.mat",
      "37: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial304_notch_band_resample.mat",
      "38: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial307_notch_band_resample.mat",
      "39: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial313_notch_band_resample.mat",
      "40: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial321_notch_band_resample.mat",
      "41: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial322_notch_band_resample.mat",
      "42: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial338_notch_band_resample.mat",
      "43: sub-pt0090/sub-pt0090_ses-20210722_task-rest_run-03_meg/data_spaced_out_5_trial341_notch_band_resample.mat"
    ]
  }

[mpompolas]

Each trial's history is logged on the config.json file. It can be transferred to a different file at some point in order not to have too much information on the config file.

NOTE: It is up to the user to follow the preprocessing steps that were applied on the training dataset. Preprocessing is not automated