Joblib generator for subject consistency across different contrast trainings

[uzaymacar]

This PR introduces a script called config_generator.py that takes as input an ivadomed config (JSON) and a datasets path and replaces the following dynamic string params in the base ivadomed config file (config/meanGT_soft.json in this PR):

• <CONTRAST>: The contrast name, you can name it however you like, e.g., T1w, T2w, T1wANDT2w
• <CONTRAST_PARAMS>: The contrast param name, has to match file suffix unlike
• <FNAME_SPLIT>: The .joblib file to use to split the dataset into train-val-test
• <SEED>: The seed to use

This script creates several config files for different contrast and seed scenarios accompanied by .joblib files.

The inputs to the script are:

    --config: Path to the base ivadomed config (JSON) with dynamic string params
    --datasets: List of BIDS dataset folders
    --ofolder: Output folder for .joblib files
    --contrasts: Replaces <CONTRAST>
    --seeds: Replaces <SEED>

Example usage:

python config_generator.py --config config/meanGT_soft.json \
                           --datasets /home/GRAMES.POLYMTL.CA/uzmac/duke/projects/ivadomed/contrast-agnostic-seg/contrast-agnostic-preprocess-all-2022-08-21-final/data_processed_clean \
                           --ofolder joblibs \
                           --contrasts T1w T2w T2star rec-average_dwi \
                           --seeds 42 15 34 98 62

[uzaymacar]

This PR has been updated with a script called config_generator.py and a base config file called config/meanGT_soft.json. The details and usage is now included in the original PR body.

Running the script as shown in the PR body generates the following joblibs folder:

uzmac@romane:~/contrast-agnostic-softseg-spinalcord$ ls joblibs/
'split_datasets_all_seed=15.joblib'  'split_datasets_rec-average_dwi_seed=15.joblib'  'split_datasets_T1w_seed=15.joblib'  'split_datasets_T2star_seed=15.joblib'  'split_datasets_T2w_seed=15.joblib'
'split_datasets_all_seed=34.joblib'  'split_datasets_rec-average_dwi_seed=34.joblib'  'split_datasets_T1w_seed=34.joblib'  'split_datasets_T2star_seed=34.joblib'  'split_datasets_T2w_seed=34.joblib'
'split_datasets_all_seed=42.joblib'  'split_datasets_rec-average_dwi_seed=42.joblib'  'split_datasets_T1w_seed=42.joblib'  'split_datasets_T2star_seed=42.joblib'  'split_datasets_T2w_seed=42.joblib'
'split_datasets_all_seed=62.joblib'  'split_datasets_rec-average_dwi_seed=62.joblib'  'split_datasets_T1w_seed=62.joblib'  'split_datasets_T2star_seed=62.joblib'  'split_datasets_T2w_seed=62.joblib'
'split_datasets_all_seed=98.joblib'  'split_datasets_rec-average_dwi_seed=98.joblib'  'split_datasets_T1w_seed=98.joblib'  'split_datasets_T2star_seed=98.joblib'  'split_datasets_T2w_seed=98.joblib'

The script also generates the following config files:

uzmac@romane:~/contrast-agnostic-softseg-spinalcord$ ls config
'meanGT_soft_all_seed=98.json'  'meanGT_soft_rec-average_dwi_seed=62.json'  'meanGT_soft_T1w_seed=62.json'     'meanGT_soft_T2star_seed=62.json'  'meanGT_soft_T2w_seed=62.json'
'meanGT_soft_all_seed=15.json'   meanGT_soft.json                           'meanGT_soft_rec-average_dwi_seed=98.json'  'meanGT_soft_T1w_seed=98.json'     'meanGT_soft_T2star_seed=98.json'  'meanGT_soft_T2w_seed=98.json'
'meanGT_soft_all_seed=34.json'  'meanGT_soft_rec-average_dwi_seed=15.json'  'meanGT_soft_T1w_seed=15.json'              'meanGT_soft_T2star_seed=15.json'  'meanGT_soft_T2w_seed=15.json'
'meanGT_soft_all_seed=42.json'  'meanGT_soft_rec-average_dwi_seed=34.json'  'meanGT_soft_T1w_seed=34.json'              'meanGT_soft_T2star_seed=34.json'  'meanGT_soft_T2w_seed=34.json'
'meanGT_soft_all_seed=62.json'  'meanGT_soft_rec-average_dwi_seed=42.json'  'meanGT_soft_T1w_seed=42.json'              'meanGT_soft_T2star_seed=42.json'  'meanGT_soft_T2w_seed=42.json'

@ebadrian For fair comparison, it would be great if you can use the same .joblib files. You can use the same script but with your version of the base config file. Let me know if this makes sense!

P.S. I have started training the models and all looks good to me.

Models can be found under: ~/duke/temp/uzay/contrast-agnostic-seg-models