Comparison of MS segmentation models on MP2RAGE images

[Nilser3]

Description

Comparison of seg_ms_lesion_mp2rage and UNIseg, models for segment MS lesions on MP2RAGE images (UNIT1 contrast)

• Experiments performed using SCT v6.3
• To test both models, SC masks were generated using contrast-agnostic-softseg-spinalcord, then a cropping around the SC. Preprocessing pipeline for training and testing based on https://github.com/ivadomed/model_seg_ms_mp2rage/issues/63#issue-1562548573

Preprocessing pipeline

```console subjects=( sub-P002 sub-P016 sub-P021 sub-P037...) bids_basel_input="basel-mp2rage" for subject in "${subjects[@]}" do # Set orient image and lesions mask to RPI sct_image -i $bids_basel_input/$subject/anat/$subject"_UNIT1.nii.gz" -setorient RPI # Generate SC with contrast agnostic model sct_deepseg -i $bids_basel_input/$subject/anat/$subject"_UNIT1.nii.gz" -task seg_sc_contrast_agnostic -o $bids_basel_input/derivatives/labels/$subject/anat/$subject"_UNIT1_contrast-agnostic-SC_seg.nii.gz" # Cropping with dilation (around the SC with 30 pixels in axial plane and 5 pixels in Z) sct_crop_image -i $bids_basel_input/$subject/anat/$subject"_UNIT1.nii.gz" -m $bids_basel_input/derivatives/labels/$subject/anat/$subject"_UNIT1_contrast-agnostic-SC_seg.nii.gz" -o $bids_basel_input/$subject/anat/$subject"_UNIT1_crop.nii.gz" -dilate 30x30x5 # MS lesion segmentation using seg_ms_lesion_mp2rage sct_deepseg -i $bids_basel_input/$subject/anat/$subject"_UNIT1.nii.gz" -task seg_ms_lesion_mp2rage -o $bids_basel_input/derivatives/labels/$subject/anat/$subject"_UNIT1_crop_ivadomed_bin.nii.gz" # Binarizing seg_ms_lesion_mp2rage soft seg sct_maths -i $bids_basel_input/derivatives/labels/$subject/anat/$subject"_UNIT1_crop_ivadomed_seg.nii.gz" -bin 0.5 -o $bids_basel_input/derivatives/labels/$subject/anat/$subject"_UNIT1_crop_ivadomed_bin.nii.gz" # MS lesion segmentation using UNIseg sct_deepseg -i $bids_basel_input/$subject/anat/$subject"_UNIT1.nii.gz" -task UNIseg -o $bids_basel_input/derivatives/labels/$subject/anat/$subject"_UNIT1_UNIseg.nii.gz" done ```

Models

1. seg_ms_lesion_mp2rage (ivadomed) Current benchmark for segmenting MS lesions MP2RAGE. It was trained with ivadomed framework, using basel-mp2rage dataset, with a ensembling/bagging approaches. The soft segmentation will be binarized with a threshold of 0.5. It is implemented on sct_deepseg -task seg_ms_lesion_mp2rage

2. UNIseg Model trained with 3D nnUNetv2 framework, using basel-mp2rage, nih-ms-mp2rage and marseille-3t-mp2rage datasets. The best fold is locally implemented on sct_deepseg -task UNIseg

Details

Fig.01
Learning curves of the best fold of UNIseg training

Fig.02 Comparison of the Dice Score in few test subjects from different centers.

Fig.03 Comparison of the average inference time (seconds) of each model at different centers. Processing using a CPU (Intel(R) Core(TM) i3-1005G1 CPU @ 1.20GHz).

QC

Here the QC for test subjects (Fig.02) Legend

• GT_lesion.nii.gz -> GT
• ivadomed_bin.nii.gz -> prediction of seg_ms_lesion_mp2rage with binarization at 0.5
• UNIseg.nii.gz -> prediction of UNIseg

Since UNIseg performs the same task on the same image modality, but with a higher performance (although a bit slower), we should replace seg_ms_lesion_mp2rage by UNIseg? creating a release?

Related issues

75

[jcohenadad]

we should replace seg_ms_lesion_mp2rage by UNIseg? creating a release?

Agreed. Thank you for the thorough investigation @Nilser3. When doing the release you should link to this issue.