SC segmentation using contrast-agnostic model

[plbenveniste]

On branch plb/sc_seg-contrast-agnostic Issue to document work on spinal cord segmentation using the contrast-agnostic model.

Related to #46

As discussed with @valosekj, I am doing this to have the code in Python (@valosekj is doing it in .sh).

Parameters used:

• for PSIR : multiplication by -1 and --crop-size 100x320x320
• for STIR : --crop-size 100x320x320

[valosekj]

Hold on; I believe there is no need to reimplement the same thing to Python.

[plbenveniste]

That's true. But since I had already done it, I might as well add it to the code. Also, in my opinion, in makes more sense to have everything in Python (instead of having both .py and .sh code)

[plbenveniste]

Optimisation idea: as mentionned in issue 46 : could work to change the parameter to 64 x 192 x -1

Experiment results: great quality overall for both PSIR and STIR.

Results of QC:

Big issues : required manual correction (20 subjects / 446)

- sub-cal149_ses-M0_STIR.nii.gz - sub-edm075_ses-M0_PSIR.nii.gz - sub-mon014_ses-M0_PSIR.nii.gz - sub-mon032_ses-M0_PSIR.nii.gz - sub-mon097_ses-M0_PSIR.nii.gz - sub-mon104_ses-M0_PSIR.nii.gz - sub-mon107_ses-M0_PSIR.nii.gz - sub-mon118_ses-M0_PSIR.nii.gz - sub-mon138_ses-M0_PSIR.nii.gz - sub-mon148_ses-M0_PSIR.nii.gz - sub-mon152_ses-M0_PSIR.nii.gz - sub-mon168_ses-M0_PSIR.nii.gz - sub-mon191_ses-M0_PSIR.nii.gz - sub-tor092_ses-M0_PSIR.nii.gz - sub-tor133_ses-M0_PSIR.nii.gz - sub-van159_ses-M0_PSIR.nii.gz - sub-van176_ses-M0_PSIR.nii.gz - sub-van182_ses-M0_PSIR.nii.gz - sub-van189_ses-M0_PSIR.nii.gz - sub-van206_ses-M0_PSIR.nii.gz

Minor correction (small points outside of the sc): (8/446)

- sub-mon077_ses-M0_PSIR.nii.gz - sub-mon096_ses-M0_PSIR.nii.gz - sub-mon137_ses-M0_PSIR.nii.gz - sub-mon190_ses-M0_PSIR.nii.gz - sub-van148_ses-M0_PSIR.nii.gz - sub-van168_ses-M0_PSIR.nii.gz - sub-van171_ses-M0_PSIR.nii.gz - sub-van184_ses-M0_PSIR.nii.gz

For the minor correction, often, a part of the eye is included in the segmentation : this is easily solvable by changing the cropping regions.

As for the bigger problems, sometimes it is linked to poor image quality, sometimes there is a hole in middle of the segmentation (as in the sc is segmented entirely expect between certain vertebral levels)... However, something that we can note is that it happens almost only for PSIR and majorly for sub-mon et sub-van subjects (currently investigating this).

The QC is available on ~/duke/temp/plben/sc_seg_canproco_contrast_agnostic/qc_folder_2

[jcohenadad]

The QC is available on ~/duke/temp/plben/sc_seg_canproco_contrast_agnostic/qc_folder_2

If not too big (<10MB) could you please upload a ZIP of the QC? If not no worries I'll look into duke.

[plbenveniste]

Its about 100 MB.

[jcohenadad]

In general, I notice that the edge slice is not well segmented, example:

Could it be because of:

• resampling before inference
• cropping towards the edge?
• anything else?

[jcohenadad]

Another observation, which points to a possible bug, is that the slice on the right is better segmented than the one on the left:

Maybe an issue with resampling?

[jcohenadad]

idea from @plbenveniste : flip RL and re-run the inference as a quick check to see if there is an asymmetry bias in the inference

[jcohenadad]

For the minor correction, often, a part of the eye is included in the segmentation : this is easily solvable by changing the cropping regions.

that's another argument for adding post-processing (https://github.com/sct-pipeline/contrast-agnostic-softseg-spinalcord/issues/73)

[plbenveniste]

Spinal cord segmentation was performed using x, y, z flip and sum of the 4 files (the original one as well). The output was saved before and after binarization (in case we want to modify the segmentation mask before binarization). Inference was performed: it took a long time: around 15 hours (25 sec per inference; 4 per subjects; 446 subjects).

The QC folder is available at : ~/duke/temp/plben/sc_seg_canproco_contrast_agnostic/qc_folder_flip_xyz

Visual observation of QC :

Problems with the following files

- sub-cal149_ses-M0_STIR.nii.gz : small points outside the spinal cord - sub-cal154_ses-M0_STIR.nii.gz : chunk forgotten (because of artifact) - sub-edm075_ses-M0_PSIR.nii.gz : chunk forgotten - sub-edm105_ses-M0_PSIR.nii.gz : small points outside the spinal cord - sub-edm183_ses-M0_PSIR.nii.gz : small points outside the spinal cord - sub-mon009_ses-M0_PSIR.nii.gz : (weird seg: need to see what happened) - sub-mon014_ses-M0_PSIR.nii.gz : chunk forgotten - sub-mon032_ses-M0_PSIR.nii.gz : no segmentation - sub-mon052_ses-M0_PSIR.nii.gz : small points outside the spinal cord - sub-mon077_ses-M0_PSIR.nii.gz : small points outside the spinal cord - sub-mon096_ses-M0_PSIR.nii.gz : small points outside the spinal cord - sub-mon097_ses-M0_PSIR.nii.gz : (weird seg: need to see what happened) - sub-mon104_ses-M0_PSIR.nii.gz : chunk forgotten - sub-mon118_ses-M0_PSIR.nii.gz : (weird seg: need to see what happened) - sub-mon129_ses-M0_PSIR.nii.gz : small points outside the spinal cord - sub-mon137_ses-M0_PSIR.nii.gz : small points outside the spinal cord - sub-mon148_ses-M0_PSIR.nii.gz : (weird seg: need to see what happened) - sub-mon168_ses-M0_PSIR.nii.gz : (weird seg: need to see what happened) - sub-mon190_ses-M0_PSIR.nii.gz : small points outside the spinal cord - sub-mon191_ses-M0_PSIR.nii.gz : no segmentation - sub-tor079_ses-M0_PSIR.nii.gz : small points outside the spinal cord - sub-tor097_ses-M0_PSIR.nii.gz : small points outside the spinal cord - sub-tor124_ses-M0_PSIR.nii.gz : small points outside the spinal cord - sub-tor133_ses-M0_PSIR.nii.gz : no segmentation - sub-van065_ses-M0_PSIR.nii.gz : small points outside the spinal cord - sub-van148_ses-M0_PSIR.nii.gz : small points outside the spinal cord - sub-van159_ses-M0_PSIR.nii.gz : chunk forgotten - sub-van164_ses-M0_PSIR.nii.gz : small points outside the spinal cord - sub-van168_ses-M0_PSIR.nii.gz : small points outside the spinal cord - sub-van171_ses-M0_PSIR.nii.gz : small points outside the spinal cord - sub-van176_ses-M0_PSIR.nii.gz : bad segmentation (likely because of image quality: to be investigated) - sub-van184_ses-M0_PSIR.nii.gz : small points outside the spinal cord - sub-van189_ses-M0_PSIR.nii.gz : chunk forgotten - sub-van201_ses-M0_PSIR.nii.gz : small points outside the spinal cord - sub-van206_ses-M0_PSIR.nii.gz : bad segmentation (likely because of image quality: to be investigated)

To summarize this : only 35/ 446 files (8%) have segmentation issues. Among them, 19 have points outside the spinal cord which are segmented : easily solvable using keep_largest_object function (@valosekj). For the rest, 6 have a chunk of the spinal cord which is forgotten and 10 have major segmentation issue.

After investigation: the 10 major segmentation issues come from image quality issues : see issue 53.

[jcohenadad]

I confirm that segmentation results look much better with "flip_xyz". In general, much less false negatives. Below are four representative examples at each site, comparing the previous method (single inference) with the "flip_xyz" method (for posterity):

Example results

   

So

• Should we move forward with this approach, with the risk of introducing an over-segmentation bias for future training?
  • On that topic, should we explore avenues to slightly erode the output segmentation so that it matches the segmentation of 'averaged' test time (vs. summed).
• Should we further explore DA at test time to further improve segmentation results?