Contrast-agnostic segmentations

[sandrinebedard]

Description

The idea here is to get contrast-agnostic spinal cord segmentation to have the same CSA values between contrasts.

Suggestion

Train DL model on multi-contrast data with a single segmentation.

• Start on spine-generic.
• Segment the spinal cord on all contrasts.
• Register all images to same space using the spinal cord segmentation --> (algo=centerline ; algo=slicereg; NO rotation)
• Apply the transformation on spinal cord segmentations and images (maybe it is not necessary on the images --> to be discussed).
• Get the mean of all segmentations to have a unique soft segmentation.
• Train with SoftSeg.

[sandrinebedard]

UPDATE on the processing pipeline

1. Spinal cord segmentation

Segmentations from T1w, T2w, T2s, MTon, T1w_MT and dwi are used.

2. Registration

All images are regsistered to T2w.

3. Bringing spinal cord segmentation to T2w space

2 options:

• Apply transformation on segmentation:
  • Use interpolation : nearest neighbors
  • Use interpolation : linear and apply a threshhold after
• Rerun sct_deepseg_sc on registered image:
  • Makes it more difficult to perform manual corrections

Results: Calling sct_deeeseg_sc again gives better segmentations --> softseg would be better. Segmentations with sct_apply_transfo are more noisy.

4. Creating softseg

Since all images do not have the same FOV, we have multiple choices for the unique segmentation:

1. Use T2w segmentation on all images.
   • With T2w, CSA is usally overestimated, will propagate this.
2. Use mean of T2w_seg and T1w_reg_seg as a softseg for all images.
   • The softseg will only have 2 values : 0,5 and 1. Not very precise.
3. Use T2w seg + T1w seg on T2w and T2w, for other contrast, use all the segmentations.
   • The concept of using the same segmentation is not really applided here.
4. Use all segmentations, the center of the segmentation will have all contrasts, the extremities will only include T1w and T2w.
   • it requires good alignement of all images, if we include less contrasts in the softseg, it will be easier to exclude a contrast for one subject if it is not included in the segmentation

[jcohenadad]

2 options: Apply transformation on segmentation: Use interpolation : nearest neighbors Use interpolation : linear and apply a threshhold after

If we go with SoftSeg, we actually don't need to apply a threshold, so we can just do linear interpolation and use a soft mask as input for the training (and testing).

[jcohenadad]

3. Bringing spinal cord segmentation to T2w space

Do we absolutely need to do that? If we are treating each contrast (and associated segmentation) independently, then what is the need for registration?

[sandrinebedard]

Do we absolutely need to do that? If we are treating each contrast (and associated segmentation) independently, then what is the need for registration?

The idea here is to use one unique segmentation for all contrasts, an option is to compute the mean of segmentations from various contrasts to create the softseg, so we need registration before we can average all segmentations!

[jcohenadad]

Do we absolutely need to do that? If we are treating each contrast (and associated segmentation) independently, then what is the need for registration?

The idea here is to use one unique segmentation for all contrasts, an option is to compute the mean of segmentations from various contrasts to create the softseg, so we need registration before we can average all segmentations!

ah yes, that's right. With the goal of reducing the contrast-specific bias in segmentation output. Makes sense 👍

[charleygros]

Use all segmentations, the center of the segmentation will have all contrast, the extremities will only include T1w and T2w.

Is it the approach you are leaning to @sandrinebedard ? :-)

[sandrinebedard]

Is it the approach you are leaning to @sandrinebedard ? :-)

@charleygros I think so, I am working on generating that right now! The only thing is that it requires good alignement of all images, if we include less contrasts in the softseg, it will be easier to exclude a contrast for one subject if it is not included in the segmentation! (I should add that point in my comment above 😊)

[charleygros]

Thanks for these insights @sandrinebedard ! Makes sense to me!

'd be great to have some visual inspections of the different options to get additional insights. For instance, I'd be curious to see "how large are the extremities" you are mentioning here:

the extremities will only include T1w and T2w.

[sandrinebedard]

'd be great to have some visual inspections of the different options to get additional insights.

Yes definitely, I will include that! Just to give you a quick idea @charleygros :

Here is the segmentation (from T1w and T2w) as an overlay on T2star

On T1w_MTS:

On dwi:

[sandrinebedard]

UPDATE # 2 on processing pipeline

1. Spinal cord segmentation

Segmentations from T1w, T2w, T2s, MTon, T1w_MT and dwi are used.

2. Registration

Curently testing the registration to T2w space on all the dataset (spine-generic-processed) and adjusting the parameters for each contrast. Will open a PR soon for this!

3. Bringing spinal cord segmentation to T2w space

Apply transformation on segmentation with linear interpolation. (outputs a softseg) Chose this option (can be changed) because the manual segmentation is used in this case.

4. Creating softseg

Currently creating the softseg by averaging all contrasts (see method issue #7 )

I opened an issue about this #8 .

[naga-karthik]

What started as a GitHub issue is now a full paper! We just submitted it to Medical Image Analysis and uploaded the manuscript to ArXiv! 🚀

@sandrinebedard we can close this issue now, what do you think? 😉

[sandrinebedard]

I think it is okay if we close 😅