search

ivadomed / canproco

Code for preprocessing the CanProCo brain and spinal cord dataset
MIT License
4 stars 1 forks source link

Test new contrast-agnostic model #98

Open jcohenadad opened 1 month ago

jcohenadad commented 1 month ago

Following up on https://github.com/ivadomed/canproco/pull/95#issuecomment-2142735201, the purpose of this issue is to compare the performance of the contrast-agnostic model v2.3 vs. v2.4 on PSIR data.

Example:

ezgif-6-3f0c728f1b

The quality of segmentation is definitely better (see upper thoracic cord). Interestingly however, the model seems more "confident" in that there is much less smoothness in sagittal slices adjacent to the spinal cord:

ezgif-6-0174a979d8

Is that a good thing? Further validation required

@naga-karthik

naga-karthik commented 1 month ago

Interesting gifs! thank you for posting them!

could you also post the soft values from the 2nd gif? my guess that the soft values from the v2.3 model are already too soft from the looks of the label (i.e. they will be gone if we binarize with 0.5).

Note that these binarized labels (which don't have segmentations on the slices adj to the cord) were used for training so I think the model learned to output less soft values here

jcohenadad commented 1 month ago

could you also post the soft values from the 2nd gif?

I'm not sure I know what you mean by that. If you meant what threshold I used, the answer is '-thr 0' for both models.

Note that these binarized labels (which don't have segmentations on the slices adj to the cord) were used for training so I think the model learned to output less soft values here

right, good point

jcohenadad commented 1 month ago

Note: the 2.4 model is now part of SCT's master branch: https://github.com/spinalcordtoolbox/spinalcordtoolbox/commit/bb479d82ea1e2076dd50343177056a61bd17e260