Open valosekj opened 1 month ago
bash run_training.sh 1 301 Dataset301_LumbarRootlets
raised the following errors when running nnUNetv2_plan_and_preprocess
:
-------------------------------------------------------
Running preprocessing and verifying dataset integrity
-------------------------------------------------------
Fingerprint extraction...
Dataset301_LumbarRootlets
Using <class 'nnunetv2.imageio.simpleitk_reader_writer.SimpleITKIO'> reader/writer
Error: Shape mismatch between segmentation and corresponding images.
Shape images: (192, 372, 1024).
Shape seg: (125, 417, 85).
Image files: ['/home/GRAMES.POLYMTL.CA/p118175/data/nnunetv2/nnUNet_raw/Dataset301_LumbarRootlets/imagesTr/sub-CTS05_ses-SPpre_T2w_001_0000.nii.gz'].
Seg file: /home/GRAMES.POLYMTL.CA/p118175/data/nnunetv2/nnUNet_raw/Dataset301_LumbarRootlets/labelsTr/sub-CTS05_ses-SPpre_T2w_001.nii.gz
Error: Spacing mismatch between segmentation and corresponding images.
Spacing images: [0.5, 0.29296875, 0.29296875].
Spacing seg: [2.0, 0.6000000238418579, 0.5999999642372131].
Image files: ['/home/GRAMES.POLYMTL.CA/p118175/data/nnunetv2/nnUNet_raw/Dataset301_LumbarRootlets/imagesTr/sub-CTS05_ses-SPpre_T2w_001_0000.nii.gz'].
Seg file: /home/GRAMES.POLYMTL.CA/p118175/data/nnunetv2/nnUNet_raw/Dataset301_LumbarRootlets/labelsTr/sub-CTS05_ses-SPpre_T2w_001.nii.gz
Warning: Origin mismatch between segmentation and corresponding images.
Origin images: (153.98794555664062, 107.47212982177734, 37.20260238647461).
Origin seg: (26.90328598022461, 137.39739990234375, -24.306747436523438).
Image files: ['/home/GRAMES.POLYMTL.CA/p118175/data/nnunetv2/nnUNet_raw/Dataset301_LumbarRootlets/imagesTr/sub-CTS05_ses-SPpre_T2w_001_0000.nii.gz'].
Seg file: /home/GRAMES.POLYMTL.CA/p118175/data/nnunetv2/nnUNet_raw/Dataset301_LumbarRootlets/labelsTr/sub-CTS05_ses-SPpre_T2w_001.nii.gz
Warning: Direction mismatch between segmentation and corresponding images.
Direction images: (-0.9999999926680287, -4.673998152578654e-08, 0.00012109475305146024, 7.87272500310792e-06, -0.9979094409945485, 0.06462775934903711, 0.00012083858162973154, 0.0646277616349623, 0.9979094337952635).
Direction seg: (-0.9999750728554918, 0.0009042621747940346, 0.007002569075477366, -0.0009042399688199635, -0.999999591154876, 6.3321783911189024e-06, 0.0070025722225890786, 1.7274973968599142e-11, 0.9999754816925497).
Image files: ['/home/GRAMES.POLYMTL.CA/p118175/data/nnunetv2/nnUNet_raw/Dataset301_LumbarRootlets/imagesTr/sub-CTS05_ses-SPpre_T2w_001_0000.nii.gz'].
Seg file: /home/GRAMES.POLYMTL.CA/p118175/data/nnunetv2/nnUNet_raw/Dataset301_LumbarRootlets/labelsTr/sub-CTS05_ses-SPpre_T2w_001.nii.gz
Checking the downloaded data:
function pixdim { sct_image -i ${1} -header | grep pixdim }
for file in *nii.gz;do echo $file; pixdim $file;done
T10-S2_RD_LD_sub-CTS09_relabeled.nii.gz
pixdim [1.0, 0.292969, 0.292969, 0.5, 0.0, 0.0, 0.0, 0.0]
T10-S2_RD_LD_sub-CTS15_relabeled.nii.gz
pixdim [1.0, 0.292969, 0.292969, 0.5, 0.0, 0.0, 0.0, 0.0]
T11-S1_RD_LD_sub-CTS04_relabeled.nii.gz
pixdim [1.0, 0.292969, 0.292969, 0.5, 0.0, 0.0, 0.0, 0.0]
T11-S1_RD_LD_sub-CTS05_relabeled.nii.gz
pixdim [1.0, 0.6, 0.6, 2.0, 0.0, 0.0, 0.0, 0.0]
T11-S1_RD_LD_sub-CTS10_relabeled.nii.gz
pixdim [1.0, 0.292969, 0.292969, 0.5, 0.0, 0.0, 0.0, 0.0]
T11-S1_RD_LD_sub-CTS14_relabeled.nii.gz
pixdim [1.0, 0.292969, 0.292969, 0.5, 0.0, 0.0, 0.0, 0.0]
sub-CTS04_ses-SPpre_acq-zoomit_T2w.nii.gz
pixdim [1.0, 0.292969, 0.292969, 0.5, 3.08, 0.0, 0.0, 0.0]
sub-CTS05_ses-SPpre_acq-zoomit_T2w.nii.gz
pixdim [1.0, 0.292969, 0.292969, 0.5, 3.08, 0.0, 0.0, 0.0]
sub-CTS09_ses-SPpre_acq-zoomit_T2w.nii.gz
pixdim [1.0, 0.292969, 0.292969, 0.5, 3.08, 0.0, 0.0, 0.0]
sub-CTS10_ses-SPanat_acq-zoomit_T2w.nii.gz
pixdim [1.0, 0.292969, 0.292969, 0.5, 3.08, 0.0, 0.0, 0.0]
sub-CTS14_ses-SPpre_acq-zoomit_T2w.nii.gz
pixdim [1.0, 0.292969, 0.292969, 0.5, 3.08, 0.0, 0.0, 0.0]
sub-CTS15_ses-SPpre_acq-zoomit_T2w.nii.gz
pixdim [1.0, 0.292969, 0.292969, 0.5, 3.08, 0.0, 0.0, 0.0]
It seems that T11-S1_RD_LD_sub-CTS05_relabeled.nii.gz
has a different pixel size and dimensions than its source image (sub-CTS05_ses-SPpre_acq-zoomit_T2w.nii.gz
):
pixdim [1.0, 0.292969, 0.292969, 0.5, 0.0, 0.0, 0.0, 0.0]
@RaphaSchl, could you please double-check this subject on your side? Thank you.
Hi Jan, Did you mean sub-CTS05 ? From the pixdims shown above and what I've checked, it seems there is an issue with the first 4 pixdims for sub-CTS05's segmentation. It appears saved in a different space - I went back and saved it from Slicer with the correct reference volume this time, what do you think ?
sub-CTS10's segmentation and reference volume both have the same 4 first pixdims, as you showed above. I get the same result when double-checking.
Let me know if anything else comes up !
Did you mean sub-CTS05
Yes, I meant sub-CTS05
. Apologies for confusion. I fixed my comment.
I went back and saved it from Slicer with the correct reference volume this time, what do you think ?
Thank you, Raphaelle! Looks good now!
pixdim T11-S1_RD_LD_sub-CTS05_relabeled.nii.gz
pixdim [1.0, 0.292969, 0.292969, 0.5, 0.0, 0.0, 0.0, 0.0]
Just FYI, I'm going to resolve (hide) the comments as the issue is now resolved.
The training of both models has finished.
TL;DR: predictions of the binary model (Dataset302_LumbarRootlets
) on unseen subjects are quite good.
Interestingly, some rootlets predicted by the model (Dataset202_LumbarRootlets
; https://github.com/ivadomed/model-spinal-rootlets/issues/48) trained on GT created using the FSLeyes were not predicted by the model (Dataset302_LumbarRootlets
) trained on GT created using the Slicer Draw Tube
tool and vice versa.
The predictions of the semantic model on unseen subjects are not good. The rootlets are predicted only the caudal part of the FOV.
The predictions of the binary model on unseen subjects are reasonable; see the example on 2 testing subjects below. The comparison shows predictions obtained using:
Draw Tube
tool) - shown in light blueInterestingly, some rootlets predicted by the older model (202) were not predicted by the new model (302) and vice versa.
Next steps: tweak model training parameters to improve the model, namely:
Looking at nnUNetPlans.json
for the lumbar models:
(the axis order here is: SI, AP, RL)
"3d_fullres": {
"batch_size": 2,
"patch_size": [
64,
112,
320
],
"median_image_size_in_voxels": [
192.0,
372.0,
1023.0
],
"spacing": [
0.5,
0.29296875,
0.29296875
],
The patch_size
is relatively small compared to the median_image_size_in_voxels
.
For example, when comparing with nnUNetPlans.json
for the T2w cervical model; here patch_size
is closer to median_image_size_in_voxels
:
"3d_fullres": {
"batch_size": 2,
"patch_size": [
224,
224,
48
],
"median_image_size_in_voxels": [
320.0,
320.0,
64.0
],
"spacing": [
0.800000011920929,
0.800000011920929,
0.7999992370605469
],
This brings me to the idea that maybe I could try to crop the images around the SC before running the training. The contrast-agnostic model seems to work well! So, the cropping could be done easily.
Training on the cropped images started: binary model (Dataset312_LumbarRootlets
).
[!NOTE] Note that I had to change
"overwrite_image_reader_writer"
toNibabelIO
indataset.json
. For some reason,SimpleITKIO
read the original image dimensions!
Okay, training on the cropped images is done (binary model Dataset312_LumbarRootlets
).
The Mean Validation Dice is 0.360, which is lower than 0.409 for the non-cropped model (binary model Dataset302_LumbarRootlets
). Also, visually, the prediction on an unseen test subject is comparable (or maybe even worse) than for the non-cropped model (binary model Dataset302_LumbarRootlets
):
Preliminary conclusion: training on images cropped around the SC does not increase the segmentation performance. In contrast, it introduces a dependency on the SC seg used for cropping (which is a disadvantage).
I've tested Dataset302 model (non-cropped images) on other subject images. When we compare 3D renderings of sub-CTS13, sub-CTS17, and sub-CTS20 model rootlet segmentations (from FSLeyes), we see continuous rootlets, as we aspire to, for sub-CTS20, and for most sub-CTS13. For CTS17, there is a lot of discontinuity.
I'm not sure how this could be improved, but thought this variability in performance was worth noting. It does not seem correlated to artefacts in the CSF as I initially thought, as sub-CTS20 is quite artefacted but has good model segmentation.
Thank you for testing the model on additional images, @RaphaSchl! The 3D rendering is useful here!
I had a discussion with @naga-karthik, and he suggested using the 96x160x192
patch size for the cropped model instead of 128, 128, 128
. I'll try it!
Okay, the training of the model with 96x160x192
patch size (Dataset322_LumbarRootlets) is done.
I trained this model with the default nnUNet trainer and also with nnUNetTrainerDA5
(extensive data augmentation) and nnUNetTrainer_1000epochs_NoMirroring
(no axis mirroring during the data augmentation) to see whether there is any impact on the performance.
Comparison with the non-cropped model Dataset302_LumbarRootlets on sub-CTS17_ses-SPpre_acq-zoomit_T2w.nii.gz
:
Notes about running the models on other testing images (done by @RaphaSchl -- thank you!):
For CTS03 :
For CTS09 (remarkable!):
For CTS13:
For CTS17:
This issue summarizes model training on T2w lumbar data with relabeled rootlets using the 3D Slicer
Draw Tube
module.This is a follow up of https://github.com/ivadomed/model-spinal-rootlets/issues/48.
0. Data overview
We have 6 subjects with the following labels:
1. Preparing nnUNet folders
details
```bash # imagesTr cp sub-CTS04_ses-SPpre_acq-zoomit_T2w.nii.gz Dataset301_LumbarRootlets/imagesTr/sub-CTS04_ses-SPpre_T2w_001_0000.nii.gz cp sub-CTS05_ses-SPpre_acq-zoomit_T2w.nii.gz Dataset301_LumbarRootlets/imagesTr/sub-CTS05_ses-SPpre_T2w_001_0000.nii.gz cp sub-CTS09_ses-SPpre_acq-zoomit_T2w.nii.gz Dataset301_LumbarRootlets/imagesTr/sub-CTS09_ses-SPpre_T2w_001_0000.nii.gz cp sub-CTS10_ses-SPanat_acq-zoomit_T2w.nii.gz Dataset301_LumbarRootlets/imagesTr/sub-CTS10_ses-SPanat_T2w_001_0000.nii.gz cp sub-CTS14_ses-SPpre_acq-zoomit_T2w.nii.gz Dataset301_LumbarRootlets/imagesTr/sub-CTS14_ses-SPpre_T2w_001_0000.nii.gz cp sub-CTS15_ses-SPpre_acq-zoomit_T2w.nii.gz Dataset301_LumbarRootlets/imagesTr/sub-CTS15_ses-SPpre_T2w_001_0000.nii.gz # labelsTr cp T11-S1_RD_LD_sub-CTS04_relabeled.nii.gz Dataset301_LumbarRootlets/labelsTr/sub-CTS04_ses-SPpre_T2w_001.nii.gz cp T11-S1_RD_LD_sub-CTS05_relabeled.nii.gz Dataset301_LumbarRootlets/labelsTr/sub-CTS05_ses-SPpre_T2w_001.nii.gz cp T10-S2_RD_LD_sub-CTS09_relabeled.nii.gz Dataset301_LumbarRootlets/labelsTr/sub-CTS09_ses-SPpre_T2w_001.nii.gz cp T11-S1_RD_LD_sub-CTS10_relabeled.nii.gz Dataset301_LumbarRootlets/labelsTr/sub-CTS10_ses-SPanat_T2w_001.nii.gz cp T11-S1_RD_LD_sub-CTS14_relabeled.nii.gz Dataset301_LumbarRootlets/labelsTr/sub-CTS14_ses-SPpre_T2w_001.nii.gz cp T10-S2_RD_LD_sub-CTS15_relabeled.nii.gz Dataset301_LumbarRootlets/labelsTr/sub-CTS15_ses-SPpre_T2w_001.nii.gz ```2. Changing label values to be consecutive (this is required by nnUNet)
details
Original values ```bash cd labelsTr for file in *nii.gz;do get_unique_values $file;done [ 0. 19. 20. 21. 22. 23. 24. 25. 26.] [ 0. 19. 20. 21. 22. 23. 24. 25. 26.] [ 0. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27.] [ 0. 19. 20. 21. 22. 23. 24. 25. 26.] [ 0. 19. 20. 21. 22. 23. 24. 25. 26.] [ 0. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27.] ``` Removing label 18 (presented only for two subjects) for now: ```bash sct_maths -i sub-CTS09_ses-SPpre_T2w_001.nii.gz -thr 19 -o sub-CTS09_ses-SPpre_T2w_001.nii.gz sct_maths -i sub-CTS15_ses-SPpre_T2w_001.nii.gz -thr 19 -o sub-CTS15_ses-SPpre_T2w_001.nii.gz ``` Recoding using [recode_nii.py](https://github.com/ivadomed/model-spinal-rootlets/blob/main/utilities/recode_nii.py): ```bash for file in *nii.gz;do python ~/code/model-spinal-rootlets/utilities/recode_nii.py -i $file -o $file;done Unique values in the data: [ 0. 19. 20. 21. 22. 23. 24. 25. 26.] Unique values in the recoded data: [0 1 2 3 4 5 6 7 8] Unique values in the data: [ 0. 19. 20. 21. 22. 23. 24. 25. 26.] Unique values in the recoded data: [0 1 2 3 4 5 6 7 8] Unique values in the data: [ 0. 19. 20. 21. 22. 23. 24. 25. 26. 27.] Unique values in the recoded data: [0 1 2 3 4 5 6 7 8 9] Unique values in the data: [ 0. 19. 20. 21. 22. 23. 24. 25. 26.] Unique values in the recoded data: [0 1 2 3 4 5 6 7 8] Unique values in the data: [ 0. 19. 20. 21. 22. 23. 24. 25. 26.] Unique values in the recoded data: [0 1 2 3 4 5 6 7 8] Unique values in the data: [ 0. 19. 20. 21. 22. 23. 24. 25. 26. 27.] Unique values in the recoded data: [0 1 2 3 4 5 6 7 8 9] ```3. Training
fold1
, 4 training and 2 validation images.Semantic (level-specific) model:
Dataset301_LumbarRootlets
Binary model (all rootlets set to 1):
Dataset302_LumbarRootlets
Binarize labels and modify
dataset.json
: