Error for T1 dataset

[kartik7737]

Describe the bug I am running the spineps for t1 dataset, it runs smoothly for t2_dataset but giving the error assert.affine errors for zoom, origin and shape of the segmentation generated and input MR. the error report on the terminal: (/physical_sciences/spineps/spineps-env) [kkumar@papr-res-gpu01 spineps]$ spineps dataset -i "/physical_sciences/spineps/Temp_data/Dataset/dataset-T1" -model_semantic T1w_Segmentor -model_instance Inst_Vertebra_3.0 -smrm -ibf -imc ───────────────────────────────────────────────────── Thank you for using SPINEPS ────────────────────────────────────────────────────── Please support our development by citing GitHub: https://github.com/Hendrik-code/spineps ArXiv: https://arxiv.org/abs/2402.16368 Thank you! ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── [Models] Check available models... Namespace(cmd='dataset', directory='/physical_sciences/spineps/Temp_data/Dataset/dataset-T1', raw_name='rawdata', model_semantic='T1w_Segmentor', model_instance='Inst_Vertebra_3.0', ignore_bids_filter=True, ignore_model_compatibility=True, save_log=False, save_snaps_folder=False, der_name='derivatives_seg', save_debug=False, save_softmax_logits=False, save_modelres_mask=True, override_semantic=False, override_instance=False, override_postpair=False, override_ctd=False, ignore_inference_compatibility=False, nocrop=False, non4=False, cpu=False, run_cprofiler=False, verbose=False) /physical_sciences/spineps/spineps-env/spineps/spineps/utils/predictor.py:85: FutureWarning:

You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.

[] Model loaded from /physical_sciences/spineps/spineps-env/spineps/spineps/models/T1w_Segmentor [] Model loaded from /physical_sciences/spineps/spineps-env/spineps/spineps/models/Inst_Vertebra_3.0 [SPINEPS] Initialize setup for dataset in /physical_sciences/spineps/Temp_data/Dataset/dataset-T1 [SPINEPS] (Modality.T2w, Acquisition.sag): model incompatible, model modalities [Modality.T1w] [SPINEPS] Processing dataset in /physical_sciences/spineps/Temp_data/Dataset/dataset-T1 [SPINEPS] Found 19 Subjects in /physical_sciences/spineps/Temp_data/Dataset/dataset-T1, parents=['rawdata', 'derivatives_seg']

[SPINEPS] Processing 1 / 19 subject: 12a [SPINEPS] Subject 1: 12a had no scans to be processed

[SPINEPS] Processing 2 / 19 subject: 12b [SPINEPS] Subject 2: 12b had no scans to be processed

[SPINEPS] Processing 3 / 19 subject: 14a [SPINEPS] Processing sub-14a__20190827_Sag_T1_Spine_Gd_Comp_scan_1_T1w.nii.gz

• [SPINEPS] Input image (0.4167, 0.4167, 3.3000002) ('P', 'I', 'R') (770, 1467, 14)
• [SPINEPS] Prepare input image ---- [SPINEPS] Shifted from range (0, 1398) to range (0, 1398) ---- [SPINEPS] N4 Bias field correction done in 26.40070692449808 sec ---- [SPINEPS] Shifted from range (0, 980) to range (0, 980)
• [SPINEPS] Predict Semantic Mask 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 21/21 [00:03<00:00, 5.77it/s] 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 21/21 [00:01<00:00, 11.90it/s] 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 21/21 [00:01<00:00, 11.91it/s] ---- [SPINEPS] Post-process semantic mask... ---- [SPINEPS] Cleaned (label, n_components) {41: 3, 42: 1, 43: 1, 44: 4, 46: 4, 47: 3, 48: 6, 61: 2, 100: 3} ---- [SPINEPS] Found 5 unique connected components in semantic mask ---- [SPINEPS] semantic_bounding_box_clean did not remove anything ---- [SPINEPS] Fill holes called
• [SPINEPS] Save /physical_sciences/spineps/Temp_data/Dataset/dataset-T1/derivatives_seg/output_raw_T1w/sub-14a_mod-T1w_seg-spine-raw_msk.nii.gz as uint8
• [SPINEPS] Predict instance mask ---- [SPINEPS] Image is already rotated to ('P', 'I', 'R') ---- [SPINEPS] Image resampled from (0.8571, 0.8571, 3.3) to voxel size (0.75, 0.75, 1.65) ---- [SPINEPS] Cleaned (label, n_components) {1: 2} ---- [SPINEPS] Same height, just merge [1, 101, 2, 102, 3, 103, 4, 104, 5, 105, 6, 106, 7, 107, 10, 111, 12, 116, 14, 118, 16, 120, 19, 124, 22, 126, 23, 125, 21, 123, 20, 122, 18, 121, 17, 119, 15, 117, 13, 113, 11, 109, 9, 108, 8, 112, 114, 115] ---- [SPINEPS] Same height, just merge [1, 101, 2, 102, 3, 103, 4, 104, 5, 105, 6, 106, 7, 107, 10, 111, 12, 116, 14, 118, 16, 120, 19, 124, 22, 126, 23, 125, 21, 123, 20, 122, 18, 121, 17, 119, 15, 117, 13, 113, 11, 109, 9, 108, 8, 114, 115] ---- [SPINEPS] Same height, just merge [1, 101, 2, 102, 3, 103, 4, 104, 5, 105, 6, 106, 7, 107, 10, 111, 12, 116, 14, 118, 16, 120, 19, 124, 22, 126, 23, 125, 21, 123, 20, 122, 18, 121, 17, 119, 15, 117, 13, 113, 11, 109, 9, 108, 8, 115] ---- [SPINEPS] Vertebra Body predictions: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 23/23 [00:26<00:00, 1.13s/it] ---- [SPINEPS] Cleaned (label, n_components) {9: 4, 11: 2, 16: 1, 17: 1, 23: 15, 24: 14} ---- [SPINEPS] Merged predictions into vert mask ---- [SPINEPS] Fill holes called ---- [SPINEPS] Predicted 24 vertebrae
• [SPINEPS] Save /physical_sciences/spineps/Temp_data/Dataset/dataset-T1/derivatives_seg/output_raw_T1w/sub-14a_mod-T1w_seg-vert-raw_msk.nii.gz as uint8 Traceback (most recent call last): File "/physical_sciences/spineps/spineps-env/bin/spineps", line 8, in sys.exit(entry_point()) ^^^^^^^^^^^^^ File "/physical_sciences/spineps/spineps-env/spineps/spineps/utils/citation_reminder.py", line 21, in wrapper func_result = func(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/physical_sciences/spineps/spineps-env/spineps/spineps/entrypoint.py", line 161, in entry_point run_dataset(opt) File "/physical_sciences/spineps/spineps-env/spineps/spineps/utils/citation_reminder.py", line 21, in wrapper func_result = func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/physical_sciences/spineps/spineps-env/spineps/spineps/entrypoint.py", line 293, in run_dataset process_dataset(kwargs) File "/physical_sciences/spineps/spineps-env/spineps/spineps/utils/citation_reminder.py", line 21, in wrapper func_result = func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/physical_sciences/spineps/spineps-env/spineps/spineps/seg_run.py", line 179, in process_dataset output_paths, errcode = process_img_nii( ^^^^^^^^^^^^^^^^ File "/physical_sciences/spineps/spineps-env/spineps/spineps/utils/citation_reminder.py", line 21, in wrapper func_result = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/physical_sciences/spineps/spineps-env/spineps/spineps/seg_run.py", line 464, in process_img_nii seg_nii_back.assert_affine(other=input_nii) #changed to ignore the matching of raw_mask with nifti image as can be corrected outside ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/physical_sciences/spineps/spineps-env/lib/python3.11/site-packages/TPTBox/core/nii_poi_abstract.py", line 90, in assert_affine other_match = self.assert_affine( ^^^^^^^^^^^^^^^^^^^ File "/physical_sciences/spineps/spineps-env/lib/python3.11/site-packages/TPTBox/core/nii_poi_abstract.py", line 151, in assert_affine raise AssertionError(f"assert_affine failed with {found_errors}") AssertionError: assert_affine failed with ['zoom mismatch (0.8571, 0.8571, 3.3), (0.4167, 0.4167, 3.3000002)', 'origin mismatch (11.4587356, 147.2670258, -16.7962786), (24.6032639, 145.598526, -18.8539352)', 'shape mismatch (378, 717, 22), (770, 1467, 14)']

I think there is some problem in running the pre-processing of T1 MR images which changes the resolution , shape and origin of the input mr and assert.affine function is failing.

[kartik7737]

[SPINEPS] Processing sub-14a__20190827_Sag_T1_Spine_Gd_Comp_scan_1_T1w.nii.gz

• [SPINEPS] Input image (0.4167, 0.4167, 3.3000002) ('P', 'I', 'R') (770, 1467, 14)
• [SPINEPS] Prepare input image ---- [SPINEPS] Shifted from range (0, 1398) to range (0, 1398) ---- [SPINEPS] Crop down from (770, 1467, 14) to (769, 1467, 14) ---- [SPINEPS] N4 Bias field correction done in 26.054202798753977 sec ---- [SPINEPS] Shifted from range (0, 980) to range (0, 980) ---- [SPINEPS] Uncrop back from (769, 1467, 14) to (770, 1467, 14) ---- [SPINEPS] Padded from (770, 1467, 14) to (778, 1475, 22)
• [SPINEPS] Predict Semantic Mask ---- [ ] Image is already rotated to ('P', 'I', 'R') ---- [ ] Image resampled from (0.4167, 0.4167, 3.3000002) to voxel size (0.8571, 0.8571, 3.3) ---- [] input shp=(378, 717, 22); ori=('P', 'I', 'R'), zoom=(0.86, 0.86, 3.3), seg=False 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 21/21 [00:03<00:00, 5.95it/s] 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 21/21 [00:01<00:00, 11.93it/s] 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 21/21 [00:01<00:00, 11.95it/s] ---- [] out_inf (0.8571, 0.8571, 3.3) ('P', 'I', 'R') (378, 717, 22) ---- [ ] labelmap = { 1: 41; 2: 42; 3: 43; 4: 44; 5: 45; 6: 46; 7: 47; 8: 48; 9: 49; 11: 61; 10: 60; 12: 100; 13: 62; 14: 26; } ---- [ ] N = 14 labels reassigned, before labels: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14] after: [26, 41, 42, 43, 44, 45, 46, 47, 48, 49, 60, 61, 62, 100] ---- [*] out_seg OutputType.seg (0.8571, 0.8571, 3.3) ('P', 'I', 'R') (378, 717, 22) ---- [SPINEPS] Post-process semantic mask... ---- [SPINEPS] Cleaned (label, n_components) {41: 3, 42: 1, 43: 1, 44: 4, 46: 4, 47: 3, 48: 6, 61: 2, 100: 3} ---- [SPINEPS] Found 5 unique connected components in semantic mask ---- [SPINEPS] semantic_bounding_box_clean did not remove anything ---- [SPINEPS] Fill holes called
• [SPINEPS] seg_nii out (0.8571, 0.8571, 3.3) ('P', 'I', 'R') (378, 717, 22)
• [SPINEPS] Output seg_nii (0.8571, 0.8571, 3.3) ('P', 'I', 'R') (378, 717, 22)
• [SPINEPS] Save /physical_sciences/spineps/Temp_data/Dataset/dataset-T1/derivatives_seg/output_raw_T1w/sub-14a_mod-T1w_seg-spine-raw_msk.nii.gz as uint8
• [SPINEPS] Predict instance mask ---- [SPINEPS] Vertebra input (0.8571, 0.8571, 3.3) ('P', 'I', 'R') (378, 717, 22) ---- [SPINEPS] Image is already rotated to ('P', 'I', 'R') ---- [SPINEPS] zms (0.8571, 0.8571, 3.3) ---- [SPINEPS] Image resampled from (0.8571, 0.8571, 3.3) to voxel size (0.75, 0.75, 1.65) ---- [SPINEPS] Vertebra seg_nii_uncropped (0.75, 0.75, 1.65) ('P', 'I', 'R') (432, 819, 44) ---- [SPINEPS] Vertebra uncropped_vert_mask empty (432, 819, 44) ---- [SPINEPS] Crop down from (432, 819, 44) to (158, 779, 34) ---- [SPINEPS] Vertebra seg_nii_rdy (0.75, 0.75, 1.65) ('P', 'I', 'R') (158, 779, 34) ---- [SPINEPS] cleaning...: 0%| | 0/1 [00:00<?, ?it/s] ---- [SPINEPS] Label 1, cc13, at [64, 683, 17], volume 1: deleted ---- [SPINEPS] Label 1, cc25, at [113, 431, 25], volume 1: deleted ---- [SPINEPS] cleaning...: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 8.88it/s] ---- [SPINEPS] Cleaned (label, n_components) {1: 2} ---- [SPINEPS] Found 23 Corpus ccs (naively) ---- [SPINEPS] Found 26 IVD ccs (naively) ---- [SPINEPS] Wrong ivd-vert alternation found in label 110, is_ivd = True, neighbor 112 ---- [SPINEPS] Same height, just merge [1, 101, 2, 102, 3, 103, 4, 104, 5, 105, 6, 106, 7, 107, 10, 111, 12, 116, 14, 118, 16, 120, 19, 124, 22, 126, 23, 125, 21, 123, 20, 122, 18, 121, 17, 119, 15, 117, 13, 113, 11, 109, 9, 108, 8, 112, 114, 115] ---- [SPINEPS] Wrong ivd-vert alternation found in label 112, is_ivd = True, neighbor 114 ---- [SPINEPS] Same height, just merge [1, 101, 2, 102, 3, 103, 4, 104, 5, 105, 6, 106, 7, 107, 10, 111, 12, 116, 14, 118, 16, 120, 19, 124, 22, 126, 23, 125, 21, 123, 20, 122, 18, 121, 17, 119, 15, 117, 13, 113, 11, 109, 9, 108, 8, 114, 115] ---- [SPINEPS] Wrong ivd-vert alternation found in label 114, is_ivd = True, neighbor 115 ---- [SPINEPS] Same height, just merge [1, 101, 2, 102, 3, 103, 4, 104, 5, 105, 6, 106, 7, 107, 10, 111, 12, 116, 14, 118, 16, 120, 19, 124, 22, 126, 23, 125, 21, 123, 20, 122, 18, 121, 17, 119, 15, 117, 13, 113, 11, 109, 9, 108, 8, 115] ---- [SPINEPS] Found 23 final Corpus ccs ---- [SPINEPS] Vertebra collect in (0.75, 0.75, 1.65) ('P', 'I', 'R') (158, 779, 34) ---- [SPINEPS] Vertebra Body predictions: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 23/23 [00:26<00:00, 1.14s/it] ---- [SPINEPS] Vertebra Predictions done! ---- [SPINEPS] Cleaned (label, n_components) {9: 4, 11: 2, 16: 1, 17: 1, 23: 15, 24: 14} ---- [SPINEPS] Merged predictions into vert mask ---- [SPINEPS] Vertebra whole_vert_nii_cropped (0.75, 0.75, 1.65) ('P', 'I', 'R') (158, 779, 34) ---- [SPINEPS] Fill holes called ---- [SPINEPS] Predicted 24 vertebrae ---- [SPINEPS] Vertebra whole_vert_nii_cropped2 (0.75, 0.75, 1.65) ('P', 'I', 'R') (158, 779, 34) ---- [SPINEPS] Uncrop back from (158, 779, 34) to (432, 819, 44)
• [SPINEPS] vert_out (0.75, 0.75, 1.65) ('P', 'I', 'R') (432, 819, 44)
• [SPINEPS] Save /physical_sciences/spineps/Temp_data/Dataset/dataset-T1/derivatives_seg/output_raw_T1w/sub-14a_mod-T1w_seg-vert-raw_msk.nii.gz as uint8 Traceback (most recent call last): File "/physical_sciences/spineps/spineps-env/bin/spineps", line 8, in sys.exit(entry_point()) ^^^^^^^^^^^^^ File "/physical_sciences/spineps/spineps-env/spineps/spineps/utils/citation_reminder.py", line 21, in wrapper func_result = func(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/physical_sciences/spineps/spineps-env/spineps/spineps/entrypoint.py", line 161, in entry_point run_dataset(opt) File "/physical_sciences/spineps/spineps-env/spineps/spineps/utils/citation_reminder.py", line 21, in wrapper func_result = func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/physical_sciences/spineps/spineps-env/spineps/spineps/entrypoint.py", line 293, in run_dataset process_dataset(kwargs) File "/physical_sciences/spineps/spineps-env/spineps/spineps/utils/citation_reminder.py", line 21, in wrapper func_result = func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/physical_sciences/spineps/spineps-env/spineps/spineps/seg_run.py", line 179, in process_dataset output_paths, errcode = process_img_nii( ^^^^^^^^^^^^^^^^ File "/physical_sciences/spineps/spineps-env/spineps/spineps/utils/citation_reminder.py", line 21, in wrapper func_result = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/physical_sciences/spineps/spineps-env/spineps/spineps/seg_run.py", line 464, in process_img_nii seg_nii_back.assert_affine(other=input_nii) #changed to ignore the matching of raw_mask with nifti image as can be corrected outside ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/physical_sciences/spineps/spineps-env/lib/python3.11/site-packages/TPTBox/core/nii_poi_abstract.py", line 90, in assert_affine other_match = self.assert_affine( ^^^^^^^^^^^^^^^^^^^ File "/physical_sciences/spineps/spineps-env/lib/python3.11/site-packages/TPTBox/core/nii_poi_abstract.py", line 151, in assert_affine raise AssertionError(f"assert_affine failed with {found_errors}") AssertionError: assert_affine failed with ['zoom mismatch (0.8571, 0.8571, 3.3), (0.4167, 0.4167, 3.3000002)', 'origin mismatch (11.4587356, 147.2670258, -16.7962786), (24.6032639, 145.598526, -18.8539352)', 'shape mismatch (378, 717, 22), (770, 1467, 14)'] this is the verbose output it looks like the _assertaffine failed with ['zoom mismatch (0.8571, 0.8571, 3.3), (0.4167, 0.4167, 3.3000002)', 'origin mismatch (11.4587356, 147.2670258, -16.7962786), (24.6032639, 145.598526, -18.8539352)', 'shape mismatch (378, 717, 22), (770, 1467, 14)'] is matching the original image (zoom, shape and origin) to resampled image ?

[Hendrik-code]

Hey very interesting problem. The thing is, if the assertion compares the wrong things, this should always fail, also for my data, which it doesn't. Can you send me the scan in question? It would make my debugging faster and easier. Anyway, thanks for bringing this up! =)

[kartik7737]

Hi Thanks you for the quick reply and for this amazing work. I tried running the code using the spineps sample on individual T1 files, it works. This issue happens when I tried to run on whole dataset

spineps dataset -i "/physical_sciences/spineps/Temp_data/Dataset/dataset-T1" -model_semantic "/physical_sciences/spineps/spineps-env/spineps/spineps/models/T1w_Segmentor" -model_instance "/physical_sciences/spineps/spineps-env/spineps/spineps/models/Inst_Vertebra_3.0" -v

fails to recognize T1 models and give the following error:

[SPINEPS] Processing 14 / 19 subject: 4b [SPINEPS] Subject 14: 4b had no scans to be processed

[SPINEPS] Processing 15 / 19 subject: 4c [SPINEPS] Subject 15: 4c had no scans to be processed

[SPINEPS] Processing 16 / 19 subject: 4d [SPINEPS] Subject 16: 4d had no scans to be processed

[SPINEPS] Processing 17 / 19 subject: 4e [SPINEPS] Subject 17: 4e had no scans to be processed

[SPINEPS] Processing 18 / 19 subject: 4f [SPINEPS] Subject 18: 4f had no scans to be processed

[SPINEPS] Processing 19 / 19 subject: 4g [SPINEPS] Subject 19: 4g had no scans to be processed

[SPINEPS] Processed 0 scans with [(Modality.T2w, Acquisition.sag)]

If I run the same command using (using -imc flag) spineps dataset -i "/physical_sciences/spineps/Temp_data/Dataset/dataset-T1" -model_semantic "/physical_sciences/spineps/spineps-env/spineps/spineps/models/T1w_Segmentor" -model_instance "/physical_sciences/spineps/spineps-env/spineps/spineps/models/Inst_Vertebra_3.0" -v -imc

it gives the error that I mentioned in my first comment.

[kartik7737]

These scans are part of clinical dataset and cannot share them because of ethics :(

[Hendrik-code]

Hey, your dataset is probably not bids-conform, so what happens when you run this with both -imc and -ibf flag? But it always works if you run it on only one of those files? This is strange, because the "running on dataset" pipeline also just runs the single processing under the hood... I will have a closer look later.