search

valosekj / dcm-brno

GNU General Public License v3.0
0 stars 0 forks source link

`dcm-brno_2024-07-05` processing errors #16

Open valosekj opened 1 month ago

valosekj commented 1 month ago

Even after increasing the crop size to 35x35x0 (https://github.com/valosekj/dcm-brno/issues/14#issuecomment-2210748905), sct_dmri_moco is still crashing for the following subjects.

https://github.com/valosekj/dcm-brno/blob/905e734c803800963d1e39d2fed5babd1239c721/scripts/01_process_data.sh#L337-L341

Error logs for all the subjects:

$ dcm-brno_2024-07-05/log$ ls -1 err*
err.01_process_data_sub-1836B6029B_ses-1836B.log
err.01_process_data_sub-1860B6472B_ses-6472B.log
err.01_process_data_sub-2060B4733B_ses-2060B.log
err.01_process_data_sub-2249B4725B_ses-4725B.log
err.01_process_data_sub-2295B4676B_ses-2295B.log
err.01_process_data_sub-2295B4676B_ses-4676B.log
err.01_process_data_sub-2330B6137B_ses-6137B.log
err.01_process_data_sub-2371B4687B_ses-4687B.log
err.01_process_data_sub-2372B2863B_ses-2372B.log
err.01_process_data_sub-2372B2863B_ses-2863B.log
err.01_process_data_sub-2407B5757B_ses-5757B.log
err.01_process_data_sub-2411B4591B_ses-2411B.log
err.01_process_data_sub-2416B6027B_ses-2416B.log
err.01_process_data_sub-2417B4965B_ses-4965B.log
err.01_process_data_sub-2418B4628B_ses-4628B.log
err.01_process_data_sub-2450B6177B_ses-2450B.log
err.01_process_data_sub-2450B6177B_ses-6177B.log
err.01_process_data_sub-2481B6079B_ses-2481B.log
err.01_process_data_sub-2590B4647B_ses-2590B.log
err.01_process_data_sub-2590B4647B_ses-4647B.log
err.01_process_data_sub-2773B4661B_ses-2773B.log
err.01_process_data_sub-2804B4632B_ses-4632B.log
err.01_process_data_sub-3782B6193B_ses-3782B.log
err.01_process_data_sub-3998B6406B_ses-3998B.log
err.01_process_data_sub-4891B6572B_ses-4891B.log
details ```bash $ dcm-brno_2024-07-05/log$ for err in err*;do echo $err; tail -3 $err; echo "";done err.01_process_data_sub-1836B6029B_ses-1836B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x21cf840): Joint PDF summed to zero err.01_process_data_sub-1860B6472B_ses-6472B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x2b13cf0): Joint PDF summed to zero err.01_process_data_sub-2060B4733B_ses-2060B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x1f3bcf0): Joint PDF summed to zero err.01_process_data_sub-2249B4725B_ses-4725B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x24d2220): Joint PDF summed to zero err.01_process_data_sub-2295B4676B_ses-2295B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x315ae70): Joint PDF summed to zero err.01_process_data_sub-2295B4676B_ses-4676B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x2c0ee90): Joint PDF summed to zero err.01_process_data_sub-2330B6137B_ses-6137B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x3659f20): Joint PDF summed to zero err.01_process_data_sub-2371B4687B_ses-4687B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x1f12c70): Joint PDF summed to zero err.01_process_data_sub-2372B2863B_ses-2372B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x38af280): Joint PDF summed to zero err.01_process_data_sub-2372B2863B_ses-2863B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x314bcf0): Joint PDF summed to zero err.01_process_data_sub-2407B5757B_ses-5757B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x296f670): Joint PDF summed to zero err.01_process_data_sub-2411B4591B_ses-2411B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x38c7af0): Joint PDF summed to zero err.01_process_data_sub-2416B6027B_ses-2416B.log File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/centerline/nurbs.py", line 103, in __init__ raise ArithmeticError('There are too few points to compute. The number of points of the curve must ' ArithmeticError: There are too few points to compute. The number of points of the curve must be strictly superior to degre + 2, in this case: 5. Either change degree to a lower value, or add points to the curve. err.01_process_data_sub-2417B4965B_ses-4965B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x36ae3a0): Joint PDF summed to zero err.01_process_data_sub-2418B4628B_ses-4628B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x1fa9170): Joint PDF summed to zero err.01_process_data_sub-2450B6177B_ses-2450B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x1e97d60): Joint PDF summed to zero err.01_process_data_sub-2450B6177B_ses-6177B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x3d74690): Joint PDF summed to zero err.01_process_data_sub-2481B6079B_ses-2481B.log File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/cropping.py", line 116, in get_bbox_from_mask min(data_nonzero[0]), max(data_nonzero[0]), ValueError: min() arg is an empty sequence err.01_process_data_sub-2590B4647B_ses-2590B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x3d33960): Joint PDF summed to zero err.01_process_data_sub-2590B4647B_ses-4647B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x3517980): Joint PDF summed to zero err.01_process_data_sub-2773B4661B_ses-2773B.log File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/image.py", line 321, in __init__ raise ValueError("Image sform does not match qform") ValueError: Image sform does not match qform err.01_process_data_sub-2804B4632B_ses-4632B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x3622c70): Joint PDF summed to zero err.01_process_data_sub-3782B6193B_ses-3782B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x3048b70): Joint PDF summed to zero err.01_process_data_sub-3998B6406B_ses-3998B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x3286240): Joint PDF summed to zero err.01_process_data_sub-4891B6572B_ses-4891B.log File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x2c02c70): Joint PDF summed to zero ```
valosekj commented 1 month ago

Debugging sub-1836B6029B_ses-1836B_acq-ZOOMit_dir-AP_dwi.nii.gz: there is a complete signal loss at the bottom part of the image (around C7) (consistent with the original publication). Cropping the last 6 slices resolves the crashing sct_dmri_moco:

sct_crop_image -i "${file_dwi}_crop".nii.gz  -zmin 6 -zmax -1 -o "${file_dwi}_crop_crop".nii.gz
image

Change pushed in https://github.com/valosekj/dcm-brno/commit/f0df460a9de007672718f2be590948711c3eeb02

valosekj commented 1 month ago

Cropping the last 6 slices resolves issues for a lot of subjects. Current log files are:

dcm-brno_2024-07-08/log$ ls -1 err*
err.01_process_data_sub-2249B4725B_ses-4725B.log
err.01_process_data_sub-2416B6027B_ses-2416B.log
err.01_process_data_sub-2481B6079B_ses-2481B.log
err.01_process_data_sub-2590B4647B_ses-4647B.log
err.01_process_data_sub-2773B4661B_ses-2773B.log
details ```bash dcm-brno_2024-07-08/log$ for err in err*;do echo $err; tail $err; echo "";done err.01_process_data_sub-2249B4725B_ses-4725B.log File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/moco.py", line 753, in register status, output = run_proc(cmd, verbose=1 if param.verbose == 2 else 0, env=env, **kw) File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/utils/sys.py", line 342, in run_proc raise RuntimeError(output) RuntimeError: Exception caught: itk::ExceptionObject (0x245d270) Location: "unknown" File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x244e4f0): Joint PDF summed to zero err.01_process_data_sub-2416B6027B_ses-2416B.log fname_straight = sc_straight.straighten() File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/straightening.py", line 154, in straighten _, arr_ctl_phys, arr_ctl_der_phys, _ = get_centerline(img_ctl, self.param_centerline, File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/centerline/core.py", line 168, in get_centerline z_centerline_deriv, error = b_spline_nurbs(x_mean_interp, y_mean_interp, z_ref, nbControl=None, File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/centerline/nurbs.py", line 991, in b_spline_nurbs nurbs = NURBS(degree, point_number, data, False, nbControl, verbose, all_slices=all_slices, twodim=twodim) File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/centerline/nurbs.py", line 103, in __init__ raise ArithmeticError('There are too few points to compute. The number of points of the curve must ' ArithmeticError: There are too few points to compute. The number of points of the curve must be strictly superior to degre + 2, in this case: 5. Either change degree to a lower value, or add points to the curve. err.01_process_data_sub-2481B6079B_ses-2481B.log QcImage( File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/reports/qc.py", line 295, in layout img, *mask = qcslice_layout(qcslice) File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/reports/qc.py", line 757, in qcslice_layout def qcslice_layout(x): return x.mosaic()[:2] File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/reports/slice.py", line 458, in mosaic cropper.get_bbox_from_mask(self._image_seg) File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/cropping.py", line 116, in get_bbox_from_mask min(data_nonzero[0]), max(data_nonzero[0]), ValueError: min() arg is an empty sequence err.01_process_data_sub-2590B4647B_ses-4647B.log File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/moco.py", line 753, in register status, output = run_proc(cmd, verbose=1 if param.verbose == 2 else 0, env=env, **kw) File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/utils/sys.py", line 342, in run_proc raise RuntimeError(output) RuntimeError: Exception caught: itk::ExceptionObject (0x2e90370) Location: "unknown" File: /__w/build_ANTs/build_ANTs/antsbin/staging/include/ITK-5.1/itkMattesMutualInformationImageToImageMetricv4.hxx Line: 312 Description: itk::ERROR: MattesMutualInformationImageToImageMetricv4(0x2e37be0): Joint PDF summed to zero err.01_process_data_sub-2773B4661B_ses-2773B.log Traceback (most recent call last): File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/scripts/sct_register_to_template.py", line 820, in main(sys.argv[1:]) File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/scripts/sct_register_to_template.py", line 564, in main sc_straight.straighten() File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/straightening.py", line 108, in straighten Image(fname_anat, check_sform=True).save(os.path.join(path_tmp, "data.nii")) File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/image.py", line 321, in __init__ raise ValueError("Image sform does not match qform") ValueError: Image sform does not match qform ```
valosekj commented 1 month ago

Changing the metric for sct_dmri_moco to CC helped for sub-2249B4725B_ses-4725B and sub-2590B4647B_ses-4647B.

Fixed in https://github.com/valosekj/dcm-brno/commit/cf162b86ae1be6a15eb722119652dc9b3c96b11d

valosekj commented 1 month ago

sub-2416B6027B_ses-2416B had disc labels for two runs under derivatives.

Fixed in https://github.com/valosekj/dcm-brno_derivatives/commit/0415eb769bb4bb5531a7b1118f72b20a874b1f86

valosekj commented 1 month ago

sub-2481B6079B_ses-2481B has an empty SC seg; probably related to https://github.com/spinalcordtoolbox/spinalcordtoolbox/issues/4556

I ran the prediction locally and pushed to derivatives in https://github.com/valosekj/dcm-brno_derivatives/commit/b6e5d9ac65d2dc66a59427764144651e866c8f70

valosekj commented 1 month ago

sub-2773B4661B_ses-2773B finished with an error for sct_register_to_template.

details ```bash -- Spinal Cord Toolbox (git-master-65e44b5068fe5d8a094e3e0571566c01d2d460ab) sct_register_to_template -i sub-2773B4661B_ses-2773B_T2w.nii.gz -s sub-2773B4661B_ses-2773B_T2w_seg.nii.gz -l sub-2773B4661B_ses-2773B_T2w_label-disc_c3c5.nii.gz -c t2 -param step=1,type=seg,algo=centermassrot:step=2,type=seg,algo=syn,slicewise=1,smooth=0,iter=5:step=3,type=im,algo=syn,slicewise=1,smooth=0,iter=3 -qc /md1/valosek/results/dcm-brno_2024-07-08/qc -qc-subject sub-2773B4661B_ses-2773B -- Check template files... OK: /home/valosek/code/sct_v6.3/data/PAM50/template/PAM50_t2.nii.gz OK: /home/valosek/code/sct_v6.3/data/PAM50/template/PAM50_levels.nii.gz OK: /home/valosek/code/sct_v6.3/data/PAM50/template/PAM50_cord.nii.gz Check parameters: Data: sub-2773B4661B_ses-2773B_T2w.nii.gz Landmarks: sub-2773B4661B_ses-2773B_T2w_label-disc_c3c5.nii.gz Segmentation: sub-2773B4661B_ses-2773B_T2w_seg.nii.gz Path template: /home/valosek/code/sct_v6.3/data/PAM50 Remove temp files: 1 Check input labels... Creating temporary folder (/tmp/sct_2024-07-08_20-52-25_register-to-template_m84547qm) Copying input data to tmp folder and convert to nii... Generate labels from template vertebral labeling Check if provided labels are available in the template Binarize segmentation Change orientation of input images to RPI... Resample data to 1mm isotropic... load data... Converting image from type 'int16' to type 'float64' for linear interpolation Done! To view results, run one of the following commands (depending on your preferred viewer): fsleyes data_rpi_1mm.nii & fslview_deprecated data_rpi_1mm.nii & fslview data_rpi_1mm.nii & itksnap -g data_rpi_1mm.nii load data... Converting image from type 'uint8' to type 'float64' for linear interpolation Done! To view results, run one of the following commands (depending on your preferred viewer): fsleyes seg_bin_rpi_1mm.nii.gz & fslview_deprecated seg_bin_rpi_1mm.nii.gz & fslview seg_bin_rpi_1mm.nii.gz & itksnap -g seg_bin_rpi_1mm.nii.gz To avoid intensity overflow due to convertion to +uint8+, intensity will be rescaled to the maximum quantization scale Straighten the spinal cord using centerline/segmentation... Creating temporary folder (/tmp/sct_2024-07-08_20-52-40_straighten-spinalcord_wagfke1l) Copy files to tmp folder... Image /tmp/sct_2024-07-08_20-52-25_register-to-template_m84547qm/seg_bin_rpi_1mm_crop.nii.gz has different qform and sform matrices. This can produce incorrect results. Please use 'sct_image -i /tmp/sct_2024-07-08_20-52-25_register-to-template_m84547qm/seg_bin_rpi_1mm_crop.nii.gz -header' to check that both affine matrices are valid. Then, consider running either 'sct_image -set-sform-to-qform' or 'sct_image -set-qform-to-sform' to fix any discrepancies you may find. Traceback (most recent call last): File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/scripts/sct_register_to_template.py", line 820, in main(sys.argv[1:]) File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/scripts/sct_register_to_template.py", line 564, in main sc_straight.straighten() File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/straightening.py", line 108, in straighten Image(fname_anat, check_sform=True).save(os.path.join(path_tmp, "data.nii")) File "/home/valosek/code/sct_v6.3/spinalcordtoolbox/image.py", line 321, in __init__ raise ValueError("Image sform does not match qform") ValueError: Image sform does not match qform ```

When trying the command locally, it worked fine, though.