MASILab / Synb0-DISCO

Distortion correction of diffusion weighted MRI without reverse phase-encoding scans or field-maps
https://my.vanderbilt.edu/masi
57 stars 28 forks source link

Error running inference #51

Closed mrgrist closed 1 year ago

mrgrist commented 1 year ago

Hi there,

I seem to get the following errors when running inference. Not quite sure what is gonig on? I'm running a MacBook Pro with a pre-arm chip

Removing job directory... Performing inference on FOLD: 1 Performing inference on FOLD: 2 /extra/pipeline.sh: line 42: 1899 Killed python3.6 /extra/inference.py /OUTPUTS/T1_norm_lin_atlas_2_5.nii.gz /OUTPUTS/b0_d_lin_atlas_2_5.nii.gz /OUTPUTS/b0_u_lin_atlas_2_5FOLD"$i".nii.gz /extra/dual_channel_unet/numfold"$i"_totalfolds"$NUM_FOLDS"_seed_1_num_epochs_100_lr_0.0001betas(0.9\,\ 0.999)_weight_decay_1e-05_numepoch.pth /extra/pipeline.sh: line 42: 1944 Killed python3.6 /extra/inference.py /OUTPUTS/T1_norm_lin_atlas_2_5.nii.gz /OUTPUTS/b0_d_lin_atlas_2_5.nii.gz /OUTPUTS/b0_u_lin_atlas_2_5FOLD"$i".nii.gz /extra/dual_channel_unet/numfold"$i"_totalfolds"$NUM_FOLDS"_seed_1_num_epochs_100_lr_0.0001betas(0.9\,\ 0.999)_weight_decay_1e-05_numepoch.pth Performing inference on FOLD: 3 /extra/pipeline.sh: line 42: 1989 Killed python3.6 /extra/inference.py /OUTPUTS/T1_norm_lin_atlas_2_5.nii.gz /OUTPUTS/b0_d_lin_atlas_2_5.nii.gz /OUTPUTS/b0_u_lin_atlas_2_5FOLD"$i".nii.gz /extra/dual_channel_unet/numfold"$i"_totalfolds"$NUM_FOLDS"_seed_1_num_epochs_100_lr_0.0001betas(0.9\,\ 0.999)_weight_decay_1e-05_numepoch.pth Performing inference on FOLD: 4 Performing inference on FOLD: 5 /extra/pipeline.sh: line 42: 2034 Killed python3.6 /extra/inference.py /OUTPUTS/T1_norm_lin_atlas_2_5.nii.gz /OUTPUTS/b0_d_lin_atlas_2_5.nii.gz /OUTPUTS/b0_u_lin_atlas_2_5FOLD"$i".nii.gz /extra/dual_channel_unet/numfold"$i"_totalfolds"$NUM_FOLDS"_seed_1_num_epochs_100_lr_0.0001betas(0.9\,\ 0.999)_weight_decay_1e-05_numepoch.pth Taking ensemble average /extra/pipeline.sh: line 42: 2079 Killed python3.6 /extra/inference.py /OUTPUTS/T1_norm_lin_atlas_2_5.nii.gz /OUTPUTS/b0_d_lin_atlas_2_5.nii.gz /OUTPUTS/b0_u_lin_atlas_2_5FOLD"$i".nii.gz /extra/dual_channel_unet/numfold"$i"_totalfolds"$NUM_FOLDS"_seed_1_num_epochs_100_lr_0.0001betas(0.9\,\ 0.999)_weight_decay_1e-05_numepoch.pth Image Exception : #63 :: No image files match: /OUTPUTS/b0_u_lin_atlas_2_5FOLD terminate called after throwing an instance of 'std::runtime_error' what(): No image files match: /OUTPUTS/b0_u_lin_atlas_2_5FOLD /extra/pipeline.sh: line 49: 2124 Aborted fslmerge -t /OUTPUTS/b0_u_lin_atlas_2_5_merged.nii.gz /OUTPUTS/b0_u_lin_atlas_2_5FOLD.nii.gz Image Exception : #63 :: No image files match: /OUTPUTS/b0_u_lin_atlas_2_5_merged terminate called after throwing an instance of 'std::runtime_error' what(): No image files match: /OUTPUTS/b0_u_lin_atlas_2_5_merged /extra/pipeline.sh: line 50: 2125 Aborted fslmaths /OUTPUTS/b0_u_lin_atlas_2_5_merged.nii.gz -Tmean /OUTPUTS/b0_u_lin_atlas_2_5.nii.gz Applying inverse xform to undistorted b0 file /OUTPUTS/b0_u_lin_atlas_2_5.nii.gz does not exist . terminate called after throwing an instance of 'itk::ExceptionObject' what(): /home/local/VANDERBILT/blaberj/ANTS_13_FEB_2019/bin/ants/ITKv5/Modules/Core/Common/src/itkProcessObject.cxx:1412: itk::ERROR: ResampleImageFilter(0x34affd0): Input Primary is required but not set. Applying slight smoothing to distorted b0 /extra/pipeline.sh: line 54: 2126 Aborted antsApplyTransforms -d 3 -i /OUTPUTS/b0_u_lin_atlas_2_5.nii.gz -r /INPUTS/b0.nii.gz -n BSpline -t [/OUTPUTS/epi_reg_d_ANTS.txt,1] -t [/OUTPUTS/ANTS0GenericAffine.mat,1] -o /OUTPUTS/b0_u.nii.gz

schillkg commented 1 year ago

This is a memory issue. Give the container more memory (we recommend 12Gb, but 16Gb will be safe) and it will successfully run.