search

MASILab / Synb0-DISCO

Distortion correction of diffusion weighted MRI without reverse phase-encoding scans or field-maps
https://my.vanderbilt.edu/masi
57 stars 28 forks source link

Syn0-disco crashes with python3.6 command #61

Open hoptman opened 5 months ago

hoptman commented 5 months ago

To whom it may concern: I am using a syn0-disco v. 3.0 image and am getting a recurrent crash in which the program appears to be looking for python3.6 to run inferences on the various folds. I do not have python3.6 on my system (I have python3.8). I tried to create a symlink of python3.8 to python3.6, but that didn't work. Any suggestions are appreciated.

My call to docker is: docker run --rm -v pwd/:/INPUTS/ -v pwd/outputs/:/OUTPUTS -v /mnt/software/freesurfer/license.txt:/extra/freesurfer/license.txt leonyichencai/synb0-disco:v3.0 /INPUTS /OUTPUTS participant

Thanks, Matt

The error messages look like:

Removing job directory... Performing inference on FOLD: 1 /extra/pipeline.sh: line 42: 1761 Killed python3.6 /extra/inference.py /OUTPUTS/T1_norm_lin_atlas_2_5.nii.gz /OUTPUTS/b0_d_lin_atlas_2_5.nii.gz /OUTPUTS/b0_u_lin_atlas_2_5FOLD"$i".nii.gz /extra/dual_channel_unet/numfold"$i"_totalfolds"$NUM_FOLDS"_seed_1_num_epochs_100_lr_0.0001betas(0.9\,\ 0.999)_weight_decay_1e-05_numepoch*.pth

schillkg commented 5 months ago

Hi Matt - I believe this is a memory issue. Our testing found you need to allow the docker a minimum of 8Gb of RAM, but we've found many cases where this is not enough and the process is killed. If you could give it 12-16Gb you will find the process will finish well.

The good news is this isn't a python issue - the beauty of docker/singularity is that it is all contained within the container so it shouldn't matter what your system is running.

Thank you.

hoptman commented 5 months ago

Hi Matt - I believe this is a memory issue. Our testing found you need to allow the docker a minimum of 8Gb of RAM, but we've found many cases where this is not enough and the process is killed. If you could give it 12-16Gb you will find the process will finish well.

The good news is this isn't a python issue - the beauty of docker/singularity is that it is all contained within the container so it shouldn't matter what your system is running.

Thank you.

Thanks. I'll give it a try!