ds003500-fmriprep: sub-229 hung indefinitely at resume recon-all

[jbwexler]

This was a rerun, the last time got cut-off at:

221013-21:21:44,674 nipype.workflow INFO: [MultiProc] Running 1 tasks, and 0 jobs ready. Free memory (GB): 32.50/37.50, Free processors: 7/14. Currently running:

• _autorecon_surfs1

Should I just delete freesurfer outputs and rerun?

Output:

221021-09:39:39,863 nipype.workflow INFO: [MultiProc] Running 0 tasks, and 1 jobs ready. Free memory (GB): 37.50/37.50, Free processors: 14/14. 221021-09:39:41,866 nipype.workflow INFO: [MultiProc] Running 0 tasks, and 2 jobs ready. Free memory (GB): 37.50/37.50, Free processors: 14/14. 221021-09:39:41,982 nipype.workflow INFO: [Node] Setting-up "_autorecon_surfs0" in "/scratch1/03201/jbwexler/work_dir/fmriprep/ds003500_sub-229/fmriprep_22_0_wf/single_subject_229_wf/anat_preproc_wf/surface_recon_wf/autorecon_resume_wf/autorecon_surfs/mapflow/_autorecon_surfs0". 221021-09:39:42,9 nipype.workflow INFO: [Node] Executing "_autorecon_surfs0" 221021-09:39:42,31 nipype.interface INFO: recon-all complete : Not running 221021-09:39:42,33 nipype.interface INFO: recon-all complete : Not running 221021-09:39:42,153 nipype.workflow INFO: [Node] Finished "_autorecon_surfs0", elapsed time 0.14234s. 221021-09:39:43,869 nipype.workflow INFO: [Job 2535] Completed (_autorecon_surfs0). 221021-09:39:43,873 nipype.workflow INFO: [MultiProc] Running 1 tasks, and 0 jobs ready. Free memory (GB): 32.50/37.50, Free processors: 7/14. Currently running:

• _autorecon_surfs1 221021-09:39:49,911 nipype.workflow INFO: [Node] Setting-up "_autorecon_surfs1" in "/scratch1/03201/jbwexler/work_dir/fmriprep/ds003500_sub-229/fmriprep_22_0_wf/single_subject_229_wf/anat_preproc_wf/surface_recon_wf/autorecon_resume_wf/autorecon_surfs/mapflow/_autorecon_surfs1". 221021-09:39:49,952 nipype.workflow INFO: [Node] Executing "_autorecon_surfs1" 221021-09:39:49,955 nipype.interface INFO: resume recon-all : recon-all -autorecon-hemi rh -noparcstats -noparcstats2 -noparcstats3 -nohyporelabel -nobalabels -rh-only -openmp 7 -subjid sub-229 -sd /scratch1/03201/jbwexler/openneuro_derivatives/derivatives/fmriprep/ds003500-fmriprep/sourcedata/freesurfer -notessellate -nosmooth1 -noinflate1 -noqsphere 221021-09:39:49,956 nipype.interface INFO: resume recon-all : recon-all -autorecon-hemi rh -noparcstats -noparcstats2 -noparcstats3 -nohyporelabel -nobalabels -rh-only -openmp 7 -subjid sub-229 -sd /scratch1/03201/jbwexler/openneuro_derivatives/derivatives/fmriprep/ds003500-fmriprep/sourcedata/freesurfer -notessellate -nosmooth1 -noinflate1 -noqsphere

[effigies]

Looking at https://openneuro.org/datasets/ds003500/versions/1.2.0/file-display/sub-229:anat:sub-229_acq-a_T1w.nii.gz and https://openneuro.org/datasets/ds003500/versions/1.2.0/file-display/sub-229:anat:sub-229_acq-b_T1w.nii.gz, it seems there might be some scaling issues and possibly some noise in the top slice. Do we have desc-preproc_T1w.nii.gz in the derivatives at this point? If so, it would be worth seeing the quality of this file, as that's the input to FreeSurfer.

[jbwexler]

Yep, it's slightly too big to send here, I'll slack it to you.

[effigies]

Yeah, these images are not mergeable:

Fetching all the T1w images, it looks like sub-229_acq-a_T1w.nii.gz is the odd one out:

(base) ➜  ds003500 git:(master) nib-ls */anat/*.nii.gz
sub-213/anat/sub-213_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-213/anat/sub-213_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-214/anat/sub-214_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-218/anat/sub-218_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-218/anat/sub-218_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-219/anat/sub-219_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-219/anat/sub-219_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-220/anat/sub-220_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-220/anat/sub-220_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-221/anat/sub-221_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-221/anat/sub-221_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-222/anat/sub-222_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-223/anat/sub-223_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-224/anat/sub-224_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-225/anat/sub-225_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-225/anat/sub-225_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-226/anat/sub-226_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-226/anat/sub-226_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-228/anat/sub-228_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-228/anat/sub-228_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-229/anat/sub-229_acq-a_T1w.nii.gz int16 [256, 256, 124] 1.00x1.00x1.00
sub-229/anat/sub-229_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-230/anat/sub-230_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-230/anat/sub-230_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-231/anat/sub-231_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-232/anat/sub-232_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-232/anat/sub-232_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-233/anat/sub-233_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-233/anat/sub-233_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-234/anat/sub-234_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-234/anat/sub-234_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-235/anat/sub-235_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-236/anat/sub-236_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-236/anat/sub-236_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-237/anat/sub-237_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-238/anat/sub-238_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-240/anat/sub-240_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-240/anat/sub-240_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-241/anat/sub-241_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-242/anat/sub-242_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-245/anat/sub-245_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-245/anat/sub-245_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-246/anat/sub-246_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-246/anat/sub-246_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-248/anat/sub-248_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-248/anat/sub-248_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-250/anat/sub-250_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-250/anat/sub-250_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-251/anat/sub-251_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-251/anat/sub-251_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-253/anat/sub-253_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-254/anat/sub-254_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-254/anat/sub-254_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-256/anat/sub-256_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-256/anat/sub-256_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-258/anat/sub-258_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-258/anat/sub-258_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-259/anat/sub-259_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-259/anat/sub-259_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-260/anat/sub-260_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-260/anat/sub-260_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-262/anat/sub-262_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-262/anat/sub-262_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-268/anat/sub-268_acq-a_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10
sub-268/anat/sub-268_acq-b_T1w.nii.gz int16 [256, 256, 124] 0.86x0.86x1.10

I would probably just skip this subject for now, and we can contact the authors and ask them to re-convert and upload that subject.

[jbwexler]

Ok I'll remove this subject, leave a note in the README and then publish.