search

nipreps / fmriprep

fMRIPrep is a robust and easy-to-use pipeline for preprocessing of diverse fMRI data. The transparent workflow dispenses of manual intervention, thereby ensuring the reproducibility of the results.
https://fmriprep.org
Apache License 2.0
615 stars 288 forks source link

Randomly fail when using functions in anat_fit_wf #3277

Open yufang1039 opened 2 months ago

yufang1039 commented 2 months ago

What happened?

We are running fmriprep within docker for our own subject database (rs-fMRI, T1, T2). The laptop I'm using has M3 chip, 32GB memory, and 12 cores. Some of the subjects pass successfully, but the rest of them failed on different types of nodes:

240427-02:38:36,572 nipype.workflow ERROR:
     could not run node: fmriprep_23_2_wf.sub_01_wf.anat_fit_wf.surface_recon_wf.autorecon1
240427-02:38:36,594 nipype.workflow ERROR:
     could not run node: fmriprep_23_2_wf.sub_02_wf.anat_fit_wf.brain_extraction_wf.atropos_wf.24_depad_segm
240427-02:38:36,605 nipype.workflow ERROR:
     could not run node: fmriprep_23_2_wf.sub_13_wf.anat_fit_wf.surface_recon_wf.autorecon_resume_wf.autorecon2_vol
240427-02:38:36,613 nipype.workflow ERROR:
     could not run node: fmriprep_23_2_wf.sub_14_wf.anat_fit_wf.surface_recon_wf.autorecon_resume_wf.autorecon2_vol
240427-02:38:36,620 nipype.workflow ERROR:
     could not run node: fmriprep_23_2_wf.sub_18_wf.anat_fit_wf.surface_recon_wf.autorecon_resume_wf.autorecon2_vol
240427-02:38:36,626 nipype.workflow ERROR:
     could not run node: fmriprep_23_2_wf.sub_18_wf.anat_fit_wf.t2w_template_wf.anat_merge
240427-02:38:36,631 nipype.workflow ERROR:
     could not run node: fmriprep_23_2_wf.sub_11_wf.anat_fit_wf.surface_recon_wf.autorecon_resume_wf.autorecon_surfs
240427-02:38:36,636 nipype.workflow ERROR:
     could not run node: fmriprep_23_2_wf.sub_08_wf.anat_fit_wf.surface_recon_wf.autorecon_resume_wf.autorecon_surfs
240427-02:38:36,640 nipype.workflow ERROR:
     could not run node: fmriprep_23_2_wf.sub_25_wf.anat_fit_wf.surface_recon_wf.autorecon_resume_wf.autorecon_surfs
240427-02:38:36,645 nipype.workflow ERROR:
     could not run node: fmriprep_23_2_wf.sub_28_wf.anat_fit_wf.surface_recon_wf.autorecon_resume_wf.cortribbon
240427-02:38:36,649 nipype.workflow ERROR:
     could not run node: fmriprep_23_2_wf.sub_06_wf.anat_fit_wf.surface_recon_wf.autorecon_resume_wf.parcstats
240427-02:38:36,652 nipype.workflow ERROR:
     could not run node: fmriprep_23_2_wf.sub_15_wf.anat_fit_wf.surface_recon_wf.autorecon_resume_wf.parcstats
240427-02:38:36,656 nipype.workflow ERROR:
     could not run node: fmriprep_23_2_wf.sub_23_wf.anat_fit_wf.surface_recon_wf.autorecon_resume_wf.parcstats
240427-02:38:36,659 nipype.workflow ERROR:
     could not run node: fmriprep_23_2_wf.sub_24_wf.anat_fit_wf.surface_recon_wf.autorecon_resume_wf.autorecon3
240427-02:38:36,890 nipype.workflow CRITICAL:
     fMRIPrep failed: 14 raised. Re-raising first.
240427-02:38:51,253 cli ERROR:
     Preprocessing did not finish successfully. Errors occurred while processing data from participants: 01 (1), 02 (1), 06 (1), 08 (1), 11 (1), 13 (1), 14 (1), 15 (1), 18 (2), 23 (1), 24 (1), 25 (1), 28 (1).

I tried running fmriprep on only one subject (subject 14) with two sessions, it failed on a different error.

240430-19:17:18,914 nipype.workflow WARNING:
     [Node] Error on "fmriprep_23_2_wf.sub_14_wf.anat_fit_wf.surface_recon_wf.autorecon1" (/scratch/fmriprep_23_2_wf/sub_14_wf/anat_fit_wf/surface_recon_wf/autorecon1)
240430-19:17:20,459 nipype.workflow ERROR:
     Node autorecon1 failed to run on host 005abe4ae096.
240430-19:17:20,464 nipype.workflow ERROR:
     Saving crash info to /out/sub-14/log/20240430-190529_47e10b7b-b8e4-470a-b82e-8aa193ebf15d/crash-20240430-191720-root-autorecon1-cb641879-36eb-42b0-9e06-aca5f6b0a846.txt

Another attempt fmriprep on only subject 15 with two sessions is successful. Since it failed at the first time, I think it's similar to issue #3220:https://github.com/nipreps/fmriprep/issues/3220#issue-2119212392, but our data is located at Desktop directory of Mac which I believe is not FAT/NTFS.

What command did you use?

fmriprep-docker tDCS_BIDS_test tDCS_BIDS_test_output -w fmriprep_working --omp-nthreads=6 --mem_mb=24000
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
RUNNING: docker run --rm -e DOCKER_VERSION_8395080871=25.0.3 -it -v /Users/jungheeha/Desktop/tDCS_BIDS_test:/data:ro -v /Users/jungheeha/Desktop/tDCS_BIDS_test_output:/out -v /Users/jungheeha/Desktop/fmriprep_working:/scratch nipreps/fmriprep:23.2.1 /data /out participant --omp-nthreads=6 --mem_mb=24000 -w /scratch
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

What version of fMRIPrep are you running?

23.2.1

How are you running fMRIPrep?

Docker

Is your data BIDS valid?

Yes

Are you reusing any previously computed results?

No

Please copy and paste any relevant log output.

This is the log from the attempt with subject 14 only:

Node Name: fmriprep_23_2_wf.sub_14_wf.anat_fit_wf.surface_recon_wf.autorecon1

File: /out/sub-14/log/20240430-190529_47e10b7b-b8e4-470a-b82e-8aa193ebf15d/crash-20240430-191720-root-autorecon1-cb641879-36eb-42b0-9e06-aca5f6b0a846.txt
Working Directory: /scratch/fmriprep_23_2_wf/sub_14_wf/anat_fit_wf/surface_recon_wf/autorecon1
Inputs:
FLAIR_file:
T1_files:
T2_file:
args:
big_ventricles:
brainstem:
directive: autorecon1
environ: {}
expert:
flags:
hemi:
hippocampal_subfields_T1:
hippocampal_subfields_T2:
hires:
mprage:
mri_aparc2aseg:
mri_ca_label:
mri_ca_normalize:
mri_ca_register:
mri_edit_wm_with_aseg:
mri_em_register:
mri_fill:
mri_mask:
mri_normalize:
mri_pretess:
mri_remove_neck:
mri_segment:
mri_segstats:
mri_tessellate:
mri_watershed:
mris_anatomical_stats:
mris_ca_label:
mris_fix_topology:
mris_inflate:
mris_make_surfaces:
mris_register:
mris_smooth:
mris_sphere:
mris_surf2vol:
mrisp_paint:
openmp: 6
parallel:
steps:
subject_id: recon_all
subjects_dir:
talairach:
use_FLAIR:
use_T2:
xopts:
Traceback (most recent call last):
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node autorecon1.

Cmdline:
    recon-all -autorecon1 -i /scratch/fmriprep_23_2_wf/sub_14_wf/anat_fit_wf/anat_template_wf/anat_merge/sub-14_ses-01_T1w_noise_corrected_ras_template.nii -T2 /data/sub-14/ses-01/anat/sub-14_ses-01_T2w.nii -noskullstrip -noT2pial -noFLAIRpial -openmp 6 -subjid sub-14 -sd /out/sourcedata/freesurfer
Stdout:
    fs-check-version --s sub-14 --o /tmp/tmp.OIwy39
    Tue Apr 30 19:11:07 UTC 2024

    setenv SUBJECTS_DIR /out/sourcedata/freesurfer
    cd /scratch/fmriprep_23_2_wf/sub_14_wf/anat_fit_wf/surface_recon_wf/autorecon1
    /opt/freesurfer/bin/fs-check-version --s sub-14 --o /tmp/tmp.OIwy39
    -rwxrwxr-x 1 root root 18565 Aug  4  2022 /opt/freesurfer/bin/fs-check-version

    freesurfer-linux-ubuntu22_x86_64-7.3.2-20220804-6354275
    $Id$
    Linux 005abe4ae096 6.6.16-linuxkit #1 SMP Fri Feb 16 11:54:02 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
    pid 468
    Current FS Version freesurfer-linux-ubuntu22_x86_64-7.3.2-20220804-6354275
    Subject does not have a bstampfile, copying /opt/freesurfer/build-stamp.txt
    Subject FS Version: freesurfer-linux-ubuntu22_x86_64-7.3.2-20220804-6354275
    No constraints on version because REQ=UnSet and FsVerFile=NotThere
    #@#% fs-check-version match = 1
    fs-check-version Done
    INFO: SUBJECTS_DIR is /out/sourcedata/freesurfer
    Actual FREESURFER_HOME /opt/freesurfer
    Linux 005abe4ae096 6.6.16-linuxkit #1 SMP Fri Feb 16 11:54:02 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
    /out/sourcedata/freesurfer/sub-14

     mri_convert /scratch/fmriprep_23_2_wf/sub_14_wf/anat_fit_wf/anat_template_wf/anat_merge/sub-14_ses-01_T1w_noise_corrected_ras_template.nii /out/sourcedata/freesurfer/sub-14/mri/orig/001.mgz 

    mri_convert /scratch/fmriprep_23_2_wf/sub_14_wf/anat_fit_wf/anat_template_wf/anat_merge/sub-14_ses-01_T1w_noise_corrected_ras_template.nii /out/sourcedata/freesurfer/sub-14/mri/orig/001.mgz 
    reading from /scratch/fmriprep_23_2_wf/sub_14_wf/anat_fit_wf/anat_template_wf/anat_merge/sub-14_ses-01_T1w_noise_corrected_ras_template.nii...
    TR=0.00, TE=0.00, TI=0.00, flip angle=0.00
    i_ras = (0.998126, 0.0459843, -0.0403783)
    j_ras = (-0.0317161, 0.952999, 0.301308)
    k_ras = (0.052336, -0.299462, 0.952672)
    writing to /out/sourcedata/freesurfer/sub-14/mri/orig/001.mgz...
    #--------------------------------------------
    #@# T2/FLAIR Input Tue Apr 30 19:11:17 UTC 2024
    /out/sourcedata/freesurfer/sub-14

     mri_convert --no_scale 1 /data/sub-14/ses-01/anat/sub-14_ses-01_T2w.nii /out/sourcedata/freesurfer/sub-14/mri/orig/T2raw.mgz 

    mri_convert --no_scale 1 /data/sub-14/ses-01/anat/sub-14_ses-01_T2w.nii /out/sourcedata/freesurfer/sub-14/mri/orig/T2raw.mgz 
    reading from /data/sub-14/ses-01/anat/sub-14_ses-01_T2w.nii...
    TR=4800.00, TE=0.00, TI=0.00, flip angle=0.00
    i_ras = (0.0317161, -0.952999, -0.301308)
    j_ras = (0.052336, -0.299462, 0.952672)
    k_ras = (0.998126, 0.0459843, -0.0403783)
    writing to /out/sourcedata/freesurfer/sub-14/mri/orig/T2raw.mgz...
    #--------------------------------------------
    #@# MotionCor Tue Apr 30 19:11:34 UTC 2024
    Found 1 runs
    /out/sourcedata/freesurfer/sub-14/mri/orig/001.mgz
    Checking for (invalid) multi-frame inputs...
    Only one run found so motion
    correction will not be performed. I'll
    copy the run to rawavg and continue.

     cp /out/sourcedata/freesurfer/sub-14/mri/orig/001.mgz /out/sourcedata/freesurfer/sub-14/mri/rawavg.mgz 

     mri_info /out/sourcedata/freesurfer/sub-14/mri/rawavg.mgz 

    rawavg.mgz ========================================
    Volume information for /out/sourcedata/freesurfer/sub-14/mri/rawavg.mgz
              type: MGH
        dimensions: 224 x 220 x 224
       voxel sizes: 1.000000, 1.000000, 1.000000
              type: FLOAT (3)
               fov: 224.000
               dof: 1
            xstart: -112.0, xend: 112.0
            ystart: -110.0, yend: 110.0
            zstart: -112.0, zend: 112.0
                TR: 0.00 msec, TE: 0.00 msec, TI: 0.00 msec, flip angle: 0.00 degrees
           nframes: 1
           PhEncDir: UNKNOWN
           FieldStrength: 0.000000
    ras xform present
        xform info: x_r =   0.9981, y_r =  -0.0317, z_r =   0.0523, c_r =     2.7663
                  : x_a =   0.0460, y_a =   0.9530, z_a =  -0.2995, c_a =   -11.0132
                  : x_s =  -0.0404, y_s =   0.3013, z_s =   0.9527, c_s =    -5.3312

    talairach xfm : 
    Orientation   : RAS
    Primary Slice Direction: axial

    voxel to ras transform:
                    0.9981  -0.0317   0.0523  -111.3966
                    0.0460   0.9530  -0.2995   -87.4536
                   -0.0404   0.3013   0.9527  -140.6519
                    0.0000   0.0000   0.0000     1.0000

    voxel-to-ras determinant 1

    ras to voxel transform:
                    0.9981   0.0460  -0.0404   109.5300
                   -0.0317   0.9530   0.3013   122.1897
                    0.0523  -0.2995   0.9527   113.6361
                    0.0000   0.0000   0.0000     1.0000
    /out/sourcedata/freesurfer/sub-14

     mri_convert /out/sourcedata/freesurfer/sub-14/mri/rawavg.mgz /out/sourcedata/freesurfer/sub-14/mri/orig.mgz --conform 

    mri_convert /out/sourcedata/freesurfer/sub-14/mri/rawavg.mgz /out/sourcedata/freesurfer/sub-14/mri/orig.mgz --conform 
    reading from /out/sourcedata/freesurfer/sub-14/mri/rawavg.mgz...
    TR=0.00, TE=0.00, TI=0.00, flip angle=0.00
    i_ras = (0.998126, 0.0459843, -0.0403783)
    j_ras = (-0.0317161, 0.952999, 0.301308)
    k_ras = (0.052336, -0.299462, 0.952672)
    changing data type from float to uchar (noscale = 0)...
    MRIchangeType: Building histogram -12222.6 240095 1000, flo=0, fhi=0.999, dest_type=0
    Reslicing using trilinear interpolation 
    writing to /out/sourcedata/freesurfer/sub-14/mri/orig.mgz...

     mri_add_xform_to_header -c /out/sourcedata/freesurfer/sub-14/mri/transforms/talairach.xfm /out/sourcedata/freesurfer/sub-14/mri/orig.mgz /out/sourcedata/freesurfer/sub-14/mri/orig.mgz 

    INFO: extension is mgz

     mri_info /out/sourcedata/freesurfer/sub-14/mri/orig.mgz 

    orig.mgz ========================================
    Volume information for /out/sourcedata/freesurfer/sub-14/mri/orig.mgz
              type: MGH
        dimensions: 256 x 256 x 256
       voxel sizes: 1.000000, 1.000000, 1.000000
              type: UCHAR (0)
               fov: 256.000
               dof: 1
            xstart: -128.0, xend: 128.0
            ystart: -128.0, yend: 128.0
            zstart: -128.0, zend: 128.0
                TR: 0.00 msec, TE: 0.00 msec, TI: 0.00 msec, flip angle: 0.00 degrees
           nframes: 1
           PhEncDir: UNKNOWN
           FieldStrength: 0.000000
    ras xform present
        xform info: x_r =  -1.0000, y_r =   0.0000, z_r =   0.0000, c_r =     2.7663
                  : x_a =   0.0000, y_a =   0.0000, z_a =   1.0000, c_a =   -11.0132
                  : x_s =   0.0000, y_s =  -1.0000, z_s =  -0.0000, c_s =    -5.3312

    talairach xfm : /out/sourcedata/freesurfer/sub-14/mri/transforms/talairach.xfm
    Orientation   : LIA
    Primary Slice Direction: coronal

    voxel to ras transform:
                   -1.0000   0.0000   0.0000   130.7663
                    0.0000   0.0000   1.0000  -139.0132
                    0.0000  -1.0000  -0.0000   122.6688
                    0.0000   0.0000   0.0000     1.0000

    voxel-to-ras determinant -1

    ras to voxel transform:
                   -1.0000  -0.0000  -0.0000   130.7663
                   -0.0000  -0.0000  -1.0000   122.6688
                    0.0000   1.0000  -0.0000   139.0132
                   -0.0000  -0.0000  -0.0000     1.0000
    #--------------------------------------------
    #@# Talairach Tue Apr 30 19:11:42 UTC 2024
    /out/sourcedata/freesurfer/sub-14/mri

     mri_nu_correct.mni --no-rescale --i orig.mgz --o orig_nu.mgz --ants-n4 --n 1 --proto-iters 1000 --distance 50 

    /usr/bin/bc
    /out/sourcedata/freesurfer/sub-14/mri
    /opt/freesurfer/bin/mri_nu_correct.mni
    --no-rescale --i orig.mgz --o orig_nu.mgz --ants-n4 --n 1 --proto-iters 1000 --distance 50
    nIters 1
    mri_nu_correct.mni 7.3.2
    Linux 005abe4ae096 6.6.16-linuxkit #1 SMP Fri Feb 16 11:54:02 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
    Tue Apr 30 19:11:43 UTC 2024
    tmpdir is ./tmp.mri_nu_correct.mni.835
    cd /out/sourcedata/freesurfer/sub-14/mri
    AntsN4BiasFieldCorrectionFs -i orig.mgz -o ./tmp.mri_nu_correct.mni.835/nu0.mgz --dtype uchar
    AntsN4BiasFieldCorrectionFs done
    mri_convert ./tmp.mri_nu_correct.mni.835/nu0.mgz orig_nu.mgz --like orig.mgz --conform
    mri_convert ./tmp.mri_nu_correct.mni.835/nu0.mgz orig_nu.mgz --like orig.mgz --conform 
    reading from ./tmp.mri_nu_correct.mni.835/nu0.mgz...
    TR=0.00, TE=0.00, TI=0.00, flip angle=0.00
    i_ras = (-1, 1.86265e-09, 0)
    j_ras = (0, 0, -1)
    k_ras = (0, 1, -2.98023e-08)
    INFO: transform src into the like-volume: orig.mgz
    writing to orig_nu.mgz...

    Tue Apr 30 19:14:07 UTC 2024
    mri_nu_correct.mni done

     talairach_avi --i orig_nu.mgz --xfm transforms/talairach.auto.xfm 

    talairach_avi log file is transforms/talairach_avi.log...
    mv -f /out/sourcedata/freesurfer/sub-14/mri/talsrcimg_to_711-2C_as_mni_average_305_t4_vox2vox.txt /out/sourcedata/freesurfer/sub-14/mri/transforms/talsrcimg_to_711-2C_as_mni_average_305_t4_vox2vox.txt
    Started at Tue Apr 30 19:14:08 UTC 2024
    Ended   at Tue Apr 30 19:14:29 UTC 2024
    talairach_avi done

     cp transforms/talairach.auto.xfm transforms/talairach.xfm 

    lta_convert --src orig.mgz --trg /opt/freesurfer/average/mni305.cor.mgz --inxfm transforms/talairach.xfm --outlta transforms/talairach.xfm.lta --subject fsaverage --ltavox2vox
    7.3.2

    --src: orig.mgz src image (geometry).
    --trg: /opt/freesurfer/average/mni305.cor.mgz trg image (geometry).
    --inmni: transforms/talairach.xfm input MNI/XFM transform.
    --outlta: transforms/talairach.xfm.lta output LTA.
    --s: fsaverage subject name
    --ltavox2vox: output LTA as VOX_TO_VOX transform.
     LTA read, type : 1
     0.98559   0.03697  -0.04689  -0.52249;
    -0.02291   1.06278   0.24999  -3.72581;
     0.04425  -0.25222   0.99025  -0.46518;
     0.00000   0.00000   0.00000   1.00000;
    setting subject to fsaverage
    Writing  LTA to file transforms/talairach.xfm.lta...
    lta_convert successful.
    /out/sourcedata/freesurfer/sub-14/mri/transforms /out/sourcedata/freesurfer/sub-14/mri 
    /out/sourcedata/freesurfer/sub-14/mri 
    #--------------------------------------------
    #@# Talairach Failure Detection Tue Apr 30 19:14:31 UTC 2024
    /out/sourcedata/freesurfer/sub-14/mri

     talairach_afd -T 0.005 -xfm transforms/talairach.xfm 

    talairach_afd: Talairach Transform: transforms/talairach.xfm OK (p=0.6727, pval=0.3663 >= threshold=0.0050)

     awk -f /opt/freesurfer/bin/extract_talairach_avi_QA.awk /out/sourcedata/freesurfer/sub-14/mri/transforms/talairach_avi.log 

     tal_QC_AZS /out/sourcedata/freesurfer/sub-14/mri/transforms/talairach_avi.log 

    TalAviQA: 0.97662
    z-score: 0
    #--------------------------------------------
    #@# Nu Intensity Correction Tue Apr 30 19:14:31 UTC 2024

     mri_nu_correct.mni --i orig.mgz --o nu.mgz --uchar transforms/talairach.xfm --n 2 --ants-n4 

    /usr/bin/bc
    /out/sourcedata/freesurfer/sub-14/mri
    /opt/freesurfer/bin/mri_nu_correct.mni
    --i orig.mgz --o nu.mgz --uchar transforms/talairach.xfm --n 2 --ants-n4
    nIters 2
    mri_nu_correct.mni 7.3.2
    Linux 005abe4ae096 6.6.16-linuxkit #1 SMP Fri Feb 16 11:54:02 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
    Tue Apr 30 19:14:32 UTC 2024
    tmpdir is ./tmp.mri_nu_correct.mni.1094
    cd /out/sourcedata/freesurfer/sub-14/mri
    AntsN4BiasFieldCorrectionFs -i orig.mgz -o ./tmp.mri_nu_correct.mni.1094/nu0.mgz --dtype uchar
    AntsN4BiasFieldCorrectionFs done
    mri_binarize --i ./tmp.mri_nu_correct.mni.1094/nu0.mgz --min -1 --o ./tmp.mri_nu_correct.mni.1094/ones.mgz

    7.3.2
    cwd /out/sourcedata/freesurfer/sub-14/mri
    cmdline mri_binarize --i ./tmp.mri_nu_correct.mni.1094/nu0.mgz --min -1 --o ./tmp.mri_nu_correct.mni.1094/ones.mgz 
    sysname  Linux
    hostname 005abe4ae096
    machine  x86_64
    user     root

    input      ./tmp.mri_nu_correct.mni.1094/nu0.mgz
    frame      0
    nErode3d   0
    nErode2d   0
    output     ./tmp.mri_nu_correct.mni.1094/ones.mgz
    Binarizing based on threshold
    min        -1
    max        +infinity
    binval        1
    binvalnot     0
    fstart = 0, fend = 0, nframes = 1
    Starting parallel 1
    Found 16777216 values in range
    Counting number of voxels in first frame
    Found 16777215 voxels in final mask
    Writing output to ./tmp.mri_nu_correct.mni.1094/ones.mgz
    Count: 16777215 16777215.000000 16777216 99.999994
    mri_binarize done
    mri_segstats --id 1 --seg ./tmp.mri_nu_correct.mni.1094/ones.mgz --i orig.mgz --sum ./tmp.mri_nu_correct.mni.1094/sum.junk --avgwf ./tmp.mri_nu_correct.mni.1094/input.mean.dat

    7.3.2
    cwd 
    cmdline mri_segstats --id 1 --seg ./tmp.mri_nu_correct.mni.1094/ones.mgz --i orig.mgz --sum ./tmp.mri_nu_correct.mni.1094/sum.junk --avgwf ./tmp.mri_nu_correct.mni.1094/input.mean.dat 
    sysname  Linux
    hostname 005abe4ae096
    machine  x86_64
    user     root
    whitesurfname  white
    UseRobust  0
    Loading ./tmp.mri_nu_correct.mni.1094/ones.mgz
    Loading orig.mgz
    Voxel Volume is 1 mm^3
    Generating list of segmentation ids
    Found   1 segmentations
    Computing statistics for each segmentation

    Reporting on   1 segmentations
    Using PrintSegStat
    Computing spatial average of each frame

    Writing to ./tmp.mri_nu_correct.mni.1094/input.mean.dat
    mri_segstats done
    mri_segstats --id 1 --seg ./tmp.mri_nu_correct.mni.1094/ones.mgz --i ./tmp.mri_nu_correct.mni.1094/nu0.mgz --sum ./tmp.mri_nu_correct.mni.1094/sum.junk --avgwf ./tmp.mri_nu_correct.mni.1094/output.mean.dat

    7.3.2
    cwd 
    cmdline mri_segstats --id 1 --seg ./tmp.mri_nu_correct.mni.1094/ones.mgz --i ./tmp.mri_nu_correct.mni.1094/nu0.mgz --sum ./tmp.mri_nu_correct.mni.1094/sum.junk --avgwf ./tmp.mri_nu_correct.mni.1094/output.mean.dat 
    sysname  Linux
    hostname 005abe4ae096
    machine  x86_64
    user     root
    whitesurfname  white
    UseRobust  0
    Loading ./tmp.mri_nu_correct.mni.1094/ones.mgz
    Loading ./tmp.mri_nu_correct.mni.1094/nu0.mgz
    Voxel Volume is 1 mm^3
    Generating list of segmentation ids
    Found   1 segmentations
    Computing statistics for each segmentation

    Reporting on   1 segmentations
    Using PrintSegStat
    Computing spatial average of each frame

    Writing to ./tmp.mri_nu_correct.mni.1094/output.mean.dat
    mri_segstats done
    mris_calc -o ./tmp.mri_nu_correct.mni.1094/nu0.mgz ./tmp.mri_nu_correct.mni.1094/nu0.mgz mul 1.37393233565938209479
    Saving result to './tmp.mri_nu_correct.mni.1094/nu0.mgz' (type = MGH )                       [ ok ]
    mri_convert ./tmp.mri_nu_correct.mni.1094/nu0.mgz nu.mgz --like orig.mgz
    mri_convert ./tmp.mri_nu_correct.mni.1094/nu0.mgz nu.mgz --like orig.mgz 
    reading from ./tmp.mri_nu_correct.mni.1094/nu0.mgz...
    TR=0.00, TE=0.00, TI=0.00, flip angle=0.00
    i_ras = (-1, 1.86265e-09, 0)
    j_ras = (0, 0, -1)
    k_ras = (0, 1, -2.98023e-08)
    INFO: transform src into the like-volume: orig.mgz
    writing to nu.mgz...
    mri_make_uchar nu.mgz transforms/talairach.xfm nu.mgz
    type change took 0 minutes and 4 seconds.
    FIRST_PERCENTILE 0.010000
    WM_PERCENTILE    0.900000
    MAX_R 50.000000
    i1 = 10, i2 = 64
    #mri_make_uchar# mapping 24 152 to  3 110  :  b -17.3481 m 0.838262 : thresh 20.6954 maxsat 324.896 : nzero 12380809 nsat 0

    Tue Apr 30 19:17:17 UTC 2024
    mri_nu_correct.mni done

     mri_add_xform_to_header -c /out/sourcedata/freesurfer/sub-14/mri/transforms/talairach.xfm nu.mgz nu.mgz 

    INFO: extension is mgz
    #--------------------------------------------
    #@# Intensity Normalization Tue Apr 30 19:17:18 UTC 2024
    /out/sourcedata/freesurfer/sub-14/mri

     mri_normalize -g 1 -seed 1234 -mprage nu.mgz T1.mgz 

    upt: Subscript out of range.
    Linux 005abe4ae096 6.6.16-linuxkit #1 SMP Fri Feb 16 11:54:02 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

    recon-all -s sub-14 exited with ERRORS at Tue Apr 30 19:17:18 UTC 2024

    For more details, see the log file /out/sourcedata/freesurfer/sub-14/scripts/recon-all.log
    To report a problem, see http://surfer.nmr.mgh.harvard.edu/fswiki/BugReporting

Stderr:

Traceback:
    RuntimeError: subprocess exited with code 1.

Additional information / screenshots

No response

effigies commented 2 months ago

The only thing I can find about "upt: Subscript out of range." is https://github.com/freesurfer/freesurfer/issues/1042, but uptime is part of our docker images, so that seems unlikely to be the problem. Did you look for errors in /out/sourcedata/freesurfer/sub-14/scripts/recon-all.log?