search

bids-apps / HCPPipelines

A BIDS App for minimal preprocessing using the HCP Pipelines
33 stars 30 forks source link

ERROR: crypt() returned null with 4-line file #68

Closed Terf closed 3 years ago

Terf commented 3 years ago

When I run the HCP pipelines like:

singularity run -e \
    -B $PWD/nifti:/bids_dataset:ro \
    -B $PWD/hcppipelines/sub-${sub}:/output \
    $PWD/simg/hcppipelines_latest.sif \
    /bids_dataset /output participant --participant_label $sub \
    --n_cpus 1 --processing-mode legacy --license_key "XXXXX"

it's erroring out with an odd message: ERROR: crypt() returned null with 4-line file. Does anyone have any idea what that error could indicate? Below is the full output, written to /output/sub-01/T1w/sub-01/scripts/recon-all.log

[...]
Tue Mar  9 16:27:16 EST 2021:PreFreeSurferPipeline.sh: Completed!

FreeSurfer in LegacyStyleData mode
/opt/HCP-Pipelines/FreeSurfer/FreeSurferPipeline.sh --subject="sub-01" --subjectDIR="/output/sub-01/T1w" --t1="/output/sub-01/T1w/T1w_acpc_dc_restore.nii.gz" --t1brain
="/output/sub-01/T1w/T1w_acpc_dc_restore_brain.nii.gz" --processing-mode="LegacyStyleData"
========================================
  DIRECTORY: /opt/HCP-Pipelines  
    PRODUCT: HCP Pipeline Scripts
    VERSION: v4.1.3
========================================
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: HCPPIPEDIR: /opt/HCP-Pipelines
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: FREESURFER_HOME: /opt/freesurfer
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: Platform Information Follows:
Linux pennsive06 3.10.0-1160.11.1.el7.x86_64 #1 SMP Fri Dec 18 16:34:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: WARNING: We were not able to locate one of the following required tools:
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: WARNING: recon-all.v6.hires, conf2hires, or longmc
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: WARNING:
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: WARNING: To be able to run this script using the standard versions of these tools,
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: WARNING: we added /opt/HCP-Pipelines/FreeSurfer/custom to the beginning of the PATH.
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: WARNING:
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: WARNING: If you intended to use some other version of these tools, please configure
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: WARNING: your PATH before invoking this script, such that the tools you intended to
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: WARNING: use can be found on the PATH.
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: WARNING:
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: WARNING: PATH set to: /opt/HCP-Pipelines/FreeSurfer/custom:/usr/local/fsl/bin:/usr/local/miniconda/bin:/opt/freesur
fer/bin:/opt/freesurfer/fsfast/bin:/opt/freesurfer/tktools:/opt/freesurfer/mni/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: Showing HCP Pipelines version
========================================
  DIRECTORY: /opt/HCP-Pipelines  
    PRODUCT: HCP Pipeline Scripts
    VERSION: v4.1.3
========================================
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: Showing recon-all.v6.hires version
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: /opt/HCP-Pipelines/FreeSurfer/custom/recon-all.v6.hires
freesurfer-Linux-centos6_x86_64-stable-pub-v6.0.1-f53a55a
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: Showing tkregister version
/opt/freesurfer/tktools/tkregister
stable6
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: Showing mri_concatenate_lta version
/opt/freesurfer/bin/mri_concatenate_lta
stable6
Tue Mar  9 16:27:21 EST 2021:FreeSurferPipeline.sh: Showing mri_surf2surf version
/opt/freesurfer/bin/mri_surf2surf
stable6
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: Showing fslmaths location
/usr/local/fsl/bin/fslmaths
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: INFO: Determined that FreeSurfer full version string is: freesurfer-Linux-centos6_x86_64-stable-pub-v6.0.1-f53a55a
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: INFO: Determined that FreeSurfer version is: 6.0.1
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: Using named parameters
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: LegacyStyleData mode requested.
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: NOTICE: You are using a mode that enables processing of acquisitions that do not
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh:         conform to the HCP specification as described in Glasser et al. (2013)!
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh:         Be aware that if the HCP requirements are not met, the level of data quality
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh:         can not be guaranteed and the Glasser et al. (2013) paper should not be used
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh:         in support of this workflow. A manuscript with comprehensive evaluation for
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh:         the LegacyStyleData processing mode is in active preparation and should be
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh:         appropriately cited when published.
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh:
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: The following LegacyStyleData settings were requested: --t2w-image= or --t2= not present or set to NONE
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: Subject Directory: /output/sub-01/T1w
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: Subject: sub-01
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: T1w Image: /output/sub-01/T1w/T1w_acpc_dc_restore.nii.gz
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: T1w Brain: /output/sub-01/T1w/T1w_acpc_dc_restore_brain.nii.gz
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: Include -conf2hires flag in recon-all: TRUE
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: ProcessingMode: LegacyStyleData
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: Starting main functionality
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: Retrieve positional parameters
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: SubjectDIR: /output/sub-01/T1w
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: SubjectID: sub-01
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: T1wImage: /output/sub-01/T1w/T1w_acpc_dc_restore.nii.gz
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: T1wImageBrain: /output/sub-01/T1w/T1w_acpc_dc_restore_brain.nii.gz
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: T2wImage: NONE
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: recon_all_seed:
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: flair: FALSE
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: existing_subject: FALSE
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: extra_reconall_args:
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: conf2hires: TRUE
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: Figure out the number of cores to use.
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: num_cores: 1
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: Thresholding T1w image to eliminate negative voxel values
Tue Mar  9 16:27:22 EST 2021:FreeSurferPipeline.sh: ...This produces a new file named: /output/sub-01/T1w/T1w_acpc_dc_restore_zero_threshold.nii.gz
Tue Mar  9 16:27:23 EST 2021:FreeSurferPipeline.sh: Call custom recon-all: recon-all.v6.hires
Tue Mar  9 16:27:23 EST 2021:FreeSurferPipeline.sh: ...recon_all_cmd: recon-all.v6.hires -subjid sub-01 -sd /output/sub-01/T1w -all -i /output/sub-01/T1w/T1w_acpc_dc_restore_zero_threshold.nii.gz -emregmask /output/sub-01/T1w/T1w_acpc_dc_restore_brain.nii.gz -openmp 1 -conf2hires
Subject Stamp: freesurfer-Linux-centos6_x86_64-stable-pub-v6.0.1-f53a55a
Current Stamp: freesurfer-Linux-centos6_x86_64-stable-pub-v6.0.1-f53a55a
INFO: SUBJECTS_DIR is /output/sub-01/T1w
Actual FREESURFER_HOME /opt/freesurfer
Linux pennsive06 3.10.0-1160.11.1.el7.x86_64 #1 SMP Fri Dec 18 16:34:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
'/opt/HCP-Pipelines/FreeSurfer/custom/recon-all.v6.hires' -> '/output/sub-01/T1w/sub-01/scripts/recon-all.local-copy'
/output/sub-01/T1w/sub-01

 mri_convert /output/sub-01/T1w/T1w_acpc_dc_restore_zero_threshold.nii.gz /output/sub-01/T1w/sub-01/mri/orig/001.mgz

mri_convert.bin /output/sub-01/T1w/T1w_acpc_dc_restore_zero_threshold.nii.gz /output/sub-01/T1w/sub-01/mri/orig/001.mgz
$Id: mri_convert.c,v 1.226 2016/02/26 16:15:24 mreuter Exp $
reading from /output/sub-01/T1w/T1w_acpc_dc_restore_zero_threshold.nii.gz...
ERROR: crypt() returned null with 4-line file
Linux pennsive06 3.10.0-1160.11.1.el7.x86_64 #1 SMP Fri Dec 18 16:34:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

recon-all -s sub-01 exited with ERRORS at Tue Mar  9 16:27:25 EST 2021

For more details, see the log file /output/sub-01/T1w/sub-01/scripts/recon-all.log
To report a problem, see http://surfer.nmr.mgh.harvard.edu/fswiki/BugReporting

Tue Mar  9 16:27:25 EST 2021:FreeSurferPipeline.sh: While running '/opt/HCP-Pipelines/FreeSurfer/FreeSurferPipeline.sh --subject=sub-01 --subjectDIR=/output/sub-01/T1w --t1=/output/sub-01/T1w/T1w_acpc_dc_restore.nii.gz --t1brain=/output/sub-01/T1w/T1w_acpc_dc_restore_brain.nii.gz --processing-mode=LegacyStyleData':
Tue Mar  9 16:27:25 EST 2021:FreeSurferPipeline.sh: ERROR: recon-all command failed with return_code: 1
Tue Mar  9 16:27:25 EST 2021:FreeSurferPipeline.sh: ERROR: recon-all command failed with return_code: 1
Tue Mar  9 16:27:25 EST 2021:FreeSurferPipeline.sh: ABORTING

Traceback (most recent call last):
  File "/run.py", line 402, in <module>
    stage_func()
  File "/run.py", line 103, in run_freesurfer
    "OMP_NUM_THREADS": str(args["n_cpus"])})
  File "/run.py", line 31, in run
    raise Exception("Non zero return code: %d"%process.returncode)
Exception: Non zero return code: 1
[INFO   ] == Command exit (modification check follows) =====
[INFO   ] The command had a non-zero exit code. If this is expected, you can save the changes with 'datalad save -d . -r -F .git/COMMIT_EDITMSG'
CommandError: 'singularity run -e -B /scratch/8275975/ds/nifti:/bids_dataset:ro -B /scratch/8275975/ds/hcppipelines/sub-01:/output /scratch/8275975/ds/simg/hcppipelines_latest.sif /bids_dataset /output participant --participant_label 01 --n_cpus 1 --processing-mode legacy --license_key 46946' failed with exitcode 1 under /scratch/8275975/ds

Originally I posted this issue in the HCP pipelines repository but the error seems to be coming from the BIDS app.

Terf commented 3 years ago

related: https://github.com/freesurfer/freesurfer/issues/848

Terf commented 3 years ago

closing as this was related to the configuration of my cluster (FIPS)

mgxd commented 3 years ago

Hi @Terf, any chance you can elaborate on the configuration changes needed to get past this issue - we have a user running into the same problem in nipreps/fmriprep#2480

Terf commented 3 years ago

@katjz is was who dealt with this- do you remember?

Terf commented 3 years ago

@mgxd Apparently the only solution is to turn FIPS off, or the FS license check will need to be bypassed, which there is a special backdoor for, but the FS developers asked us not to share this publicly. Sorry, wish I could help more; presumably they are working on a fix that doesn't require skipping the license check, I'd ask on the FS mailing list.

mgxd commented 3 years ago

thanks! we'll just force users to turn off FIPS for now