search

edickie / ciftify

The tools of the Human Connectome Project (HCP) adapted for working with non-HCP datasets
https://edickie.github.io/ciftify/
MIT License
111 stars 156 forks source link

ciftify_recon_all fails occasionally with returncode 1 #168

Open CompuNeuroPsychiatryLabEinKerem opened 2 years ago

CompuNeuroPsychiatryLabEinKerem commented 2 years ago

Hi,

I'm running ciftify_bids_app using singularity. Usually it works great, but sometimes it fails with message:

cmd: ciftify_recon_all --n_cpus 1 --ciftify-work-dir /data/ciftify --fs-subjects-dir /data/freesurfer --surf-reg MSMSulc sub-SBJ1
 Failed with returncode 1

setNativeLocks failed: Resource temporarily unavailable
. . .

For the subjects I get this error for, I always get this error - also if I run them again. However, if I try to run them via docker on my desktop with the same inputs, it runs properly.

The shell I'm using is:

singularity run --cleanenv \
    -B $DATA_DIR:/data \
    -B $FS_LICENSE:/fs-license \
    $LAB_PATH/Singularity/tigrlab_fmriprep_ciftify_latest-2019-08-16-6ea864911724.simg \
    /data/BIDS /data participant  \
    --participant_label=${SBJ} \
    --fs-license="/fs-license/license.txt"  \
    --read-from-derivatives /data --debug --anat_only

And I'm running it via sbatch

Any Ideas?

Thanks you so much!

DESm1th commented 1 year ago

This is a very late response, and probably not needed anymore buuuut... are any of those directories you're using on NFS? That error crops up sometimes with singularity and it tends to happen much more frequently with NFS directories. Unfortunately it's not really something we can fix on our end

edickie commented 1 year ago

Also sorry to late respond to this. We've seen the "setNativeLocks failed:" message in other runs of ciftify - but it ciftify still finishes correctly - so it's more of an anoying warning message than a try error.