search

nipreps / fmriprep

fMRIPrep is a robust and easy-to-use pipeline for preprocessing of diverse fMRI data. The transparent workflow dispenses of manual intervention, thereby ensuring the reproducibility of the results.
https://fmriprep.org
Apache License 2.0
638 stars 294 forks source link

nipype.workflow CRITICAL: fMRIPrep failed: [Errno 13] Permission denied #3307

Closed CogBrainHealthLab closed 5 months ago

CogBrainHealthLab commented 5 months ago

What happened?

the above error appeared shortly after the fmriprep command was run. There was no error log. I've chmod 700 to the BIDS, work and output directories, but the issue still persists. Previously everything ran fine, i believe this error only started appearing after a recent maintenance was carried out at the HPC cluster. Perhaps some settings were changed after the maintenance, but i'm not sure where should i look for

What command did you use?

fmriprep 999_BIDS output participant --skip_bids_validation --output-spaces MNI152NLin6Asym --participant-label 999 --nthreads 2 --md-only-boilerplate

What version of fMRIPrep are you running?

23.2.3

How are you running fMRIPrep?

Local installation ("bare-metal")

Is your data BIDS valid?

Yes

Are you reusing any previously computed results?

FreeSurfer

Please copy and paste any relevant log output.

240617-20:51:45,564 cli WARNING:
         Telemetry disabled because sentry_sdk is not installed.
240617-20:52:00,775 nipype.workflow IMPORTANT:
         Running fMRIPrep version 23.2.3

         License NOTICE ##################################################
         fMRIPrep 23.2.3
         Copyright 2023 The NiPreps Developers.

         This product includes software developed by
         the NiPreps Community (https://nipreps.org/).

         Portions of this software were developed at the Department of
         Psychology at Stanford University, Stanford, CA, US.

         This software is also distributed as a Docker container image.
         The bootstrapping file for the image ("Dockerfile") is licensed
         under the MIT License.

         This software may be distributed through an add-on package called
         "Docker Wrapper" that is under the BSD 3-clause License.
         #################################################################
240617-20:52:00,837 nipype.workflow IMPORTANT:
         Building fMRIPrep's workflow:
           * BIDS dataset path: /home/junhong.yu/syn/CISC999_BIDS.
           * Participant list: ['CISC999'].
           * Run identifier: 20240617-205143_56482254-0c65-47b0-920f-b1718b942721.
           * Output spaces: MNI152NLin6Asym:res-native.
           * Pre-run FreeSurfer's SUBJECTS_DIR: /home/junhong.yu/syn/output/sourcedata/freesurfer.
240617-20:52:03,121 nipype.workflow INFO:
         ANAT Stage 1: Adding template workflow
240617-20:52:03,277 nipype.workflow INFO:
         ANAT Stage 2: Preparing brain extraction workflow
240617-20:52:03,381 nipype.workflow INFO:
         ANAT Stage 3: Preparing segmentation workflow
240617-20:52:03,381 nipype.interface WARNING:
         FSLOUTPUTTYPE environment variable is not set. Setting FSLOUTPUTTYPE=NIFTI
240617-20:52:03,381 nipype.interface WARNING:
         FSLOUTPUTTYPE environment variable is not set. Setting FSLOUTPUTTYPE=NIFTI
240617-20:52:03,386 nipype.workflow INFO:
         ANAT Stage 4: Preparing normalization workflow for ['MNI152NLin6Asym', 'MNI152NLin2009cAsym']
240617-20:52:03,399 nipype.workflow INFO:
         ANAT Stage 5: Preparing surface reconstruction workflow
240617-20:52:03,416 nipype.workflow INFO:
         ANAT Stage 6: Preparing mask refinement workflow
240617-20:52:03,418 nipype.interface WARNING:
         FSLOUTPUTTYPE environment variable is not set. Setting FSLOUTPUTTYPE=NIFTI
240617-20:52:03,418 nipype.workflow INFO:
         ANAT No T2w images provided - skipping Stage 7
240617-20:52:03,419 nipype.workflow INFO:
         ANAT Stage 8: Creating GIFTI surfaces for ['white', 'pial', 'midthickness', 'sphere_reg', 'sphere']
240617-20:52:03,433 nipype.workflow INFO:
         ANAT Stage 8: Creating GIFTI metrics for ['thickness', 'sulc']
240617-20:52:03,439 nipype.workflow INFO:
         ANAT Stage 8a: Creating cortical ribbon mask
240617-20:52:03,442 nipype.workflow INFO:
         ANAT Stage 9: Creating fsLR registration sphere
240617-20:52:03,455 nipype.workflow INFO:
         ANAT Stage 10: Creating MSM-Sulc registration sphere
240617-20:52:03,922 nipype.workflow INFO:
         Using single-band reference file(s) sub-CISC999_task-rest_sbref.nii.gz.
240617-20:52:04,10 nipype.workflow INFO:
         Stage 1: Adding HMC boldref workflow
240617-20:52:04,18 nipype.workflow INFO:
         Stage 2: Adding motion correction workflow
240617-20:52:04,25 nipype.interface WARNING:
         FSLOUTPUTTYPE environment variable is not set. Setting FSLOUTPUTTYPE=NIFTI
240617-20:52:04,30 nipype.workflow INFO:
         Stage 3: Adding coregistration boldref workflow
240617-20:52:04,31 nipype.interface WARNING:
         FSLOUTPUTTYPE environment variable is not set. Setting FSLOUTPUTTYPE=NIFTI
240617-20:52:04,32 nipype.interface WARNING:
         FSLOUTPUTTYPE environment variable is not set. Setting FSLOUTPUTTYPE=NIFTI
240617-20:52:04,188 nipype.workflow IMPORTANT:
         BOLD series will be slice-timing corrected to an offset of 0.348s.
240617-20:52:07,803 nipype.workflow INFO:
         fMRIPrep workflow graph with 401 nodes built successfully.
240617-20:52:21,451 nipype.workflow IMPORTANT:
         fMRIPrep started!
240617-20:52:21,513 nipype.workflow CRITICAL:
         fMRIPrep failed: [Errno 13] Permission denied

Additional information / screenshots

the work directory

work/
└── 20240617-205143_56482254-0c65-47b0-920f-b1718b942721
    ├── bids_db
    │   └── layout_index.sqlite
    └── config.toml

the subject output directory

sub-CISC999
└── log
    └── 20240617-205143_56482254-0c65-47b0-920f-b1718b942721
        └── fmriprep.toml
effigies commented 5 months ago

Can you run with -vv to see if there's any more debug information?

effigies commented 5 months ago

It also looks like you're not specifying a working directory with -w. It may be that creating the working directory is the thing that's failing. Pass a path you have write permissions to.

CogBrainHealthLab commented 5 months ago

thanks for the quick response. rerunning with -vv -w work:

240617-21:54:00,949 nipype.workflow IMPORTANT:
         Building fMRIPrep's workflow:
           * BIDS dataset path: /home/junhong.yu/syn/CISC999_BIDS.
           * Participant list: ['CISC999'].
           * Run identifier: 20240617-215348_4ad4a6f0-2caf-448a-8f62-11882e2e8702.
           * Output spaces: MNI152NLin6Asym:res-native.
           * Pre-run FreeSurfer's SUBJECTS_DIR: /home/junhong.yu/syn/output/sourcedata/freesurfer.
240617-21:54:02,184 nipype.workflow INFO:
         ANAT Stage 1: Adding template workflow
240617-21:54:02,323 nipype.workflow INFO:
         ANAT Stage 2: Preparing brain extraction workflow
240617-21:54:02,407 nipype.workflow INFO:
         ANAT Stage 3: Preparing segmentation workflow
240617-21:54:02,407 nipype.interface WARNING:
         FSLOUTPUTTYPE environment variable is not set. Setting FSLOUTPUTTYPE=NIFTI
240617-21:54:02,407 nipype.interface WARNING:
         FSLOUTPUTTYPE environment variable is not set. Setting FSLOUTPUTTYPE=NIFTI
240617-21:54:02,411 nipype.workflow INFO:
         ANAT Stage 4: Preparing normalization workflow for ['MNI152NLin6Asym', 'MNI152NLin2009cAsym']
240617-21:54:02,421 nipype.workflow INFO:
         ANAT Stage 5: Preparing surface reconstruction workflow
240617-21:54:02,438 nipype.workflow INFO:
         ANAT Stage 6: Preparing mask refinement workflow
240617-21:54:02,440 nipype.interface WARNING:
         FSLOUTPUTTYPE environment variable is not set. Setting FSLOUTPUTTYPE=NIFTI
240617-21:54:02,440 nipype.workflow INFO:
         ANAT No T2w images provided - skipping Stage 7
240617-21:54:02,440 nipype.workflow INFO:
         ANAT Stage 8: Creating GIFTI surfaces for ['white', 'pial', 'midthickness', 'sphere_reg', 'sphere']
240617-21:54:02,456 nipype.workflow INFO:
         ANAT Stage 8: Creating GIFTI metrics for ['thickness', 'sulc']
240617-21:54:02,462 nipype.workflow INFO:
         ANAT Stage 8a: Creating cortical ribbon mask
240617-21:54:02,465 nipype.workflow INFO:
         ANAT Stage 9: Creating fsLR registration sphere
240617-21:54:02,473 nipype.workflow INFO:
         ANAT Stage 10: Creating MSM-Sulc registration sphere
240617-21:54:02,909 nipype.workflow INFO:
         Using single-band reference file(s) sub-CISC999_task-rest_sbref.nii.gz.
240617-21:54:02,979 nipype.workflow INFO:
         Stage 1: Adding HMC boldref workflow
240617-21:54:02,985 nipype.workflow INFO:
         Stage 2: Adding motion correction workflow
240617-21:54:02,989 nipype.interface WARNING:
         FSLOUTPUTTYPE environment variable is not set. Setting FSLOUTPUTTYPE=NIFTI
240617-21:54:02,993 nipype.workflow INFO:
         Stage 3: Adding coregistration boldref workflow
240617-21:54:02,994 nipype.interface WARNING:
         FSLOUTPUTTYPE environment variable is not set. Setting FSLOUTPUTTYPE=NIFTI
240617-21:54:02,995 nipype.interface WARNING:
         FSLOUTPUTTYPE environment variable is not set. Setting FSLOUTPUTTYPE=NIFTI
240617-21:54:03,131 nipype.workflow IMPORTANT:
         BOLD series will be slice-timing corrected to an offset of 0.348s.
240617-21:54:06,630 nipype.workflow INFO:
         fMRIPrep workflow graph with 401 nodes built successfully.
240617-21:54:19,583 nipype.workflow VERBOSE:
         fMRIPrep config:
                [environment]
                cpu_count = 160
                exec_env = "posix"
                free_mem = 977.4
                overcommit_policy = "heuristic"
                overcommit_limit = "50%"
                nipype_version = "1.8.6"
                templateflow_version = "24.2.0"
                version = "23.2.3"

                [execution]
                bids_dir = "/home/junhong.yu/syn/CISC999_BIDS"
                derivatives = []
                bids_database_dir = "/home/junhong.yu/syn/work/20240617-215348_4ad4a6f0-2caf-448a-8f62-11882e2e8702/bids_db"
                bids_description_hash = "5f6cda964c42e35f19e7c36a3e67e62e6624b64d794319303577a8238dffdd52"
                boilerplate_only = false
                sloppy = false
                debug = []
                fmriprep_dir = "/home/junhong.yu/syn/output"
                fs_license_file = "/usr/local/freesurfer-7.4.1/license.txt"
                fs_subjects_dir = "/home/junhong.yu/syn/output/sourcedata/freesurfer"
                layout = "BIDS Layout: ...me/junhong.yu/syn/CISC999_BIDS | Subjects: 2 | Sessions: 0 | Runs: 0"
                log_dir = "/home/junhong.yu/syn/output/logs"
                log_level = 15
                low_mem = false
                md_only_boilerplate = true
                notrack = true
                track_carbon = false
                country_code = "CAN"
                output_dir = "/home/junhong.yu/syn/output"
                me_output_echos = false
                output_layout = "bids"
                output_spaces = "MNI152NLin6Asym:res-native"
                reports_only = false
                run_uuid = "20240617-215348_4ad4a6f0-2caf-448a-8f62-11882e2e8702"
                participant_label = [ "CISC999",]
                templateflow_home = "/home/junhong.yu/.cache/templateflow"
                work_dir = "/home/junhong.yu/syn/work"
                write_graph = false

                [workflow]
                anat_only = false
                aroma_err_on_warn = false
                aroma_melodic_dim = 0
                bold2t1w_dof = 6
                bold2t1w_init = "register"
                cifti_output = false
                fmap_bspline = false
                force_syn = false
                hires = true
                ignore = []
                level = "full"
                longitudinal = false
                run_msmsulc = true
                medial_surface_nan = false
                project_goodvoxels = false
                regressors_all_comps = false
                regressors_dvars_th = 1.5
                regressors_fd_th = 0.5
                run_reconall = true
                skull_strip_fixed_seed = false
                skull_strip_template = "OASIS30ANTs"
                skull_strip_t1w = "force"
                slice_time_ref = 0.5
                spaces = "MNI152NLin6Asym:res-native MNI152NLin2009cAsym"
                use_aroma = false
                use_syn_sdc = false
                me_t2s_fit_method = "curvefit"

                [nipype]
                crashfile_format = "txt"
                get_linked_libs = false
                nprocs = 8
                omp_nthreads = 7
                plugin = "MultiProc"
                remove_unnecessary_outputs = true
                resource_monitor = false
                stop_on_first_crash = false

                [seeds]
                master = 44477
                ants = 4779
                numpy = 42724

                [nipype.plugin_args]
                maxtasksperchild = 1
                raise_insufficient = false
240617-21:54:19,583 nipype.workflow IMPORTANT:
         fMRIPrep started!
240617-21:54:19,605 nipype.workflow CRITICAL:
         fMRIPrep failed: [Errno 13] Permission denied
effigies commented 5 months ago

Well that's really not very helpful, huh? Can you try adding --debug pdb and see if you can get more information out of the stack trace?

It looks like the error should be here:

https://github.com/nipreps/fmriprep/blob/87487457c4afbcdc7d839c39ad1baf4f2be2554c/fmriprep/cli/run.py#L154-L171

Once in the pdb shell, you should be able to

pdb> import traceback
pdb> traceback.print_exception(e)
CogBrainHealthLab commented 5 months ago

yes this revealed that i had an invalid T1w image, and thats all i needed to know. thanks