search

nipreps / fmriprep

fMRIPrep is a robust and easy-to-use pipeline for preprocessing of diverse fMRI data. The transparent workflow dispenses of manual intervention, thereby ensuring the reproducibility of the results.
https://fmriprep.org
Apache License 2.0
639 stars 296 forks source link

error in generating confounds #2811

Closed IFaillenot closed 1 year ago

IFaillenot commented 2 years ago

What happened?

an error appears at the end of the process, during the generation of the confound_timeseries, for 16 subjects out of 20 (the 4 other subjects are nice, the same command was run for all subjects in same time) Most files were created but not the confound_timeserie.tsv files

Actually, I only need the WM and CSF times series for statistics. So may be they exist.... but where ?

What command did you use?

/opt/conda/bin/fmriprep /data/bids /data/bids/derivatives/fmriprep participant --participant-label 01CA --work-dir /data/codes --stop-on-first-crash --write-graph --use-aroma --fs-no-reconall --fd-spike-threshold 1.0 --dvars-spike-threshold 2 --fs-license-file /fsdir/license.txt --fs-subjects-dir /fsdir

in the script in was :
 cmd="srun singularity run \
    --cleanenv \
    --bind ${DATA_DIRECTORY}:/data --bind ${LOCAL_FREESURFER_DIR}:/fsdir \
    ${FMRIPREP_SINGULARITY_IMG} \
    /data/bids /data/bids/derivatives/fmriprep participant \
    --participant-label ${subject} \
    --work-dir /data/codes \
    --stop-on-first-crash \
    --write-graph \
    --use-aroma  \
    --fs-no-reconall \
    --fd-spike-threshold 1.0 \
    --dvars-spike-threshold 2 \
    --fs-license-file /fsdir/license.txt --fs-subjects-dir /fsdir"

What version of fMRIPrep are you running?

21.0.2

How are you running fMRIPrep?

Singularity

Is your data BIDS valid?

Yes

Are you reusing any previously computed results?

No

Please copy and paste any relevant log output.

Node Name: fmriprep_wf.single_subject_01CA_wf.func_preproc_ses_IRM_task_noPain_wf.aroma_confounds

File: /data/bids/derivatives/fmriprep/sub-01CA/log/20220706-120717_a7ef51a8-838f-47cc-880e-3282febe1aae/crash-20220706-121443-isabelle.faillenot-aroma_confounds-1db641e7-1820-416f-92e5-8e123e2adb1b.txt
Working Directory: /data/codes/fmriprep_wf/single_subject_01CA_wf/func_preproc_ses_IRM_task_noPain_wf/aroma_confounds
Inputs:

    function_str: def _to_join(in_file, join_file): """Join two tsv files if the join_file is not ``None``.""" from niworkflows.interfaces.utility import JoinTSVColumns if join_file is None: return in_file res = JoinTSVColumns(in_file=in_file, join_file=join_file).run() return res.outputs.out_file
    in_file: /data/codes/fmriprep_wf/single_subject_01CA_wf/func_preproc_ses_IRM_task_noPain_wf/bold_confounds_wf/spike_regressors/confounds_expansion_desc-motion_outliers.tsv
    join_file: /data/codes/fmriprep_wf/single_subject_01CA_wf/func_preproc_ses_IRM_task_noPain_wf/ica_aroma_wf/ica_aroma_confound_extraction/AROMAAggrCompAROMAConfounds.tsv

Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 516, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 741, in _run_command
    result = self._interface.run(cwd=outdir)
  File "/opt/conda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 428, in run
    runtime = self._run_interface(runtime)
  File "/opt/conda/lib/python3.8/site-packages/nipype/interfaces/utility/wrappers.py", line 142, in _run_interface
    out = function_handle(**args)
  File "", line 7, in _to_join
  File "/opt/conda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 428, in run
    runtime = self._run_interface(runtime)
  File "/opt/conda/lib/python3.8/site-packages/niworkflows/interfaces/utility.py", line 376, in _run_interface
    raise ValueError("Number of columns in datasets do not match")
ValueError: Number of columns in datasets do not match

Additional information / screenshots

hereby the output files :

/home/isabelle.faillenot/Bureau/isabelle/borgCeline/exploDMA/data/bids/derivatives/fmriprep/sub-01CA/ses-IRM/func/sub-01CA_ses-IRM_task-ctr_AROMAnoiseICs.csv /home/isabelle.faillenot/Bureau/isabelle/borgCeline/exploDMA/data/bids/derivatives/fmriprep/sub-01CA/ses-IRM/func/sub-01CA_ses-IRM_task-ctr_desc-MELODIC_mixing.tsv /home/isabelle.faillenot/Bureau/isabelle/borgCeline/exploDMA/data/bids/derivatives/fmriprep/sub-01CA/ses-IRM/func/sub-01CA_ses-IRM_task-ctr_from-scanner_to-T1w_mode-image_xfm.txt /home/isabelle.faillenot/Bureau/isabelle/borgCeline/exploDMA/data/bids/derivatives/fmriprep/sub-01CA/ses-IRM/func/sub-01CA_ses-IRM_task-ctr_from-T1w_to-scanner_mode-image_xfm.txt /home/isabelle.faillenot/Bureau/isabelle/borgCeline/exploDMA/data/bids/derivatives/fmriprep/sub-01CA/ses-IRM/func/sub-01CA_ses-IRM_task-ctr_space-MNI152NLin6Asym_desc-smoothAROMAnonaggr_bold.json /home/isabelle.faillenot/Bureau/isabelle/borgCeline/exploDMA/data/bids/derivatives/fmriprep/sub-01CA/ses-IRM/func/sub-01CA_ses-IRM_task-ctr_space-MNI152NLin6Asym_desc-smoothAROMAnonaggr_bold.nii.gz /home/isabelle.faillenot/Bureau/isabelle/borgCeline/exploDMA/data/bids/derivatives/fmriprep/sub-01CA/ses-IRM/func/sub-01CA_ses-IRM_task-ctr_space-MNI152NLin2009cAsym_boldref.nii.gz /home/isabelle.faillenot/Bureau/isabelle/borgCeline/exploDMA/data/bids/derivatives/fmriprep/sub-01CA/ses-IRM/func/sub-01CA_ses-IRM_task-ctr_space-MNI152NLin2009cAsym_desc-brain_mask.json /home/isabelle.faillenot/Bureau/isabelle/borgCeline/exploDMA/data/bids/derivatives/fmriprep/sub-01CA/ses-IRM/func/sub-01CA_ses-IRM_task-ctr_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz /home/isabelle.faillenot/Bureau/isabelle/borgCeline/exploDMA/data/bids/derivatives/fmriprep/sub-01CA/ses-IRM/func/sub-01CA_ses-IRM_task-ctr_space-MNI152NLin2009cAsym_desc-preproc_bold.json /home/isabelle.faillenot/Bureau/isabelle/borgCeline/exploDMA/data/bids/derivatives/fmriprep/sub-01CA/ses-IRM/func/sub-01CA_ses-IRM_task-ctr_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz

IFaillenot commented 2 years ago

I have a solution: I found the WM and csf time-series in: ${WD}/${my_study}/data/codes/fmriprep_wf/singlesubject${subject}_wf/func_preprocses${session_name}task${task1}_wf/bold_confounds_wf/signals/signals.tsv

effigies commented 2 years ago

It seems like the issue is with AROMA. If you're only using CSF/WM, then presumably you could just drop --use-aroma?

IFaillenot commented 2 years ago

thanks for answer ! not really a solution because I use the the nonAgg images.

effigies commented 2 years ago

Fair enough. If you could share these files:

    in_file: /data/codes/fmriprep_wf/single_subject_01CA_wf/func_preproc_ses_IRM_task_noPain_wf/bold_confounds_wf/spike_regressors/confounds_expansion_desc-motion_outliers.tsv
    join_file: /data/codes/fmriprep_wf/single_subject_01CA_wf/func_preproc_ses_IRM_task_noPain_wf/ica_aroma_wf/ica_aroma_confound_extraction/AROMAAggrCompAROMAConfounds.tsv

We should be able to see what's going on here.

IFaillenot commented 2 years ago

confounds_expansion_desc-motion_outliers.tsv.zip AROMAAggrCompAROMAConfounds.tsv.zip

effigies commented 1 year ago

Unfortunately this fell off our radar and is now stale. If you find this issue persists in 23.2.0a1, please reopen.