search

PennLINC / xcpEngine

Official public repository for the XCP Engine. This tool is deprecated in favor of XCP-D and ASLPrep.
MIT License
66 stars 42 forks source link

Regress module failure (for some of the subjects) #475

Open roeysc opened 2 years ago

roeysc commented 2 years ago

Describe the bug I ran a modified version of fc-acompcor pipeline, and it raised an error at the Regression module.

Cohort file Paste cohort file between the triple backticks

id0, id1,img
sub-038, task-cfg_run-1, sub-038/func/sub-038_task-cfg_run-1_space-T1w_desc-preproc_bold.nii.gz

Design File Paste your entire design (.dsn) file between the triple backticks

#!/usr/bin/env bash

###################################################################
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
###################################################################

###################################################################
# This design file stores the values of all variables required to
# execute a complete neuroimage processing pipeline. You may
# execute the analysis specified in this design file by calling
# (in any v4 or higher bash terminal):
#
# xcpEngine -d <design> -c <cohort> -o <output> <options>
#
# Variables fall into five general categories:
# * ANALYSIS VARIABLES are used at all stages of this analysis.
# * PIPELINE specifies the modules that comprise the analysis.
# * MODULE VARIABLES are used during one stage of the analysis.
#                  These are typically array variables with array
#                  indices equal to the index of the module that
#                  calls them.
# * OUTPUT VARIABLES may be used at all stages of the analysis.
#                  These are typically array variables with array
#                  indices equal to the value of the primary
#                  subject identifier. They will appear only in
#                  localised design files.
###################################################################

###################################################################
# ANALYSIS VARIABLES
###################################################################

analysis=fc_$(whoami)
design=/data/fc-acompcor-gsr.dsn
sequence=anatomical
standard=MNI%2x2x2

###################################################################
# PIPELINE
###################################################################

pipeline=prestats,confound2,regress,fcon,reho,alff,roiquant,norm,qcfc

###################################################################
# 1 PRESTATS
###################################################################

prestats_rerun[1]=1
prestats_cleanup[1]=1
prestats_process[1]=FMP

###################################################################
# 2 CONFOUND2
###################################################################

confound2_rps[2]=1
confound2_rms[2]=0
confound2_wm[2]=0
confound2_csf[2]=0
confound2_gsr[2]=1
confound2_acompcor[2]=1
confound2_tcompcor[2]=0
confound2_aroma[2]=0
confound2_past[2]=0
confound2_dx[2]=1
confound2_sq[2]=1
confound2_censor[2]=0
confound2_censor_contig[2]=0
confound2_rerun[2]=1
confound2_cleanup[2]=1

###################################################################
# 3  REGRESS
###################################################################

regress_tmpf[3]=butterworth
regress_hipass[3]=0.01
regress_lopass[3]=0.08
regress_tmpf_order[3]=1
regress_tmpf_pass[3]=2
regress_tmpf_ripple[3]=0.5
regress_tmpf_ripple2[3]=20
regress_dmdt[3]=2
regress_1ddt[3]=1
regress_smo[3]=6
regress_sptf[3]=susan
regress_usan[3]=default
regress_usan_space[3]=
regress_rerun[3]=0
regress_cleanup[3]=1
regress_process[3]=DMT-TMP-REG

###################################################################
# 4 FCON
###################################################################

fcon_atlas[4]=power264
fcon_metric[4]=corrcoef
fcon_rerun[4]=0
fcon_cleanup[4]=1

###################################################################
# 5 REHO
###################################################################

reho_nhood[5]=vertices
reho_roikw[5]=0 # does nothing at the moment
reho_sptf[5]=susan
reho_smo[5]=6
reho_rerun[5]=0
reho_cleanup[5]=1

###################################################################
# 6 ALFF
###################################################################

alff_hipass[6]=0.01
alff_lopass[6]=0.08
alff_sptf[6]=susan
alff_smo[6]=6
alff_rerun[6]=0
alff_cleanup[6]=1

###################################################################
# 7 ROIQUANT
###################################################################

roiquant_atlas[7]=power264
roiquant_globals[7]=1
roiquant_vol[7]=0
roiquant_rerun[7]=0
roiquant_cleanup[7]=1

###################################################################
# 8 NORM
###################################################################
norm_primary[8]=1
norm_rerun[8]=0
norm_cleanup[8]=1

##################################################################
# 9 QCFC
###################################################################
qcfc_atlas[9]=power264
qcfc_sig[9]=fdr
qcfc_rerun[9]=0
qcfc_cleanup[9]=1

Error message Paste your error message between the backticks

$XCPEDIR is /xcpEngine

Constructing a pipeline based on user specifications
····································································
· [D][/data/fc-acompcor-gsr.dsn]
· [I][/tmp]
· [C][/data/xcpengine_cohort_single_sub_38.csv]
· [O][/data/derivatives/xcpengine/fc-acompcor_gsr_Singularity]
· [M][Executing serially]
····································································
Date: 202202051926
UUID: a2a9c6c6-52e3-4cd9-a545-5aa33667615f

Checking general dependencies
····································································
· Version AFNI           AFNI_21.1.01 
· Version ANTs           3.0.0.0 
· Version FSL            fsl-5.0.10 
· Version C3D            1.1.0 
· Version JQ             jq-1.5
· Version XCP Engine     xcpEngine-v1.2.3
· R version              3.3.3 
· R scripting front-end  3.3.3 
· · RNifti version       1.3.0
· · optparse version     1.6.6
· · pracma version       2.3.3
· · signal version       0.7.6
····································································
All general dependencies are present.

Checking environment
All environmental variables are defined.

Checking module-specific dependencies

###################################################################
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
#                                                                 #
#  \u2297                      XCP \u2605 \u2605 \u2605 ENGINE                     \u2297  #
#                                                                 #
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
###################################################################

###################################################################
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
#                                                                 #
                  CURRENTLY PROCESSING MODULE:
                         \u2605 LOCALISER \u2605
#                                                                 #
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
###################################################################

[Delegating control to module]

###################################################################
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
#                                                                 #
#  \u2297                   XCP ENGINE LOCALISER                    \u2297  #
#                                                                 #
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
###################################################################

Current processing step:
Localising subject metadata...
····································································
· [Parsing subject identifiers]
· [Creating output directory]
· [/data/derivatives/xcpengine/fc-acompcor_gsr_Singularity/sub-038/task-cfg_run-1]
· [Initialising space mapping]
· [Initialising derivative maps]
· [Initialising atlas system]
· Ingress
· [Native space: sub-038_task-cfg_run-1_anatomical]
· [Standard space: MNI]
· [Anatomical space: sub-038_task-cfg_run-1_anatomical]
source space map '' is not an image
· [Priming analytic audit]
· [Priming quality assessment]
····································································
Processing step complete:
Localising subject metadata...

Current processing step:
Localising image
····································································
· [Verifying that images are present]
· [Conforming and localising images]
····································································
Processing step complete:
Localising image

Current processing step:
Localising design
····································································
· [Writing localised design]
· [Evaluating subject variables]
····································································
Processing step complete:
Localising design

Localiser complete

###################################################################
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
#                                                                 #
                  CURRENTLY PROCESSING MODULE:
                         \u2605 prestats \u2605
#                                                                 #
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
###################################################################

[Delegating control to module]

###################################################################
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
#                                                                 #
#  \u2297                 FMRI PREPROCESSING MODULE                 \u2297  #
#                                                                 #
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
###################################################################

[I][/data/derivatives/xcpengine/fc-acompcor_gsr_Singularity/sub-038/task-cfg_run-1/sub-038_task-cfg_run-1.nii.gz]
[O][/data/derivatives/xcpengine/fc-acompcor_gsr_Singularity/sub-038/task-cfg_run-1/prestats]

Current processing step:
Ensure matching orientation
····································································
· Input: /tmp/sub-038_task-cfg_run-1-prestats-259968853~TEMP~.nii.gz
· Template: /xcpEngine/space/MNI/MNI-2x2x2.nii.gz
· Output root:
· checking the orientation of img and template
NATIVE:LPI TEMPLATE:LPI
· NOT re-orienting native bcos they are the same

Current processing step:
getting data from fmriprep directory
····································································

                   The template is MNI
                   xcpEngine only support MNI, PNC and OASIS templates for now

· generate new /data/derivatives/xcpengine/fc-acompcor_gsr_Singularity/sub-038/task-cfg_run-1/sub-038_task-cfg_run-1_spaces.json with spaceMetadata
ls: cannot access '/xcpEngine/space/OASIS/OASIS_transforms/MNI-OASIS_0Affine.mat,': No such file or directory
ls: cannot access '/xcpEngine/space/PNC/PNC_transforms/MNI-PNC_0Affine.mat,': No such file or directory
· Quality assessment
· [Cross-correlation:    0.9004592 ]
· [Coverage:             0.9696876 ]
· [Jaccard coefficient:  0.8719464 ]
· [Dice coefficient:     0.9315933 ]
····································································
Processing step complete:
getting data from fmriprep directory

Module complete

###################################################################
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
#                                                                 #
                  CURRENTLY PROCESSING MODULE:
                         \u2605 confound2 \u2605
#                                                                 #
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
###################################################################

[Delegating control to module]

###################################################################
#  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  #
#                                                                 #
#  ·                   CONFOUND MODEL MODULE                   ·  #
#                                                                 #
#  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  #
###################################################################

[I][/data/derivatives/xcpengine/fc-acompcor_gsr_Singularity/sub-038/task-cfg_run-1/sub-038_task-cfg_run-1.nii.gz]
[O][/data/derivatives/xcpengine/fc-acompcor_gsr_Singularity/sub-038/task-cfg_run-1/confound2]

Current processing step:
Generating confound matrix
····································································
· Generating temporal mask
· [Computing framewise displacement]
· [Computing DVARS]
Error in ftvecs[, i] <- mats[[i]][, 1] : 
  number of items to replace is not a multiple of replacement length
Execution halted
· [Selecting quality criteria]
· [Applying framewise threshold to time series]
· [Evaluating data quality]
· relative maximum motion
· relative mean motion
The nuisance matrix is being selected base on design files
· Including realignment parameters
· Including gsr
· [Including 1 derivative(s)]
· Including acompcor
····································································
Processing step complete:
Generating confound matrix

Current processing step:
Validating confound model
····································································
····································································
Processing step complete:
Validating confound model

Module complete

###################################################################
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
#                                                                 #
                  CURRENTLY PROCESSING MODULE:
                         \u2605 regress \u2605
#                                                                 #
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
###################################################################

[Delegating control to module]

###################################################################
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
#                                                                 #
#  \u2297                CONFOUND REGRESSION MODULE                 \u2297  #
#                                                                 #
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
###################################################################

[I][/data/derivatives/xcpengine/fc-acompcor_gsr_Singularity/sub-038/task-cfg_run-1/sub-038_task-cfg_run-1.nii.gz]
[O][/data/derivatives/xcpengine/fc-acompcor_gsr_Singularity/sub-038/task-cfg_run-1/regress]
Design files  
1.000000

Current processing step:
Demeaning and detrending BOLD timeseries
····································································
· [Applying polynomial detrend]
· [Order: 2]
· [Executing detrend: Confound matrix]
····································································
Processing step complete:
Demeaning and detrending BOLD timeseries

Current processing step:
Temporally filtering image and confounds
····································································
· [butterworth filter]
· [High pass frequency: 0.01]
· [Low pass frequency: 0.08]
Volume mismatch: confmat:/tmp/sub-038_task-cfg_run-1-regress-023393355~TEMP~_DMT_confmat.1D

· [A major error has occurred.]
· [The processing stream will now abort.]
· [Preparing diagnostics]
····································································

Module Workflow Map
····································································
· START
· @0.1
· @7
· @7.2
· @7.5a
· @7.5b
· @7.6
· @2
· @2.2
· @2.3
· @2.5
· @2.6
· @2.8
· @2.12a
· @2.12b
· @2.12c
· ERROR
····································································

····································································
· [An error occurred while processing module regress.]
· [The error was detected at signpost @2.12c.]
· [The most recent command logged was]
· [tfilter -i /tmp/sub-038_task-cfg_run-1-regress-023393355~TEMP~_DMT.nii.gz -o /tmp/sub-038_task-cfg_run-1-regress-023393355~TEMP~_DMT_TMP.nii.gz -a 1.000000 -f butterworth -h 0.01 -l 0.08 -m /data/derivatives/xcpengine/fc-acompcor_gsr_Singularity/sub-038/task-cfg_run-1/prestats/sub-038_task-cfg_run-1_mask.nii.gz -r 1 -d 2 -1 confmat:/tmp/sub-038_task-cfg_run-1-regress-023393355~TEMP~_DMT_confmat.1D]
· [For additional details, view the processing log. For improved]
· [diagnostics, increment verbosity using the -t option.]
· []
· [Processing failed to produce required output: /tmp/sub-038_task-cfg_run-1-regress-023393355~TEMP~_DMT_TMP.nii.gz]
· []
· [stream abort]
····································································

###################################################################
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
#                                                                 #
                  CURRENTLY PROCESSING MODULE:
                         \u2605 fcon \u2605
#                                                                 #
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
###################################################################

[Delegating control to module]

····································································
· [Module failure occurred in a number of subjects. They have been]
· [removed from further analysis.]
· · [sub-038_task-cfg_run-1 removed from analysis]
····································································

###################################################################
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
#                                                                 #
                  CURRENTLY PROCESSING MODULE:
                         \u2605 reho \u2605
#                                                                 #
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
###################################################################

[Delegating control to module]

····································································
· [Module failure occurred in a number of subjects. They have been]
· [removed from further analysis.]
· · [sub-038_task-cfg_run-1 removed from analysis]
····································································

###################################################################
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
#                                                                 #
                  CURRENTLY PROCESSING MODULE:
                         \u2605 alff \u2605
#                                                                 #
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
###################################################################

[Delegating control to module]

····································································
· [Module failure occurred in a number of subjects. They have been]
· [removed from further analysis.]
· · [sub-038_task-cfg_run-1 removed from analysis]
····································································

###################################################################
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
#                                                                 #
                  CURRENTLY PROCESSING MODULE:
                         \u2605 roiquant \u2605
#                                                                 #
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
###################################################################

[Delegating control to module]

····································································
· [Module failure occurred in a number of subjects. They have been]
· [removed from further analysis.]
· · [sub-038_task-cfg_run-1 removed from analysis]
····································································

###################################################################
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
#                                                                 #
                  CURRENTLY PROCESSING MODULE:
                         \u2605 norm \u2605
#                                                                 #
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
###################################################################

[Delegating control to module]

····································································
· [Module failure occurred in a number of subjects. They have been]
· [removed from further analysis.]
· · [sub-038_task-cfg_run-1 removed from analysis]
····································································

###################################################################
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
#                                                                 #
                  CURRENTLY PROCESSING MODULE:
                         \u2605 qcfc \u2605
#                                                                 #
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
###################################################################

[Delegating control to module]

····································································
· [Module failure occurred in a number of subjects. They have been]
· [removed from further analysis.]
· · [sub-038_task-cfg_run-1 removed from analysis]
····································································
generating report

###################################################################
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
#                                                                 #
#  \u2297                     ANALYSIS COMPLETE                     \u2297  #
#                                                                 #
#  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  \u2297  #
###################################################################

Runtime Information How were you running the software? Docker? Singularity? Directly? If Directly, please consider using Docker or Singluarity first before submitting a bug report. I used Singularity.

Additional context The same call for running this pipeline worked well on other subjects. Following Issue #209, I removed the output task directory created by xcpengine and called xcpengine again, but received the same error. Thank you for your help!