IndexError: list index out of range

[eburkevt]

I am running xcp_d version 0.0.9 using a singularity container. xcp_d runs fine at first (for a minute or so), but eventually there is an IndexError: list index out of range error. It seems there is an out-of-bounds indexing problem.

Here is the output (single subject):

220510-15:39:29,571 nipype.workflow IMPORTANT:

Running xcp_abcd version 0.0.9+0.g067fabf.dirty:
  * fMRI directory path: /onrc/home/pipeline/data/pipelineb/home/Chyatt/onrc/data/fmriprep/yaleemoer/derivatives/fmriprep.
  * Participant list: ['S3676EYE'].
  * Run identifier: 20220510-153853_40111470-7cdd-4222-880d-7e026e41aab0.

Process Process-2: Traceback (most recent call last): File "/usr/local/miniconda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/usr/local/miniconda/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, *self._kwargs) File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/cli/run.py", line 486, in build_workflow retval['workflow'] = init_xcpd_wf ( File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflow/base.py", line 145, in init_xcpd_wf single_subj_wf = init_subject_wf( File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflow/base.py", line 396, in init_subject_wf bold_postproc_wf = init_boldpostprocess_wf(bold_file=bold_file, File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflow/bold.py", line 290, in init_boldpostprocess_wf executivesummary_wf =init_execsummary_wf(tr=TR,bold_file=bold_file,layout=layout,mem_gb=mem_gbx['timeseries'], File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflow/execsummary.py", line 44, in init_execsummary_wf bold_t1w_reg = fnmatch.filter(all_files, '' + prefixbb + coregbbregfile[0])[0] IndexError: list index out of range

So the error occurs in execsummary.py at line 44. Here is a snippet of the relevant Python code

get bbregsiter file from fmriprep

all_files  =list(layout.get_files())
filenamex = os.path.basename(bold_file)
if '_space' in filenamex:
    prefixbb = filenamex.split('_space')[0]
else:
    prefixbb = filenamex.split('_desc')[0]

# check if there is bbregister or coregister
patterns = ('*bbregister_bold.svg','*coreg_bold.svg','*bbr_bold.svg')
coregbbregfile = [pat for pat in patterns if fnmatch.filter(all_files,pat)]
bold_t1w_reg = fnmatch.filter(all_files, '*' + prefixbb + coregbbregfile[0])[0]

Line 44 is the last line in the code above, _bold_t1w_reg = fnmatch.filter(allfiles, '*' + prefixbb + coregbbregfile[0])[0].

Here is my singularity call

singularity run --cleanenv ${XCPD}/xcp_d-0.0.9.simg \ ${FMRIDIR} ${OUTPUTDIR} \ -w ${basefolder}/work_xcp-d \ -p aroma \ --lower-bpf 0.008 --upper-bpf 0.15 \ --nthreads 2 --omp-nthreads 2 \ --participant-label ${subjid}

The output directory is created successfully.

I'm running under CentOS Linux release 7.5.1804, on fMRIPrep v20.2.6 output data (in BIDS format). I'm trying to process only a single subject. This subject has two sessions in two subfolders, /ses-01 and /ses-02. The /anat data is in /ses-01 only. In /ses-01, there is a func folder with one resting state run and six runs of the same task, and in /ses-02, there is a func folder with just four runs of the same task.

I'll keep looking into what's causing the error, but thought I would post to see if anyone knows a possibly reason/solution to the problem.

Thanks for your help!

[eburkevt]

This error might be related to not having fmriprep Freesurfer output turned on (--fs-no-reconall was used for this subject). I'm rerunning with freesurfer output enabled.

Follow-up, the error is not related to Freesurfer output. It could be my fMRIPrep output data (stored in derivatives folder) is not in the correct format (my output data is in BIDS format, but might be missing files?), but it's not clear from the provided instructions in what format the output data should be to work properly with xcp_d.

There is this line of code

check if there is bbregister or coregister

patterns = ('*bbregister_bold.svg','*coreg_bold.svg','*bbr_bold.svg')

Is there a requirement for these files to be available? Is version 20.2.6 of fMRIPrep okay? Please advise.

[mb3152]

Did the end up working with Freesurfer enabled? We do require some of the outputs of Freesurfer.

[eburkevt]

Did the end up working with Freesurfer enabled? We do require some of the outputs of Freesurfer.

It does appear to be working now. There must be a problem with that subject.

[eburkevt]

It's not working. Tried a new subject, ran longer, but threw a different error. I'll try additional subjects.

Here is the error

Traceback (most recent call last): File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node result["result"] = node.run(updatehash=updatehash) File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 521, in run result = self._run_interface(execute=True) File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 639, in _run_interface return self._run_command(execute) File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 750, in _run_command raise NodeExecutionError( nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node ConfoundMatrix_wf.

Traceback (most recent call last): File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 398, in run runtime = self._run_interface(runtime) File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/interfaces/confound.py", line 83, in _run_interface data_matrix = load_confound_matrix(datafile=self.inputs.in_file, File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/utils/confounds.py", line 168, in load_confound_matrix return confound UnboundLocalError: local variable 'confound' referenced before assignment

[eburkevt]

I applied to a simpler sample subject, with just a single session, and a single resting state run and found a similar error ...

"UnboundLocalError: local variable 'confound' referenced before assignment"

Node: xcpabcd_wf.single_subject_hcS1252HGX_wf.bold_postprocess_1_wf.ConfoundMatrix_wf Working directory: /onrc/home/pipeline/data/pipelineb/home/Chyatt/onrc/data/fmriprep/work_xcp-d/xcpabcd_wf/single_subject_hcS1252HGX_wf/bold_postprocess_1_wf/ConfoundMatrix_wf

Node inputs:

TR = 0.9 cutoff = 0.0 filterorder = 4 filtertype = None head_radius = 50.0 high_freq = 0.0 in_file = /onrc/home/pipeline/data/pipelineb/home/Chyatt/onrc/data/fmriprep/rest900yale/derivatives/fmriprep/sub-hcS1252HGX/func/sub-hcS1252HGX_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz low_freq = 0.0 params = aroma

Traceback (most recent call last): File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node result["result"] = node.run(updatehash=updatehash) File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 521, in run result = self._run_interface(execute=True) File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 639, in _run_interface return self._run_command(execute) File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 750, in _run_command raise NodeExecutionError( nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node ConfoundMatrix_wf.

Traceback (most recent call last): File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 398, in run runtime = self._run_interface(runtime) File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/interfaces/confound.py", line 83, in _run_interface data_matrix = load_confound_matrix(datafile=self.inputs.in_file, File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/utils/confounds.py", line 168, in load_confound_matrix return confound UnboundLocalError: local variable 'confound' referenced before assignment

[eburkevt]

Looks like this a recent error that has been fixed (in #319). I'll close this issue.