search

FCP-INDI / C-PAC

Configurable Pipeline for the Analysis of Connectomes
https://fcp-indi.github.io/
GNU Lesser General Public License v3.0
62 stars 40 forks source link

🐛 Build_nuisance_regressor crash: number of time points is inconsistent [User-reported Bug] #1782

Closed nimzodisaster closed 1 year ago

nimzodisaster commented 1 year ago

Describe the bug

Build_nuisance_regressor crash: number of time points is inconsistent

Working directory: /data/proj/app/pipeline/rs/outputs/working/cpac_103430-100_3_ses-1/nuisance_regressors_default_271/_scan_func-1/build_nuisance_regressors

Node inputs:

acompcor_file_path = <undefined>
censor_file_path = <undefined>
csf_summary_file_path = <undefined>
custom_file_paths = <undefined>
function_str = CPAC.nuisance.nuisance.gather_nuisance
functional_file_path = <undefined>
global_summary_file_path = <undefined>
grey_matter_summary_file_path = <undefined>
motion_parameters_file_path = <undefined>
selector = {'Bandpass': {'bottom_frequency': 0.01, 'method': 'default', 'top_frequency': 0.1}, 'CerebrospinalFluid': {'erode_mask': True, 'extraction_resolution': 2, 'summary': {'filter': 'Mean', 'method': 'Mean'}}, 'GlobalSignal': {'summary': {'filter': 'Mean', 'method': 'Mean'}}, 'Motion': {'include_delayed': True, 'include_delayed_squared': True, 'include_squared': True}, 'Name': 'default', 'PolyOrt': {'degree': 2}, 'aCompCor': {'extraction_resolution': 2, 'summary': {'components': 5, 'method': 'DetrendPC'}, 'tissues': ['WhiteMatter', 'CerebrospinalFluid']}}
tcompcor_file_path = <undefined>
white_matter_summary_file_path = <undefined>

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/code/CPAC/pipeline/nipype_pipeline_engine/engine.py", line 406, in run
    return super().run(updatehash)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 516, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 741, in _run_command
    result = self._interface.run(cwd=outdir)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 419, in run
    runtime = self._run_interface(runtime)
  File "/code/CPAC/utils/interfaces/function/function.py", line 152, in _run_interface
    out = function_handle(**args)
  File "/code/CPAC/nuisance/nuisance.py", line 274, in gather_nuisance
    regressor_length))
ValueError: Number of time points in /data/proj/app/pipeline/rs/outputs/working/cpac_103430-100_3_ses-1/nuisance_regressors_default_271/_scan_func-1/CerebrospinalFluid_mean/denoised_func_data_nonaggr_0_trans_tcat_flirt_roistat.1D (0) is inconsistent with length of functional file /data/proj/app/pipeline/rs/outputs/working/cpac_103430-100_3_ses-1/ICA-AROMA_ANTs_template_to_bold_234/_scan_func-1/func_concat_ICA-AROMA_ANTs_template_to_bold_234/denoised_func_data_nonaggr_0_trans_tcat.nii.gz (300)

looking at the the two files, there is indeed the same number of data points in each.

last line of the 1D file: #/data/proj/app/pipeline/rs/outputs/working/cpac_103430-100_3_ses-1/nuisance_regressors_default_271/_scan_func-1/Functional_2mm_flirt/denoised_func_data_nonaggr_0_trans_tcat_flirt.nii.gz 299[?] which is indexed from 0, indicating 300.

and the relevant image dimensions: fslinfo denoised_func_data_nonaggr_0_trans_tcat.nii.gz

data_type   FLOAT32
dim1        64
dim2        64
dim3        37
dim4        300
datatype    16
pixdim1     4.000000
pixdim2     4.000000
pixdim3     4.000000
pixdim4     2.000000
cal_max     0.000000
cal_min     0.000000
file_type   NIFTI-1+

This crashed on every participant I ran. Here is a list of missing outputs:

- desc-Mean**-1**_timeseries
- desc-Mean**-2**_timeseries
- desc-Mean**-3**_timeseries
- desc-Mean**-4**_timeseries
- desc-Mean**-5**_timeseries
- desc-Mean**-6**_timeseries
- desc-PartialNilearn**-1**_correlations
- desc-PartialNilearn**-2**_correlations
- desc-PartialNilearn**-3**_correlations
- desc-PartialNilearn**-4**_correlations
- desc-PartialNilearn**-5**_correlations
- desc-PartialNilearn**-6**_correlations
- desc-PearsonNilearn**-1**_correlations
- desc-PearsonNilearn**-2**_correlations
- desc-PearsonNilearn**-3**_correlations
- desc-PearsonNilearn**-4**_correlations
- desc-PearsonNilearn**-5**_correlations
- desc-PearsonNilearn**-6**_correlations
- desc-SpatReg**-1**_timeseries
- desc-SpatReg**-10**_timeseries
- desc-SpatReg**-11**_timeseries
- desc-SpatReg**-12**_timeseries
- desc-SpatReg**-2**_timeseries
- desc-SpatReg**-3**_timeseries
- desc-SpatReg**-4**_timeseries
- desc-SpatReg**-5**_timeseries
- desc-SpatReg**-6**_timeseries
- desc-SpatReg**-7**_timeseries
- desc-SpatReg**-8**_timeseries
- desc-SpatReg**-9**_timeseries
- desc-ndmg**-1**_correlations
- desc-ndmg**-2**_correlations
- desc-ndmg**-3**_correlations
- desc-ndmg**-4**_correlations
- desc-ndmg**-5**_correlations
- desc-ndmg**-6**_correlations
- desc-preproc**-1**_bold
- desc-preproc**-2**_bold
- desc-preproc**-3**_bold
- desc-preproc**-4**_bold
- desc-preproc**-5**_bold
- desc-preproc**-6**_bold
- desc-sm**-zstd**-10**_alff
- desc-sm**-zstd**-10**_falff
- desc-sm**-zstd**-10**_reho
- desc-sm**-zstd**-11**_alff
- desc-sm**-zstd**-11**_falff
- desc-sm**-zstd**-11**_reho
- desc-sm**-zstd**-12**_alff
- desc-sm**-zstd**-12**_falff
- desc-sm**-zstd**-12**_reho
- desc-sm**-zstd**-13**_alff
- desc-sm**-zstd**-13**_falff
- desc-sm**-zstd**-13**_reho
- desc-sm**-zstd**-14**_alff
- desc-sm**-zstd**-14**_falff
- desc-sm**-zstd**-14**_reho
- desc-sm**-zstd**-15**_alff
- desc-sm**-zstd**-15**_falff
- desc-sm**-zstd**-15**_reho
- desc-sm**-zstd**-16**_alff
- desc-sm**-zstd**-16**_falff
- desc-sm**-zstd**-16**_reho
- desc-sm**-zstd**-5**_alff
- desc-sm**-zstd**-5**_falff
- desc-sm**-zstd**-5**_reho
- desc-sm**-zstd**-6**_alff
- desc-sm**-zstd**-6**_falff
- desc-sm**-zstd**-6**_reho
- desc-sm**-zstd**-7**_alff
- desc-sm**-zstd**-7**_falff
- desc-sm**-zstd**-7**_reho
- desc-sm**-zstd**-8**_alff
- desc-sm**-zstd**-8**_falff
- desc-sm**-zstd**-8**_reho
- desc-sm**-zstd**-9**_alff
- desc-sm**-zstd**-9**_falff
- desc-sm**-zstd**-9**_reho
- desc-zstd**-10**_alff
- desc-zstd**-10**_falff
- desc-zstd**-10**_reho
- desc-zstd**-11**_alff
- desc-zstd**-11**_falff
- desc-zstd**-11**_reho
- desc-zstd**-12**_alff
- desc-zstd**-12**_falff
- desc-zstd**-12**_reho
- desc-zstd**-13**_alff
- desc-zstd**-13**_falff
- desc-zstd**-13**_reho
- desc-zstd**-14**_alff
- desc-zstd**-14**_falff
- desc-zstd**-14**_reho
- desc-zstd**-15**_alff
- desc-zstd**-15**_falff
- desc-zstd**-15**_reho
- desc-zstd**-16**_alff
- desc-zstd**-16**_falff
- desc-zstd**-16**_reho
- desc-zstd**-5**_alff
- desc-zstd**-5**_falff
- desc-zstd**-5**_reho
- desc-zstd**-6**_alff
- desc-zstd**-6**_falff
- desc-zstd**-6**_reho
- desc-zstd**-7**_alff
- desc-zstd**-7**_falff
- desc-zstd**-7**_reho
- desc-zstd**-8**_alff
- desc-zstd**-8**_falff
- desc-zstd**-8**_reho
- desc-zstd**-9**_alff
- desc-zstd**-9**_falff
- desc-zstd**-9**_reho
- regressors
- space-template**_desc-binarized**-sm**-zstd**-1**_degree-centrality
- space-template**_desc-binarized**-sm**-zstd**-1**_lfcd
- space-template**_desc-binarized**-sm**-zstd**-2**_degree-centrality
- space-template**_desc-binarized**-sm**-zstd**-2**_lfcd
- space-template**_desc-binarized**-sm**-zstd**-3**_degree-centrality
- space-template**_desc-binarized**-sm**-zstd**-3**_lfcd
- space-template**_desc-binarized**-sm**-zstd**-4**_degree-centrality
- space-template**_desc-binarized**-sm**-zstd**-4**_lfcd
- space-template**_desc-binarized**-sm**-zstd**-5**_degree-centrality
- space-template**_desc-binarized**-sm**-zstd**-5**_lfcd
- space-template**_desc-binarized**-sm**-zstd**-6**_degree-centrality
- space-template**_desc-binarized**-sm**-zstd**-6**_lfcd
- space-template**_desc-binarized**-zstd**-1**_degree-centrality
- space-template**_desc-binarized**-zstd**-1**_lfcd
- space-template**_desc-binarized**-zstd**-2**_degree-centrality
- space-template**_desc-binarized**-zstd**-2**_lfcd
- space-template**_desc-binarized**-zstd**-3**_degree-centrality
- space-template**_desc-binarized**-zstd**-3**_lfcd
- space-template**_desc-binarized**-zstd**-4**_degree-centrality
- space-template**_desc-binarized**-zstd**-4**_lfcd
- space-template**_desc-binarized**-zstd**-5**_degree-centrality
- space-template**_desc-binarized**-zstd**-5**_lfcd
- space-template**_desc-binarized**-zstd**-6**_degree-centrality
- space-template**_desc-binarized**-zstd**-6**_lfcd
- space-template**_desc-preproc**-1**_bold
- space-template**_desc-preproc**-2**_bold
- space-template**_desc-preproc**-3**_bold
- space-template**_desc-preproc**-4**_bold
- space-template**_desc-preproc**-5**_bold
- space-template**_desc-preproc**-6**_bold
- space-template**_desc-sm**-zstd**-10**_alff
- space-template**_desc-sm**-zstd**-10**_falff
- space-template**_desc-sm**-zstd**-10**_reho
- space-template**_desc-sm**-zstd**-11**_alff
- space-template**_desc-sm**-zstd**-11**_falff
- space-template**_desc-sm**-zstd**-11**_reho
- space-template**_desc-sm**-zstd**-12**_alff
- space-template**_desc-sm**-zstd**-12**_falff
- space-template**_desc-sm**-zstd**-12**_reho
- space-template**_desc-sm**-zstd**-13**_alff
- space-template**_desc-sm**-zstd**-13**_falff
- space-template**_desc-sm**-zstd**-13**_reho
- space-template**_desc-sm**-zstd**-14**_alff
- space-template**_desc-sm**-zstd**-14**_falff
- space-template**_desc-sm**-zstd**-14**_reho
- space-template**_desc-sm**-zstd**-15**_alff
- space-template**_desc-sm**-zstd**-15**_falff
- space-template**_desc-sm**-zstd**-15**_reho
- space-template**_desc-sm**-zstd**-16**_alff
- space-template**_desc-sm**-zstd**-16**_falff
- space-template**_desc-sm**-zstd**-16**_reho
- space-template**_desc-sm**-zstd**-5**_alff
- space-template**_desc-sm**-zstd**-5**_falff
- space-template**_desc-sm**-zstd**-5**_reho
- space-template**_desc-sm**-zstd**-6**_alff
- space-template**_desc-sm**-zstd**-6**_falff
- space-template**_desc-sm**-zstd**-6**_reho
- space-template**_desc-sm**-zstd**-7**_alff
- space-template**_desc-sm**-zstd**-7**_falff
- space-template**_desc-sm**-zstd**-7**_reho
- space-template**_desc-sm**-zstd**-8**_alff
- space-template**_desc-sm**-zstd**-8**_falff
- space-template**_desc-sm**-zstd**-8**_reho
- space-template**_desc-sm**-zstd**-9**_alff
- space-template**_desc-sm**-zstd**-9**_falff
- space-template**_desc-sm**-zstd**-9**_reho
- space-template**_desc-weighted**-sm**-zstd**-1**_degree-centrality
- space-template**_desc-weighted**-sm**-zstd**-1**_eigen-centrality
- space-template**_desc-weighted**-sm**-zstd**-1**_lfcd
- space-template**_desc-weighted**-sm**-zstd**-2**_degree-centrality
- space-template**_desc-weighted**-sm**-zstd**-2**_eigen-centrality
- space-template**_desc-weighted**-sm**-zstd**-2**_lfcd
- space-template**_desc-weighted**-sm**-zstd**-3**_degree-centrality
- space-template**_desc-weighted**-sm**-zstd**-3**_eigen-centrality
- space-template**_desc-weighted**-sm**-zstd**-3**_lfcd
- space-template**_desc-weighted**-sm**-zstd**-4**_degree-centrality
- space-template**_desc-weighted**-sm**-zstd**-4**_eigen-centrality
- space-template**_desc-weighted**-sm**-zstd**-4**_lfcd
- space-template**_desc-weighted**-sm**-zstd**-5**_degree-centrality
- space-template**_desc-weighted**-sm**-zstd**-5**_eigen-centrality
- space-template**_desc-weighted**-sm**-zstd**-5**_lfcd
- space-template**_desc-weighted**-sm**-zstd**-6**_degree-centrality
- space-template**_desc-weighted**-sm**-zstd**-6**_eigen-centrality
- space-template**_desc-weighted**-sm**-zstd**-6**_lfcd
- space-template**_desc-weighted**-zstd**-1**_degree-centrality
- space-template**_desc-weighted**-zstd**-1**_eigen-centrality
- space-template**_desc-weighted**-zstd**-1**_lfcd
- space-template**_desc-weighted**-zstd**-2**_degree-centrality
- space-template**_desc-weighted**-zstd**-2**_eigen-centrality
- space-template**_desc-weighted**-zstd**-2**_lfcd
- space-template**_desc-weighted**-zstd**-3**_degree-centrality
- space-template**_desc-weighted**-zstd**-3**_eigen-centrality
- space-template**_desc-weighted**-zstd**-3**_lfcd
- space-template**_desc-weighted**-zstd**-4**_degree-centrality
- space-template**_desc-weighted**-zstd**-4**_eigen-centrality
- space-template**_desc-weighted**-zstd**-4**_lfcd
- space-template**_desc-weighted**-zstd**-5**_degree-centrality
- space-template**_desc-weighted**-zstd**-5**_eigen-centrality
- space-template**_desc-weighted**-zstd**-5**_lfcd
- space-template**_desc-weighted**-zstd**-6**_degree-centrality
- space-template**_desc-weighted**-zstd**-6**_eigen-centrality
- space-template**_desc-weighted**-zstd**-6**_lfcd
- space-template**_desc-zstd**-10**_alff
- space-template**_desc-zstd**-10**_falff
- space-template**_desc-zstd**-10**_reho
- space-template**_desc-zstd**-11**_alff
- space-template**_desc-zstd**-11**_falff
- space-template**_desc-zstd**-11**_reho
- space-template**_desc-zstd**-12**_alff
- space-template**_desc-zstd**-12**_falff
- space-template**_desc-zstd**-12**_reho
- space-template**_desc-zstd**-13**_alff
- space-template**_desc-zstd**-13**_falff
- space-template**_desc-zstd**-13**_reho
- space-template**_desc-zstd**-14**_alff
- space-template**_desc-zstd**-14**_falff
- space-template**_desc-zstd**-14**_reho
- space-template**_desc-zstd**-15**_alff
- space-template**_desc-zstd**-15**_falff
- space-template**_desc-zstd**-15**_reho
- space-template**_desc-zstd**-16**_alff
- space-template**_desc-zstd**-16**_falff
- space-template**_desc-zstd**-16**_reho
- space-template**_desc-zstd**-5**_alff
- space-template**_desc-zstd**-5**_falff
- space-template**_desc-zstd**-5**_reho
- space-template**_desc-zstd**-6**_alff
- space-template**_desc-zstd**-6**_falff
- space-template**_desc-zstd**-6**_reho
- space-template**_desc-zstd**-7**_alff
- space-template**_desc-zstd**-7**_falff
- space-template**_desc-zstd**-7**_reho
- space-template**_desc-zstd**-8**_alff
- space-template**_desc-zstd**-8**_falff
- space-template**_desc-zstd**-8**_reho
- space-template**_desc-zstd**-9**_alff
- space-template**_desc-zstd**-9**_falff
- space-template**_desc-zstd**-9**_reho

To reproduce

No response

Preconfig

Custom pipeline configuration

---
# CPAC Pipeline Configuration YAML file
# Version 1.8.4.dev
#
# http://fcp-indi.github.io for more info.
#
# Pipeline config "Default (3)", version GUI-0
# Mon Sep 12 2022 16:51:44 GMT-0700 (Pacific Daylight Time)
#
# Tip: This file can be edited manually with a text editor for quick modifications.

pipeline_setup:
  pipeline_name: cpac-APP-pipeline
  output_directory:
    path: /outputs/output
    source_outputs_dir: None
    pull_source_once: true
    write_func_outputs: false
    write_debugging_outputs: false
    output_tree: default
    quality_control:
      generate_quality_control_images: true
      generate_xcpqc_files: false
  working_directory:
    path: /outputs/working
    remove_working_dir: true
  log_directory:
    run_logging: true
    path: /outputs/log
  crash_log_directory:
    path: /outputs/crash
  system_config:
    num_OMP_threads: 3
    on_grid:
      run: false
      resource_manager: SGE
      SGE:
        parallel_environment: mpi_smp
        queue: all.q
    num_participants_at_once: 10
    observed_usage:
      callback_log: null
      buffer: 10
    max_cores_per_participant: 30
    random_seed: null
    FSLDIR: /usr/share/fsl/5.0
    num_ants_threads: 2
    maximum_memory_per_participant: 20
    raise_insufficient: true
  Amazon-AWS:
    aws_output_bucket_credentials: null
    s3_encryption: false
  Debugging:
    verbose: false
timeseries_extraction:
  run: true
  tse_roi_paths:
    /cpac_templates/PNAS_Smith09_rsn10.nii.gz: SpatialReg
    /ndmg_atlases/label/Human/Yeo-17_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /ndmg_atlases/label/Human/CAPRSC_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /cpac_templates/ho_mask_pad.nii.gz: Avg
    /cpac_templates/CC400.nii.gz: Avg
    /cpac_templates/CC200.nii.gz: Avg
    /ndmg_atlases/label/Human/DesikanKlein_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /ndmg_atlases/label/Human/DKT_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /ndmg_atlases/label/Human/Yeo-7_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /ndmg_atlases/label/Human/HarvardOxfordsub-maxprob-thr25_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /ndmg_atlases/label/Human/AAL_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /ndmg_atlases/label/Human/MICCAI_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /ndmg_atlases/label/Human/Glasser_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /ndmg_atlases/label/Human/HarvardOxfordcort-maxprob-thr25_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /cpac_templates/rois_3mm.nii.gz: Avg
    /ndmg_atlases/label/Human/Slab907_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /ndmg_atlases/label/Human/Yeo-17-liberal_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /ndmg_atlases/label/Human/Desikan_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /cpac_templates/aal_mask_pad.nii.gz: Avg
    /ndmg_atlases/label/Human/Brodmann_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /ndmg_atlases/label/Human/Yeo-7-liberal_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /cpac_templates/tt_mask_pad.nii.gz: Avg
    /ndmg_atlases/label/Human/Schaefer400_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /ndmg_atlases/label/Human/Juelich_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /ndmg_atlases/label/Human/Schaefer300_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /ndmg_atlases/label/Human/Schaefer1000_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /ndmg_atlases/label/Human/Talairach_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
    /ndmg_atlases/label/Human/Schaefer200_space-MNI152NLin6_res-1x1x1.nii.gz: Avg
  realignment: ROI_to_func
  connectivity_matrix:
    using:
      - Nilearn
      - ndmg
    measure:
      - Pearson
      - Partial
segmentation:
  run: true
  tissue_segmentation:
    using:
      - FSL-FAST
    FSL-FAST:
      thresholding:
        use: Auto
        Custom:
          CSF_threshold_value: 0.95
          WM_threshold_value: 0.95
          GM_threshold_value: 0.95
      use_priors:
        run: true
        priors_path: $FSLDIR/data/standard/tissuepriors/2mm
        WM_path: $priors_path/avg152T1_white_bin.nii.gz
        GM_path: $priors_path/avg152T1_gray_bin.nii.gz
        CSF_path: $priors_path/avg152T1_csf_bin.nii.gz
    Template_Based:
      template_for_segmentation:
        - T1_Template
      WHITE: $FSLDIR/data/standard/tissuepriors/2mm/avg152T1_white_bin.nii.gz
      GRAY: $FSLDIR/data/standard/tissuepriors/2mm/avg152T1_gray_bin.nii.gz
      CSF: $FSLDIR/data/standard/tissuepriors/2mm/avg152T1_csf_bin.nii.gz
    ANTs_Prior_Based:
      template_brain_list:
        - /cpac_templates/MacaqueYerkes19_T1w_0.5mm_desc-JLC_T1w_brain.nii.gz
        - >-
          /cpac_templates/J_Macaque_11mo_atlas_nACQ_194x252x160space_0.5mm_desc-JLC_T1w_brain.nii.gz
      template_segmentation_list:
        - /cpac_templates/MacaqueYerkes19_T1w_0.5mm_desc-JLC_Segmentation.nii.gz
        - >-
          /cpac_templates/J_Macaque_11mo_atlas_nACQ_194x252x160space_0.5mm_desc-JLC_Segmentation.nii.gz
      CSF_label:
        - 24
      GM_label:
        - 3
        - 42
      WM_label:
        - 2
        - 41
    FreeSurfer:
      erode: 0
      CSF_label:
        - 24
      GM_label:
        - 3
        - 42
      WM_label:
        - 2
        - 41
longitudinal_template_generation:
  run: false
  average_method: median
  dof: 12
  interp: trilinear
  cost: corratio
  thread_pool: 2
  convergence_threshold: -1
network_centrality:
  run: true
  memory_allocation: 1
  template_specification_file: /cpac_templates/Mask_ABIDE_85Percent_GM.nii.gz
  degree_centrality:
    weight_options:
      - Binarized
      - Weighted
    correlation_threshold_option: Sparsity threshold
    correlation_threshold: 0.001
  eigenvector_centrality:
    weight_options:
      - Weighted
    correlation_threshold_option: Sparsity threshold
    correlation_threshold: 0.001
  local_functional_connectivity_density:
    weight_options:
      - Binarized
      - Weighted
    correlation_threshold_option: Correlation threshold
    correlation_threshold: 0.6
PyPEER:
  run: false
  eye_scan_names: []
  data_scan_names: []
  eye_mask_path: '$FSLDIR/data/standard/MNI152_T1_${func_resolution}_eye_mask.nii.gz'
  stimulus_path: None
  minimal_nuisance_correction:
    peer_gsr: true
    peer_scrub: false
    scrub_thresh: 0.2
registration_workflows:
  anatomical_registration:
    run: true
    resolution_for_anat: 2mm
    T1w_brain_template: >-
      /usr/share/fsl/5.0/data/standard/MNI152_T1_${resolution_for_anat}_brain.nii.gz
    T1w_template: '/usr/share/fsl/5.0/data/standard/MNI152_T1_${resolution_for_anat}.nii.gz'
    T1w_brain_template_mask: >-
      /usr/share/fsl/5.0/data/standard/MNI152_T1_${resolution_for_anat}_brain_mask.nii.gz
    reg_with_skull: true
    registration:
      using:
        - ANTS
      ANTs:
        use_lesion_mask: false
        T1_registration:
          - collapse-output-transforms: 0
          - dimensionality: 3
          - initial-moving-transform:
              initializationFeature: 0
          - transforms:
              - Rigid:
                  gradientStep: 0.1
                  metric:
                    type: MI
                    metricWeight: 1
                    numberOfBins: 32
                    samplingStrategy: Regular
                    samplingPercentage: 0.25
                  convergence:
                    iteration: 1000x500x250x100
                    convergenceThreshold: 1.e-8
                    convergenceWindowSize: 10
                  smoothing-sigmas: 3.0x2.0x1.0x0.0
                  shrink-factors: 8x4x2x1
                  use-histogram-matching: true
              - Affine:
                  gradientStep: 0.1
                  metric:
                    type: MI
                    metricWeight: 1
                    numberOfBins: 32
                    samplingStrategy: Regular
                    samplingPercentage: 0.25
                  convergence:
                    iteration: 1000x500x250x100
                    convergenceThreshold: 1.e-8
                    convergenceWindowSize: 10
                  smoothing-sigmas: 3.0x2.0x1.0x0.0
                  shrink-factors: 8x4x2x1
                  use-histogram-matching: true
              - SyN:
                  shrink-factors: 6x4x2x1
                  updateFieldVarianceInVoxelSpace: 3
                  smoothing-sigmas: 3.0x2.0x1.0x0.0
                  totalFieldVarianceInVoxelSpace: 0
                  use-histogram-matching: true
                  winsorize-image-intensities:
                    lowerQuantile: 0.01
                    upperQuantile: 0.99
                  metric:
                    type: CC
                    metricWeight: 1
                    radius: 4
                  convergence:
                    iteration: 100x100x70x20
                    convergenceThreshold: 1.e-9
                    convergenceWindowSize: 15
                  gradientStep: 0.1
        interpolation: LanczosWindowedSinc
      FSL-FNIRT:
        identity_matrix: /usr/share/fsl/5.0/etc/flirtsch/ident.mat
        FNIRT_T1w_brain_template: None
        interpolation: sinc
        T1w_template_res-2: /usr/share/fsl/5.0/data/standard/MNI152_T1_2mm.nii.gz
        ref_mask: >-
          /usr/share/fsl/5.0/data/standard/MNI152_T1_${resolution_for_anat}_brain_mask_dil.nii.gz
        ref_resolution: 2mm
        FNIRT_T1w_template: None
        fnirt_config: T1_2_MNI152_2mm
        ref_mask_res-2: /usr/share/fsl/5.0/data/standard/MNI152_T1_2mm_brain_mask_dil.nii.gz
    overwrite_transform:
      run: false
      using: FSL
  functional_registration:
    coregistration:
      using: FSL
      interpolation: trilinear
      input: brain
      run: true
      reference: brain
      cost: corratio
      arguments: None
      boundary_based_registration:
        run:
          - true
        bbr_schedule: /usr/share/fsl/5.0/etc/flirtsch/bbr.sch
        reference: whole-head
        bbr_wm_map: probability_map
        bbr_wm_mask_args: '-thr 0.5 -bin'
      dof: 6
      func_input_prep:
        reg_with_skull: false
        input:
          - Mean_Functional
        Mean Functional:
          n4_correct_func: false
        Selected Functional Volume:
          func_reg_input_volume: 0
    EPI_registration:
      run: false
      using:
        - ANTS
      EPI_template: 's3://fcp-indi/resources/cpac/resources/epi_hbn.nii.gz'
      EPI_template_mask: None
      ANTs:
        parameters:
          - collapse-output-transforms: 0
          - dimensionality: 3
          - initial-moving-transform:
              initializationFeature: 0
          - transforms:
              - Rigid:
                  gradientStep: 0.1
                  metric:
                    type: MI
                    metricWeight: 1
                    numberOfBins: 32
                    samplingStrategy: Regular
                    samplingPercentage: 0.25
                  convergence:
                    iteration: 1000x500x250x100
                    convergenceThreshold: 1.e-8
                    convergenceWindowSize: 10
                  smoothing-sigmas: 3.0x2.0x1.0x0.0
                  shrink-factors: 8x4x2x1
                  use-histogram-matching: true
              - Affine:
                  gradientStep: 0.1
                  metric:
                    type: MI
                    metricWeight: 1
                    numberOfBins: 32
                    samplingStrategy: Regular
                    samplingPercentage: 0.25
                  convergence:
                    iteration: 1000x500x250x100
                    convergenceThreshold: 1.e-8
                    convergenceWindowSize: 10
                  smoothing-sigmas: 3.0x2.0x1.0x0.0
                  shrink-factors: 8x4x2x1
                  use-histogram-matching: true
              - SyN:
                  shrink-factors: 6x4x2x1
                  updateFieldVarianceInVoxelSpace: 3
                  smoothing-sigmas: 3.0x2.0x1.0x0.0
                  totalFieldVarianceInVoxelSpace: 0
                  use-histogram-matching: true
                  winsorize-image-intensities:
                    lowerQuantile: 0.01
                    upperQuantile: 0.99
                  metric:
                    type: CC
                    metricWeight: 1
                    radius: 4
                  convergence:
                    iteration: 100x100x70x20
                    convergenceThreshold: 1.e-9
                    convergenceWindowSize: 15
                  gradientStep: 0.1
        interpolation: LanczosWindowedSinc
      FSL-FNIRT:
        fnirt_config: T1_2_MNI152_2mm
        interpolation: sinc
        identity_matrix: /usr/share/fsl/5.0/etc/flirtsch/ident.mat
    func_registration_to_template:
      run: true
      run_EPI: false
      output_resolution:
        func_preproc_outputs: 3mm
        func_derivative_outputs: 3mm
      target_template:
        using:
          - T1_template
        T1_template:
          T1w_brain_template_funcreg: >-
            /usr/share/fsl/5.0/data/standard/MNI152_T1_${func_resolution}_brain.nii.gz
          T1w_template_funcreg: '/usr/share/fsl/5.0/data/standard/MNI152_T1_${func_resolution}.nii.gz'
          T1w_brain_template_mask_funcreg: >-
            /usr/share/fsl/5.0/data/standard/MNI152_T1_${func_resolution}_brain_mask.nii.gz
          T1w_template_for_resample: $FSLDIR/data/standard/MNI152_T1_1mm_brain.nii.gz
        EPI_template:
          EPI_template_funcreg: 's3://fcp-indi/resources/cpac/resources/epi_hbn.nii.gz'
          EPI_template_mask_funcreg: None
          EPI_template_for_resample: $FSLDIR/data/standard/MNI152_T1_1mm_brain.nii.gz
      ANTs_pipelines:
        interpolation: LanczosWindowedSinc
      FNIRT_pipelines:
        interpolation: sinc
        identity_matrix: /usr/share/fsl/5.0/etc/flirtsch/ident.mat
      apply_transform:
        using: default
surface_analysis:
  freesurfer:
    run: false
    reconall_args: None
    freesurfer_dir: None
  post_freesurfer:
    smooth_fwhm: 2
    high_res_mesh: 164
    run: false
    gray_ordinates_res: 2
    subcortical_gray_labels: >-
      /opt/dcan-tools/pipeline/global/config/FreeSurferSubcorticalLabelTableLut.txt
    freesurfer_labels: /opt/dcan-tools/pipeline/global/config/FreeSurferAllLut.txt
    surf_atlas_dir: /opt/dcan-tools/pipeline/global/templates/standard_mesh_atlases
    low_res_mesh: 32
    gray_ordinates_dir: /opt/dcan-tools/pipeline/global/templates/91282_Greyordinates
    fmri_res: 2
regional_homogeneity:
  run: true
  cluster_size: 27
post_processing:
  spatial_smoothing:
    output:
      - nonsmoothed
      - smoothed
    smoothing_method:
      - FSL
    fwhm: [6]
  z-scoring:
    output:
      - z-scored
seed_based_correlation_analysis:
  run: false
  sca_roi_paths:
    /cpac_templates/PNAS_Smith09_rsn10.nii.gz: DualReg
    /cpac_templates/CC400.nii.gz: 'Avg, MultReg'
    /cpac_templates/ez_mask_pad.nii.gz: 'Avg, MultReg'
    /cpac_templates/aal_mask_pad.nii.gz: 'Avg, MultReg'
    /cpac_templates/CC200.nii.gz: 'Avg, MultReg'
    /cpac_templates/tt_mask_pad.nii.gz: 'Avg, MultReg'
    /cpac_templates/ho_mask_pad.nii.gz: 'Avg, MultReg'
    /cpac_templates/rois_3mm.nii.gz: 'Avg, MultReg'
  norm_timeseries_for_DR: true
nuisance_corrections:
  1-ICA-AROMA:
    run:
      - true
    denoising_type: nonaggr
  2-nuisance_regression:
    run:
      - true
    create_regressors: true
    Regressors:
      - Name: default
        Motion:
          include_delayed: true
          include_squared: true
          include_delayed_squared: true
        aCompCor:
          summary:
            method: DetrendPC
            components: 5
          tissues:
            - WhiteMatter
            - CerebrospinalFluid
          extraction_resolution: 2
        CerebrospinalFluid:
          summary: Mean
          extraction_resolution: 2
          erode_mask: true
        GlobalSignal:
          summary: Mean
        PolyOrt:
          degree: 2
        Bandpass:
          bottom_frequency: 0.01
          top_frequency: 0.1
          method: default
      - Name: defaultNoGSR
        Motion:
          include_delayed: true
          include_squared: true
          include_delayed_squared: true
        aCompCor:
          summary:
            method: DetrendPC
            components: 5
          tissues:
            - WhiteMatter
            - CerebrospinalFluid
          extraction_resolution: 2
        CerebrospinalFluid:
          summary: Mean
          extraction_resolution: 2
          erode_mask: true
        PolyOrt:
          degree: 2
        Bandpass:
          bottom_frequency: 0.01
          top_frequency: 0.1
          method: default
      - Name: defaultANDcensor
        Motion:
          include_delayed: true
          include_squared: true
          include_delayed_squared: true
        aCompCor:
          summary:
            method: DetrendPC
            components: 5
          tissues:
            - WhiteMatter
            - CerebrospinalFluid
          extraction_resolution: 2
        CerebrospinalFluid:
          summary: Mean
          extraction_resolution: 2
          erode_mask: true
        GlobalSignal:
          summary: Mean
        PolyOrt:
          degree: 2
        Bandpass:
          bottom_frequency: 0.01
          top_frequency: 0.1
          method: default
        Censor:
          method: Kill
          thresholds:
            - type: FD_P
              value: 0.25
    lateral_ventricles_mask: >-
      $FSLDIR/data/atlases/HarvardOxford/HarvardOxford-lateral-ventricles-thr25-2mm.nii.gz
    bandpass_filtering_order: After
    regressor_masks:
      erode_anatomical_brain_mask:
        run: false
        brain_mask_erosion_prop: null
        brain_mask_erosion_mm: 30
        brain_erosion_mm: null
      erode_csf:
        run: false
        csf_erosion_prop: null
        csf_mask_erosion_mm: 30
        csf_erosion_mm: null
      erode_wm:
        run: false
        wm_erosion_prop: 0.6
        wm_mask_erosion_mm: null
        wm_erosion_mm: null
      erode_gm:
        run: false
        gm_erosion_prop: 0.6
        gm_mask_erosion_mm: null
        gm_erosion_mm: null
amplitude_low_frequency_fluctuation:
  run: true
  highpass_cutoff:
    - 0.01
  lowpass_cutoff:
    - 0.1
voxel_mirrored_homotopic_connectivity:
  run: true
  symmetric_registration:
    T1w_brain_template_symmetric: >-
      $FSLDIR/data/standard/MNI152_T1_${resolution_for_anat}_brain_symmetric.nii.gz
    T1w_brain_template_symmetric_for_resample: $FSLDIR/data/standard/MNI152_T1_1mm_brain_symmetric.nii.gz
    T1w_template_symmetric: '$FSLDIR/data/standard/MNI152_T1_${resolution_for_anat}_symmetric.nii.gz'
    T1w_template_symmetric_for_resample: $FSLDIR/data/standard/MNI152_T1_1mm_symmetric.nii.gz
    dilated_symmetric_brain_mask: >-
      $FSLDIR/data/standard/MNI152_T1_${resolution_for_anat}_brain_mask_symmetric_dil.nii.gz
    dilated_symmetric_brain_mask_for_resample: $FSLDIR/data/standard/MNI152_T1_1mm_brain_mask_symmetric_dil.nii.gz
functional_preproc:
  distortion_correction:
    run:
      - false
    using:
      - PhaseDiff
      - Blip
    PhaseDiff:
      fmap_skullstrip_option: BET
      fmap_skullstrip_BET_frac: 0.5
      fmap_skullstrip_AFNI_threshold: 0.6
    Blip-FSL-TOPUP:
      miter: 5
      numprec: double
      lambda: 1
      scale: 0
      minmet: 0
      regmod: bending_energy
      estmov: 1
      subsamp: 1
      interp: spline
      fwhm: 8
      ssqlambda: 1
      warpres: 10
      regrid: 1
      splineorder: 3
  despiking:
    run:
      - true
      - false
  motion_estimates_and_correction:
    run: true
    motion_estimates:
      calculate_motion_first: false
      calculate_motion_after: true
    motion_correction:
      using:
        - 3dvolreg
      AFNI-3dvolreg:
        functional_volreg_twopass: true
      motion_correction_reference:
        - mean
      motion_correction_reference_volume: 0
    motion_estimate_filter:
      run:
        - false
      filter_type: notch
      filter_order: 4
      breathing_rate_min: null
      breathing_rate_max: null
      center_frequency: null
      filter_bandwidth: null
      lowpass_cutoff: null
  slice_timing_correction:
    run:
      - true
    tpattern: None
    tzero: None
  scaling:
    run: false
    scaling_factor: 10
  truncation:
    start_tr: 0
    stop_tr: None
  run: true
  normalize_func:
    run: true
  generate_func_mean:
    run: true
  func_masking:
    using:
      - AFNI
    FSL-BET:
      functional_mean_thr:
        run: false
        threshold_value: 98
      outline: false
      functional_mean_bias_correction: false
      surfaces: false
      functional_mean_boolean: false
      threshold: false
      frac: 0.3
      padding: false
      robust: false
      radius: 0
      reduce_bias: false
      mesh_boolean: false
      vertical_gradient: 0
      remove_eyes: false
      skull: false
    FSL_AFNI:
      bold_ref: null
      brain_mask: >-
        /usr/share/fsl/5.0/data/standard/MNI152_T1_${resolution_for_anat}_brain_mask.nii.gz
      brain_probseg: >-
        /usr/share/fsl/5.0/data/standard/MNI152_T1_${resolution_for_anat}_brain_mask.nii.gz
    Anatomical_Refined:
      anatomical_mask_dilation: false
    apply_func_mask_in_native_space: true
anatomical_preproc:
  run: true
  run_t2: false
  non_local_means_filtering:
    run:
      - false
    noise_model: Gaussian
  n4_bias_field_correction:
    run:
      - true
    shrink_factor: 2
  t1t2_bias_field_correction:
    run: false
    BiasFieldSmoothingSigma: 5
  acpc_alignment:
    FOV_crop: robustfov
    brain_size: 150
    run: false
    run_before_preproc: true
    align_brain_mask: false
    T2w_brain_ACPC_template: None
    T1w_brain_ACPC_template: /usr/share/fsl/5.0/data/standard/MNI152_T1_1mm_brain.nii.gz
    acpc_target: whole-head
    T2w_ACPC_template: None
    T1w_ACPC_template: /usr/share/fsl/5.0/data/standard/MNI152_T1_1mm.nii.gz
  brain_extraction:
    run: true
    using:
      - 3dSkullStrip
    AFNI-3dSkullStrip:
      mask_vol: false
      shrink_factor: 0.6
      use_skull: false
      smooth_final: 20
      n_iterations: 250
      fac: 1
      var_shrink_fac: true
      avoid_eyes: true
      NN_smooth: 72
      blur_fwhm: 0
      exp_frac: 0.1
      monkey: false
      pushout: true
      use_edge: true
      max_inter_iter: 4
      perc_int: 0
      touchup: true
      push_to_edge: false
      fill_hole: 10
      shrink_factor_bot_lim: 0.4
      avoid_vent: true
    FSL-BET:
      outline: false
      surfaces: false
      threshold: false
      frac: 0.5
      padding: false
      robust: false
      radius: 0
      reduce_bias: false
      mask_boolean: true
      mesh_boolean: false
      vertical_gradient: 0
      remove_eyes: false
      skull: false
    UNet:
      unet_model: 's3://fcp-indi/resources/cpac/resources/Site-All-T-epoch_36.model'
    niworkflows-ants:
      template_path: /ants_template/oasis/T_template0.nii.gz
      mask_path: /ants_template/oasis/T_template0_BrainCerebellumProbabilityMask.nii.gz
      regmask_path: /ants_template/oasis/T_template0_BrainCerebellumRegistrationMask.nii.gz
    FreeSurfer-BET:
      T1w_brain_template_mask_ccs: /ccs_template/MNI152_T1_1mm_first_brain_mask.nii.gz

Run command

run /data/proj/app/pipeline/rs/dat/SITES/PRIMARYSITES/MIND /data/proj/app/pipeline/rs/outputs participant --pipeline_file /data/proj/app/pipeline/rs/Pipeline_M.yml --data_config_file /data/proj/app/pipeline/rs/subject_configs/data_config_T123.yml

Expected behavior

process the nuisance regression and continue

Acceptance criteria

no crash

Screenshots

No response

C-PAC version

v1.8.4

Container platform

No response

Docker and/or Singularity version(s)

No response

Additional context

There is always a possibility that our subject id names are problematic, which follow the format 123456-100 123456-100_1

nimzodisaster commented 1 year ago

It occurred to me this morning that the 1D file: denoised_func_data_nonaggr_0_trans_tcat_flirt_roistat.1D contains lines without any data (probably) for example: #/data/proj/app/pipeline/rs/outputs/working/cpac_103430-100_3_ses-1/nuisance_regressors_default_271/_scan_func-1/Functional_2mm_flirt/denoised_func_data_nonaggr_0_trans_tcat_flirt.nii.gz 299[?]

so the error probably occurred earlier, perhaps when denoised_func_data_nonaggr_0_trans_tcat_flirt_roistat.1D was created.

nimzodisaster commented 1 year ago

This may be related to bad image registration to template space, resulting in very little overlap with priors.

nimzodisaster commented 1 year ago

closed: user error.

nimzodisaster commented 1 year ago

After fixing my own error that prevented good registration, I thought perhaps this bug went away. However, the bug still appears, though not as frequently.

My best guess right now is this: Functional dropout or bad registration in a subject causes some ROI masks to have no timeseries data crashing the node.

e-kenneally commented 1 year ago

Hi, thanks for reaching out! We'd be glad to help. As a first quick check, could you take a look at your CSF and WM masks from segmentation? These are both being used in your pipeline config as nuisance regressors and it is possible that the masks are too stringent, which could lead to the timeseries data not making it through.