search

shnizzedy / C-PAC

Configurable Pipeline for the Analysis of Connectomes
GNU Lesser General Public License v3.0
1 stars 0 forks source link

🐛 "OSError: Duplicate node name" #10

Closed shnizzedy closed 2 years ago

shnizzedy commented 3 years ago

Describe the bug

Trying to test https://github.com/FCP-INDI/C-PAC/pull/1524 (cpac-singularity-image.simg) with prelim_forks_ses_test.yml, I get

Connecting nuisance_regression...

Traceback (most recent call last):
  File "/code/run.py", line 672, in <module>
    test_config = 1 if args.analysis_level == "test_config" else 0
  File "/code/CPAC/pipeline/cpac_runner.py", line 549, in run
    p_name, plugin, plugin_args, test_config)
  File "/code/CPAC/pipeline/cpac_pipeline.py", line 395, in run_workflow
    subject_id, sub_dict, c, p_name, num_ants_cores
  File "/code/CPAC/pipeline/cpac_pipeline.py", line 1261, in build_workflow
    wf = connect_pipeline(wf, cfg, rpool, pipeline_blocks)
  File "/code/CPAC/pipeline/cpac_pipeline.py", line 976, in connect_pipeline
    wf = nb.connect_block(wf, cfg, rpool)
  File "/code/CPAC/pipeline/engine.py", line 1163, in connect_block
    pipe_x, opt)
  File "/code/CPAC/nuisance/nuisance.py", line 2141, in nuisance_regression_complete
    wf.connect(node, out, regressors, 'inputspec.functional_file_path')
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/workflows.py", line 158, in connect
    self._check_nodes(newnodes)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/workflows.py", line 766, in _check_nodes
    raise IOError('Duplicate node name "%s" found.' % node.name)
OSError: Duplicate node name "nuisance_regressors_2-aCompCor-no-GSR-no-spike-regression_128" found.

C-PAC seems to be duplicating at least that one regressor somehow?

To Reproduce Steps to reproduce the behavior:

singularity run \
  -B /$output:/outputs \
  -B ${BIDS_DIR}:/bids_dataset \
  -B ${CPAC_DIR}:/code \
  -B ${DATA_CONFIG}:/data_config \
  -B ${CPAC_DIR}/CPAC/resources/configs:${CPAC_DIR}/CPAC/resources/configs \
  -B /tmp:/scratch -B /media/ebs/Columbia_Nim/configs:/configs \
  cpac-singularity-image.simg /bids_dataset /outputs participant \
  --data_config_file ${DATA_CONFIG_FILE} \
  --pipeline_file /configs/prelim_forks_ses_test.yml \
  --mem_gb 100 \
  --participant_label ${PARTICIPANT}

Versions

Additional context

The regressor section of the pipeline file:

nuisance_corrections: 
  2-nuisance_regression: 
    run: [On]
    Regressors: 
      #1: aCompCor, GSR, no spike regression
      - Name: "1-aCompCor-GSR-no-spike-regression"
        Motion:
          include_delayed: true
          include_squared: true
          include_delayed_squared: true
        aCompCor:
          summary:
            method: DetrendPC
            components: 5
          tissues:
            - WhiteMatter
            - CerebrospinalFluid
          extraction_resolution: 2
        GlobalSignal:
          summary: Mean
        PolyOrt:
          degree: 2
        Bandpass:
          bottom_frequency: 0.01
          top_frequency: 0.1
      #2 aCompCor, no GSR, no spike regression
      - Name: "2-aCompCor-no-GSR-no-spike-regression"
        Motion:
          include_delayed: true
          include_squared: true
          include_delayed_squared: true
        aCompCor:
          summary:
            method: DetrendPC
            components: 5
          tissues:
            - WhiteMatter
            - CerebrospinalFluid
          extraction_resolution: 2
        PolyOrt:
          degree: 2
        Bandpass:
          bottom_frequency: 0.01
          top_frequency: 0.1
      #3 No aCompCor, GSR, no spike regression
      - Name: "3-No-aCompCor-GSR-no-spike-regression"
        Motion:
          include_delayed: true
          include_squared: true
          include_delayed_squared: true
        GlobalSignal:
          summary: Mean
        PolyOrt:
          degree: 2
        Bandpass:
          bottom_frequency: 0.01
          top_frequency: 0.1
      #4 No aCompCor, no GSR, no spike regression
      - Name: "4-No-aCompCor-no-GSR-no-spike-regression"
        Motion:
          include_delayed: true
          include_squared: true
          include_delayed_squared: true
        PolyOrt:
          degree: 2
        Bandpass:
          bottom_frequency: 0.01
          top_frequency: 0.1
      #5: aCompCor, GSR, spike regression
      - Name: "5-aCompCor-GSR-spike-regression"
        Motion:
          include_delayed: true
          include_squared: true
          include_delayed_squared: true
        aCompCor:
          summary:
            method: DetrendPC
            components: 5
          tissues:
            - WhiteMatter
            - CerebrospinalFluid
          extraction_resolution: 2
        GlobalSignal:
          summary: Mean
        PolyOrt:
          degree: 2
        Bandpass:
          bottom_frequency: 0.01
          top_frequency: 0.1
        Censor:
            method: SpikeRegression
            thresholds:
              - type: FD_J
                value: 0.2
            number_of_previous_trs_to_censor: 0
            number_of_subsequent_trs_to_censor: 0
      #6 aCompCor, no GSR, spike regression
      - Name: "6-aCompCor-no-GSR-spike-regression"
        Motion:
          include_delayed: true
          include_squared: true
          include_delayed_squared: true
        aCompCor:
          summary:
            method: DetrendPC
            components: 5
          tissues:
            - WhiteMatter
            - CerebrospinalFluid
          extraction_resolution: 2
        PolyOrt:
          degree: 2
        Bandpass:
          bottom_frequency: 0.01
          top_frequency: 0.1
        Censor:
            method: SpikeRegression
            thresholds:
              - type: FD_J
                value: 0.2
            number_of_previous_trs_to_censor: 0
            number_of_subsequent_trs_to_censor: 0
      #7 No aCompCor, GSR, spike regression
      - Name: "7-No-aCompCor-GSR-spike-regression"
        Motion:
          include_delayed: true
          include_squared: true
          include_delayed_squared: true
        GlobalSignal:
          summary: Mean
        PolyOrt:
          degree: 2
        Bandpass:
          bottom_frequency: 0.01
          top_frequency: 0.1
        Censor:
            method: SpikeRegression
            thresholds:
              - type: FD_J
                value: 0.2
            number_of_previous_trs_to_censor: 0
            number_of_subsequent_trs_to_censor: 0
      #8 No aCompCor, no GSR, spike regression
      - Name: "8-No-aCompCor-no-GSR-spike-regression"
        Motion:
          include_delayed: true
          include_squared: true
          include_delayed_squared: true
        PolyOrt:
          degree: 2
        Bandpass:
          bottom_frequency: 0.01
          top_frequency: 0.1
        Censor:
            method: SpikeRegression
            thresholds:
              - type: FD_J
                value: 0.2
            number_of_previous_trs_to_censor: 0
            number_of_subsequent_trs_to_censor: 0
shnizzedy commented 3 years ago

If I add print(regressors) here, we can see the first regressor gets a new pipe_num, but the second is repeated, causing the crash.

nuisance_regressors_1-aCompCor-GSR-no-spike-regression_177
nuisance_regressors_2-aCompCor-no-GSR-no-spike-regression_128
nuisance_regressors_3-No-aCompCor-GSR-no-spike-regression_128
nuisance_regressors_4-No-aCompCor-no-GSR-no-spike-regression_128
nuisance_regressors_5-aCompCor-GSR-spike-regression_128
nuisance_regressors_6-aCompCor-no-GSR-spike-regression_128
nuisance_regressors_7-No-aCompCor-GSR-spike-regression_128
nuisance_regressors_8-No-aCompCor-no-GSR-spike-regression_128
nuisance_regressors_1-aCompCor-GSR-no-spike-regression_190
nuisance_regressors_2-aCompCor-no-GSR-no-spike-regression_128

Printing opt here shows the same pattern (custom regressor 1 is different, 2 is the same):

{'Bandpass': {'bottom_frequency': 0.01, 'top_frequency': 0.1}, 'GlobalSignal': {'summary': 'Mean'}, 'Motion': {'include_delayed': True, 'include_delayed_squared': True, 'include_squared': True}, 'Name': '1-aCompCor-GSR-no-spike-regression', 'PolyOrt': {'degree': 2}, 'aCompCor': {'extraction_resolution': 2, 'summary': {'components': 5, 'method': 'DetrendPC'}, 'tissues': ['WhiteMatter', 'CerebrospinalFluid']}}
{'Bandpass': {'bottom_frequency': 0.01, 'top_frequency': 0.1}, 'Motion': {'include_delayed': True, 'include_delayed_squared': True, 'include_squared': True}, 'Name': '2-aCompCor-no-GSR-no-spike-regression', 'PolyOrt': {'degree': 2}, 'aCompCor': {'extraction_resolution': 2, 'summary': {'components': 5, 'method': 'DetrendPC'}, 'tissues': ['WhiteMatter', 'CerebrospinalFluid']}}
{'Bandpass': {'bottom_frequency': 0.01, 'top_frequency': 0.1}, 'GlobalSignal': {'summary': 'Mean'}, 'Motion': {'include_delayed': True, 'include_delayed_squared': True, 'include_squared': True}, 'Name': '3-No-aCompCor-GSR-no-spike-regression', 'PolyOrt': {'degree': 2}}
{'Bandpass': {'bottom_frequency': 0.01, 'top_frequency': 0.1}, 'Motion': {'include_delayed': True, 'include_delayed_squared': True, 'include_squared': True}, 'Name': '4-No-aCompCor-no-GSR-no-spike-regression', 'PolyOrt': {'degree': 2}}
{'Bandpass': {'bottom_frequency': 0.01, 'top_frequency': 0.1}, 'Censor': {'method': 'SpikeRegression', 'number_of_previous_trs_to_censor': 0, 'number_of_subsequent_trs_to_censor': 0, 'thresholds': [{'type': 'FD_J', 'value': 0.2}]}, 'GlobalSignal': {'summary': 'Mean'}, 'Motion': {'include_delayed': True, 'include_delayed_squared': True, 'include_squared': True}, 'Name': '5-aCompCor-GSR-spike-regression', 'PolyOrt': {'degree': 2}, 'aCompCor': {'extraction_resolution': 2, 'summary': {'components': 5, 'method': 'DetrendPC'}, 'tissues': ['WhiteMatter', 'CerebrospinalFluid']}}
{'Bandpass': {'bottom_frequency': 0.01, 'top_frequency': 0.1}, 'Censor': {'method': 'SpikeRegression', 'number_of_previous_trs_to_censor': 0, 'number_of_subsequent_trs_to_censor': 0, 'thresholds': [{'type': 'FD_J', 'value': 0.2}]}, 'Motion': {'include_delayed': True, 'include_delayed_squared': True, 'include_squared': True}, 'Name': '6-aCompCor-no-GSR-spike-regression', 'PolyOrt': {'degree': 2}, 'aCompCor': {'extraction_resolution': 2, 'summary': {'components': 5, 'method': 'DetrendPC'}, 'tissues': ['WhiteMatter', 'CerebrospinalFluid']}}
{'Bandpass': {'bottom_frequency': 0.01, 'top_frequency': 0.1}, 'Censor': {'method': 'SpikeRegression', 'number_of_previous_trs_to_censor': 0, 'number_of_subsequent_trs_to_censor': 0, 'thresholds': [{'type': 'FD_J', 'value': 0.2}]}, 'GlobalSignal': {'summary': 'Mean'}, 'Motion': {'include_delayed': True, 'include_delayed_squared': True, 'include_squared': True}, 'Name': '7-No-aCompCor-GSR-spike-regression', 'PolyOrt': {'degree': 2}}
{'Bandpass': {'bottom_frequency': 0.01, 'top_frequency': 0.1}, 'Censor': {'method': 'SpikeRegression', 'number_of_previous_trs_to_censor': 0, 'number_of_subsequent_trs_to_censor': 0, 'thresholds': [{'type': 'FD_J', 'value': 0.2}]}, 'Motion': {'include_delayed': True, 'include_delayed_squared': True, 'include_squared': True}, 'Name': '8-No-aCompCor-no-GSR-spike-regression', 'PolyOrt': {'degree': 2}}
{'Bandpass': {'bottom_frequency': 0.01, 'top_frequency': 0.1}, 'GlobalSignal': {'summary': {'filter': 'Mean', 'method': 'Mean'}}, 'Motion': {'include_delayed': True, 'include_delayed_squared': True, 'include_squared': True}, 'Name': '1-aCompCor-GSR-no-spike-regression', 'PolyOrt': {'degree': 2}, 'aCompCor': {'extraction_resolution': 2, 'summary': {'components': 5, 'method': 'DetrendPC'}, 'tissues': ['WhiteMatter', 'CerebrospinalFluid']}}
{'Bandpass': {'bottom_frequency': 0.01, 'top_frequency': 0.1}, 'Motion': {'include_delayed': True, 'include_delayed_squared': True, 'include_squared': True}, 'Name': '2-aCompCor-no-GSR-no-spike-regression', 'PolyOrt': {'degree': 2}, 'aCompCor': {'extraction_resolution': 2, 'summary': {'components': 5, 'method': 'DetrendPC'}, 'tissues': ['WhiteMatter', 'CerebrospinalFluid']}}
shnizzedy commented 3 years ago

With https://github.com/shnizzedy/C-PAC/blob/7413998587fcaf745ddc529161b0e7d27e8330df/CPAC/pipeline/engine.py#L1167-L1169 we see several warnings

Duplicate node name "nuisance_regressors_2-aCompCor-no-GSR-no-spike-regression_128" found.
Duplicate node name "nuisance_regressors_3-No-aCompCor-GSR-no-spike-regression_128" found.
Duplicate node name "nuisance_regressors_4-No-aCompCor-no-GSR-no-spike-regression_128" found.
Duplicate node name "nuisance_regressors_5-aCompCor-GSR-spike-regression_128" found.
Duplicate node name "nuisance_regressors_6-aCompCor-no-GSR-spike-regression_128" found.
Duplicate node name "nuisance_regressors_7-No-aCompCor-GSR-spike-regression_128" found.
Duplicate node name "nuisance_regressors_8-No-aCompCor-no-GSR-spike-regression_128" found.
Duplicate node name "nuisance_regressors_2-aCompCor-no-GSR-no-spike-regression_129" found.
Duplicate node name "nuisance_regressors_3-No-aCompCor-GSR-no-spike-regression_129" found.
Duplicate node name "nuisance_regressors_4-No-aCompCor-no-GSR-no-spike-regression_129" found.
Duplicate node name "nuisance_regressors_5-aCompCor-GSR-spike-regression_129" found.
Duplicate node name "nuisance_regressors_6-aCompCor-no-GSR-spike-regression_129" found.
Duplicate node name "nuisance_regressors_7-No-aCompCor-GSR-spike-regression_129" found.
Duplicate node name "nuisance_regressors_8-No-aCompCor-no-GSR-spike-regression_129" found.

but the pipeline runs