FCP-INDI / C-PAC

Configurable Pipeline for the Analysis of Connectomes
https://fcp-indi.github.io/
GNU Lesser General Public License v3.0
64 stars 41 forks source link

When CPAC BIDS app running in single participant mode, using the instructions in README, pipeline fails on 2 computers, 2 separate users #816

Closed rattmowe closed 6 years ago

rattmowe commented 6 years ago

Docker command used (directly implementing the participant level execution command from README):

docker run -i --rm -v /tmp:/scratch -v /home/matt/Documents/Projects/BIDS/ds005:/bids_dataset -v /home/matt/outputs:/outputs bids/cpac /bids_dataset /outputs participant --participant_label 01

Output:

Namespace(analysis_level='participant', anat_select_string=None, aws_config_input_creds=None, aws_data_input_creds=None, aws_output_creds=None, bids_dir='/bids_dataset', bids_validator_config=None, data_config_file=None, disable_file_logging=False, mem_gb=None, mem_mb=None, n_cpus='1', output_dir='/outputs', participant_label=['01'], participant_ndx=None, pipeline_file='/cpac_resources/default_pipeline.yaml', save_working_dir=False, skip_bids_validator=False)

Running BIDS validator 1: Invalid JSON file. The file is not formatted according the schema. (code: 55 - JSON_SCHEMA_VALIDATION_ERROR) ./dataset_description.json Evidence: .ReferencesAndLinks should be array

1: You should define 'SliceTiming' for this file. If you don't provide this information slice time correction will not be possible. (code: 13 - SLICE_TIMING_NOT_DEFINED)
    ./sub-01/func/sub-01_task-mixedgamblestask_run-01_bold.nii.gz
    ./sub-01/func/sub-01_task-mixedgamblestask_run-02_bold.nii.gz
    ./sub-01/func/sub-01_task-mixedgamblestask_run-03_bold.nii.gz
    ./sub-02/func/sub-02_task-mixedgamblestask_run-01_bold.nii.gz
    ./sub-02/func/sub-02_task-mixedgamblestask_run-02_bold.nii.gz
    ./sub-02/func/sub-02_task-mixedgamblestask_run-03_bold.nii.gz
    ./sub-03/func/sub-03_task-mixedgamblestask_run-01_bold.nii.gz
    ./sub-03/func/sub-03_task-mixedgamblestask_run-02_bold.nii.gz
    ./sub-03/func/sub-03_task-mixedgamblestask_run-03_bold.nii.gz
    ./sub-04/func/sub-04_task-mixedgamblestask_run-01_bold.nii.gz
    ... and 38 more files having this issue (Use --verbose to see them all).

    Summary:                 Available Tasks:          Available Modalities: 
    133 Files, 1.77GB        mixed-gambles task        T1w                   
    16 - Subjects                                      inplaneT2             
    1 - Session                                        bold                  

Running C-PAC on ['01']

Number of participants to run in parallel: 1 Input directory: /bids_dataset Output directory: /outputs/output Working directory: /scratch/working Crash directory: /outputs/crash Log directory: /outputs/log Remove working directory: True Available memory: 6.0 (GB) Available threads: 1 Number of threads for ANTs: 1 args.aws_data_input_creds None Checking participants.tsv file for site information: /bids_dataset/participants.tsv No site information found in the participants.tsv file.

[!] WARNING: Duplicate site-participant-session entry found in your input data directory!

Duplicate sets:

{'subject_id': '01', 'anat': '/bids_dataset/sub-01/anat/sub-01_T1w.nii.gz', 'site': 'site-1', 'unique_id': 'ses-1'}

{'subject_id': '01', 'anat': '/bids_dataset/sub-01/anat/sub-01_T1w.nii.gz', 'site': 'site-1', 'unique_id': 'ses-1'}

Only adding the first one to the data configuration file.

Starting participant level processing strategy_entries: [['_compcor_ncomponents_5_selector_pc10.linear1.wm0.global1.motion1.quadratic1.gm0.compcor1.csf1.ncomponents_5_selector_pc10.linear1.wm0.global0.motion1.quadratic1.gm0.compcor1.csf1', '_target_angle_deg_90', '_threshold_0.5']]

/usr/local/miniconda/lib/python2.7/site-packages/CPAC/pipeline/cpac_pipeline.py:201: UserWarning: Boolean Series key will be reindexed to match DataFrame index. outputs_native_nonsmooth = list(keys[keys['Optional outputs: Native space'] == 'yes'][keys['Optional outputs: Non-smoothed'] == 'yes'][keys['Multiple outputs'] != 'yes']['Resource']) /usr/local/miniconda/lib/python2.7/site-packages/CPAC/pipeline/cpac_pipeline.py:202: UserWarning: Boolean Series key will be reindexed to match DataFrame index. outputs_native_nonsmooth_mult = list(keys[keys['Optional outputs: Native space'] == 'yes'][keys['Optional outputs: Non-smoothed'] == 'yes'][keys['Multiple outputs'] == 'yes']['Resource']) /usr/local/miniconda/lib/python2.7/site-packages/CPAC/pipeline/cpac_pipeline.py:203: UserWarning: Boolean Series key will be reindexed to match DataFrame index. outputs_template_nonsmooth = list(keys[keys['Space'] == 'template'][keys['Optional outputs: Non-smoothed'] == 'yes'][keys['Multiple outputs'] != 'yes']['Resource']) /usr/local/miniconda/lib/python2.7/site-packages/CPAC/pipeline/cpac_pipeline.py:204: UserWarning: Boolean Series key will be reindexed to match DataFrame index. outputs_template_nonsmooth_mult = list(keys[keys['Space'] == 'template'][keys['Optional outputs: Non-smoothed'] == 'yes'][keys['Multiple outputs'] == 'yes']['Resource']) /usr/local/miniconda/lib/python2.7/site-packages/CPAC/pipeline/cpac_pipeline.py:207: UserWarning: Boolean Series key will be reindexed to match DataFrame index. outputs_native_smooth = list(keys[keys['Space'] != 'template'][keys['Derivative'] == 'yes'][keys['Optional outputs: Non-smoothed'] != 'yes']['Resource']) /usr/local/miniconda/lib/python2.7/site-packages/CPAC/pipeline/cpac_pipeline.py:210: UserWarning: Boolean Series key will be reindexed to match DataFrame index. outputs_template_smooth = list(keys[keys['Space'] == 'template'][keys['Derivative'] == 'yes'][keys['Optional outputs: Non-smoothed'] != 'yes']['Resource']) /usr/local/miniconda/lib/python2.7/site-packages/CPAC/pipeline/cpac_pipeline.py:215: UserWarning: Boolean Series key will be reindexed to match DataFrame index. outputs_template_raw = list(keys[keys['Space'] == 'template'][keys['Multiple outputs'] != 'yes'][keys['Optional outputs: Raw scores'] == 'yes']['Resource']) /usr/local/miniconda/lib/python2.7/site-packages/CPAC/pipeline/cpac_pipeline.py:216: UserWarning: Boolean Series key will be reindexed to match DataFrame index. outputs_template_raw_mult = list(keys[keys['Space'] == 'template'][keys['Multiple outputs'] == 'yes'][keys['Optional outputs: Raw scores'] == 'yes']['Resource']) /usr/local/miniconda/lib/python2.7/site-packages/CPAC/pipeline/cpac_pipeline.py:220: UserWarning: Boolean Series key will be reindexed to match DataFrame index. outputs_average = list(keys[keys['Calculate averages'] == 'yes'][keys['Multiple outputs'] != 'yes']['Resource']) /usr/local/miniconda/lib/python2.7/site-packages/CPAC/pipeline/cpac_pipeline.py:221: UserWarning: Boolean Series key will be reindexed to match DataFrame index. outputs_average_mult = list(keys[keys['Calculate averages'] == 'yes'][keys['Multiple outputs'] == 'yes']['Resource']) 180731-15:34:39,644 workflow INFO: VERSION: CPAC 1.1.0

Setting maximum number of cores per participant to 1 Setting number of participants at once to 1 Setting OMP_NUM_THREADS to 1 Setting MKL_NUM_THREADS to 1 Setting ANTS/ITK thread usage to 1

Maximum potential number of cores that might be used during this run: 1

++ 3dcalc: AFNI version=AFNI_18.1.18 (May 25 2018) [64-bit] ++ Authored by: A cast of thousands 180731-15:34:39,731 interface WARNING: afni_vcheck executable not found. 180731-15:34:40,70 workflow INFO: connected input to slc 180731-15:34:40,70 workflow INFO: connected TR 180731-15:34:40,70 workflow INFO: slice timing pattern ['Use NIFTI Header'] 180731-15:34:40,70 workflow INFO: finished connecting slice timing pattern 180731-15:34:40,93 workflow INFO: infile rest connected 180731-15:34:40,110 workflow INFO: Warning key movement_parameters already exists in resource pool, replacing with (resting_preproc_01_ses-1.fristons_parameter_model_0, 'outputspec.movement_file') 180731-15:34:40,161 workflow INFO: Warning key mean_functional_in_anat already exists in resource pool, replacing with (resting_preproc_01_ses-1.func_to_anat_bbreg_0, 'outputspec.anat_func') 180731-15:34:40,161 workflow INFO: Warning key functional_to_anat_linear_xfm already exists in resource pool, replacing with (resting_preproc_01_ses-1.func_to_anat_bbreg_0, 'outputspec.func_to_anat_linear_xfm') 180731-15:34:41,444 workflow INFO: Using AFNI centrality function 180731-15:34:41,449 workflow INFO: Using AFNI LFCD function 180731-15:34:42,440 workflow INFO:

Pipeline building completed.

180731-15:34:42,641 workflow INFO: no node for output: 180731-15:34:42,641 workflow INFO: sca_roi_files_to_standard [!] Connection of QA montages workflow for sca_roi_files_to_standard has failed.

Error: 'sca_roi_files_to_standard' 180731-15:34:42,641 workflow INFO: no node for output: 180731-15:34:42,641 workflow INFO: sca_roi_files_to_standard_smooth [!] Connection of QA montages workflow for sca_roi_files_to_standard_smooth has failed.

Error: 'sca_roi_files_to_standard_smooth' 180731-15:34:42,641 workflow INFO: no node for output: 180731-15:34:42,642 workflow INFO: sca_roi_files_to_standard_smooth_fisher_zstd [!] Connection of QA montages workflow for sca_roi_files_to_standard_smooth_fisher_zstd has failed.

Error: 'sca_roi_files_to_standard_smooth_fisher_zstd' 180731-15:34:42,705 workflow INFO:

ERROR: QC - unable to get resources for SNR plot

Error name: cpac_pipeline_0051

Process Process-1: Traceback (most recent call last): File "/usr/local/miniconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/local/miniconda/lib/python2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, **self._kwargs) File "/usr/local/miniconda/lib/python2.7/site-packages/CPAC/pipeline/cpac_pipeline.py", line 4342, in prep_workflow workflow.connect(preproc, out_file, stddev, 'func') File "/usr/local/miniconda/lib/python2.7/site-packages/nipype/pipeline/engine/workflows.py", line 233, in connect infostr)) Exception: Some connections were not found Module func_preproc_automask_0 has no output called outputspec.output_image

anibalsolon commented 6 years ago

@rattmowe it is a known issue for CPAC 1.1.0, that is going to be fixed in the next CPAC release. Meanwhile, in order to run this version, you may disable the quality control interface.

In order to do this, you need to download the default pipeline from: https://raw.githubusercontent.com/BIDS-Apps/CPAC/master/default_pipeline.yaml

Change this file, setting the flag generateQualityControlImages to [0]

Move this pipeline config to a folder, such as /home/matt/Documents/Projects/BIDS/cpac

Run the docker image, binding this new folder to the container and adding the pipeline argument:

docker run -i --rm -v /tmp:/scratch -v /home/matt/Documents/Projects/BIDS/cpac:/cpac_config -v /home/matt/Documents/Projects/BIDS/ds005:/bids_dataset -v /home/matt/outputs:/outputs bids/cpac /bids_dataset /outputs participant --participant_label 01 --pipeline_file /cpac_config/default_pipeline.yaml

(it assumes that your pipeline config is default_pipeline.yaml)

Sorry for this inconvenient, we are releasing soon the new version with this fix and new analysis. Please let me know if it works or if you have more questions.

sgiavasis commented 6 years ago

Hi @rattmowe,

The errors you are seeing have been resolved in our latest version. Could you please try pulling the new container (we have a new Docker Hub source, see link below) and running again, to confirm that you can get running?

Info on new version here: http://fcp-indi.github.io/docs/user/running.html#with-docker

Thanks! Steve

sgiavasis commented 6 years ago

Closing for inactivity- if the issue persists, please feel free to re-open.