search

PennLINC / qsirecon

Reconstruction of preprocessed q-space images (dMRI)
https://qsirecon.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
3 stars 2 forks source link

PyAFQ failure #129

Open araikes opened 4 days ago

araikes commented 4 days ago

Summary

Piggybacking off of #122, I ran into errors with PyAFQ (--recon-spec pyqfq_tractometry). This one seems a bit more challenging/problematic to diagnose and I understand that checking on n = 1 isn't super helpful, but it's where we're at for the moment. Essentially, I'm getting an error that there's a NaN response in the CSD fitting. I haven't tested the MRtrix + PyAFQ pipeline (MRtrix MSMT ACT HSVS took 52 hours and just finished), but I know that I get a valid CSD response function from these data based on the other pipelines, which leads me to believe that something isn't getting passed to the node.

Additional details

What were you trying to do?

Test for unanticipated failures.

What did you expect to happen?

Unanticipated failures

What actually happened?

Unanticipated failure

Reproducing the bug

apptainer run -\
-containall --writable-tmpfs \
-B $PWD/derivatives/qsiprep_0.22.1/qsiprep:/input \
-B $PWD/derivatives/qsirecon:/output \
-B /tmp:/tmp \
-B /groups/adamraikes/license.txt:/license.txt \
$RECON /input /output participant \
--participant_label 001 \
-w /tmp  \
--nthreads 8 --omp-nthreads 8 --fs-license-file /license.txt --recon-spec pyafq_tractometry--input-type qsiprep

Crash log

Node: qsirecon_0_23_wf.sub-001_pyafq_tractometry.sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf.pyafq_tractometry.run_afq
Working directory: /tmp/qsirecon_0_23_wf/sub-001_pyafq_tractometry/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/pyafq_tractometry/run_afq

Node inputs:

bval_file = <undefined>
bvec_file = <undefined>
dwi_file = <undefined>
itk_file = <undefined>
kwargs = {'directions': 'prob', 'max_angle': 30.0, 'sphere': None, 'seed_mask': None, 'seed_threshold': 0, 'n_seeds': 1, 'random_seeds': False, 'rng_seed': None, 'stop_mask': None, 'stop_threshold': 0, 'step_size': 0.5, 'odf_model': 'CSD', 'tracker': 'local', 'nb_points': False, 'nb_streamlines': False, 'seg_algo': 'AFQ', 'clip_edges': False, 'parallel_segmentation': {'n_jobs': -1, 'engine': 'joblib', 'backend': 'loky'}, 'progressive': True, 'greater_than': 50, 'rm_small_clusters': 50, 'model_clust_thr': 1.25, 'reduction_thr': 25, 'refine': False, 'pruning_thr': 12, 'b0_threshold': 50, 'prob_threshold': 0, 'roi_dist_tie_break': False, 'dist_to_waypoint': None, 'rng': None, 'return_idx': False, 'presegment_bundle_dict': None, 'presegment_kwargs': {}, 'filter_by_endpoints': True, 'dist_to_atlas': 4, 'save_intermediates': None, 'n_points': 100, 'clean_rounds': 5, 'distance_threshold': 3, 'length_threshold': 4, 'min_sl': 20, 'stat': 'mean', 'min_bval': None, 'max_bval': None, 'filter_b': True, 'robust_tensor_fitting': False, 'csd_response': None, 'csd_sh_order': None, 'csd_lambda_': 1, 'csd_tau': 0.1, 'gtol': 0.01, 'brain_mask_definition': None, 'bundle_info': None, 'reg_template_spec': 'mni_T1', 'mapping_definition': None, 'reg_subject_spec': 'power_map', 'profile_weights': 'gauss', 'scalars': ['dti_fa', 'dti_md'], 'import_tract': None, 'sbv_lims_bundles': [None, None], 'volume_opacity_bundles': 0.3, 'n_points_bundles': 40, 'sbv_lims_indiv': [None, None], 'volume_opacity_indiv': 0.3, 'n_points_indiv': 40, 'viz_backend_spec': 'plotly_no_gif', 'virtual_frame_buffer': False, 'omp_nthreads': 8}
mask_file = <undefined>
n_procs = 8
tck_file = <undefined>

Traceback (most recent call last):
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node run_afq.

Traceback:
    Traceback (most recent call last):
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/data.py", line 335, in csd_params
        csdf = csd_fit_model(
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/models/csd.py", line 67, in _fit
        return _model(gtab, data, response, sh_order, csd_fa_thr).fit(
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/models/csd.py", line 56, in _model
        raise CsdNanResponseError
    AFQ.models.csd.CsdNanResponseError

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 397, in run
        runtime = self._run_interface(runtime)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsirecon/interfaces/pyafq.py", line 114, in _run_interface
        myafq.export_all()
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/api/participant.py", line 202, in export_all
        export_all_helper(self, seg_algo, xforms, indiv, viz)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/api/utils.py", line 121, in export_all_helper
        api_afq_object.export("template_xform")
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/api/participant.py", line 155, in export
        return self.wf_dict[section][attr_name]
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
        self._run_node(self.plan.efferents[k])
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
        if not found: res = node(self)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 90, in __call__
        args.append(opts[name])
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 889, in __getitem__
        return self._examine_val(k, ps.PMap.__getitem__(self, k))
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 885, in _examine_val
        val = val()
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1164, in <lambda>
        def curry_choice(k, args): return lambda:choose_fn(k, args)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1133, in _choose_last
        return vs[-1][k]
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
        self._run_node(self.plan.efferents[k])
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
        if not found: res = node(self)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 90, in __call__
        args.append(opts[name])
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 889, in __getitem__
        return self._examine_val(k, ps.PMap.__getitem__(self, k))
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 885, in _examine_val
        val = val()
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1164, in <lambda>
        def curry_choice(k, args): return lambda:choose_fn(k, args)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1133, in _choose_last
        return vs[-1][k]
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
        self._run_node(self.plan.efferents[k])
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
        if not found: res = node(self)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 91, in __call__
        result = self.function(*args)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/mapping.py", line 182, in get_reg_subject
        reg_subject_spec = data_imap[filename_dict[reg_subject_spec]]
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
        self._run_node(self.plan.efferents[k])
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
        if not found: res = node(self)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 90, in __call__
        args.append(opts[name])
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 889, in __getitem__
        return self._examine_val(k, ps.PMap.__getitem__(self, k))
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 885, in _examine_val
        val = val()
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1164, in <lambda>
        def curry_choice(k, args): return lambda:choose_fn(k, args)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1133, in _choose_last
        return vs[-1][k]
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
        self._run_node(self.plan.efferents[k])
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
        if not found: res = node(self)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 91, in __call__
        result = self.function(*args)
      File "<string>", line 2, in wrapper_has_args_func
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/decorators.py", line 149, in wrapper_as_file
        gen, meta = func(*args[:og_arg_count], **kwargs)
      File "<string>", line 2, in wrapper_has_args_func
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/decorators.py", line 220, in wrapper_as_img
        data, meta = func(*args[:og_arg_count], **kwargs)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/data.py", line 342, in csd_params
        raise CsdNanResponseError(
    AFQ.models.csd.CsdNanResponseError: Could not compute CSD response function for file: 
    <class 'nibabel.nifti1.Nifti1Image'>
    data shape (161, 198, 171, 137)
    affine:
    [[ -1.   0.   0.  80.]
     [  0.  -1.   0.  81.]
     [  0.   0.   1. -80.]
     [  0.   0.   0.   1.]]
    metadata:
    <class 'nibabel.nifti1.Nifti1Header'> object, endian='<'
    sizeof_hdr      : 348
    data_type       : b''
    db_name         : b''
    extents         : 0
    session_error   : 0
    regular         : b''
    dim_info        : 0
    dim             : [  4 161 198 171 137   1   1   1]
    intent_p1       : 0.0
    intent_p2       : 0.0
    intent_p3       : 0.0
    intent_code     : none
    datatype        : float64
    bitpix          : 64
    slice_start     : 0
    pixdim          : [1. 1. 1. 1. 1. 1. 1. 1.]
    vox_offset      : 0.0
    scl_slope       : nan
    scl_inter       : nan
    slice_end       : 0
    slice_code      : unknown
    xyzt_units      : 0
    cal_max         : 0.0
    cal_min         : 0.0
    slice_duration  : 0.0
    toffset         : 0.0
    glmax           : 0
    glmin           : 0
    descrip         : b''
    aux_file        : b''
    qform_code      : unknown
    sform_code      : aligned
    quatern_b       : 0.0
    quatern_c       : 0.0
    quatern_d       : 1.0
    qoffset_x       : 80.0
    qoffset_y       : 81.0
    qoffset_z       : -80.0
    srow_x          : [-1.  0.  0. 80.]
    srow_y          : [ 0. -1.  0. 81.]
    srow_z          : [  0.   0.   1. -80.]
    intent_name     : b''
    magic           : b'n+1'
    .
araikes commented 3 days ago

Running this on the current state of #131. Crash is the same but now populated with file paths:

Node: qsirecon_0_0_wf.sub-001_pyafq_tractometry.sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf.pyafq_tractometry.run_afq
Working directory: /output/scratch/qsirecon_0_0_wf/sub-001_pyafq_tractometry/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/pyafq_tractometry/run_afq

Node inputs:

bval_file = /input/qsiprep/sub-001/ses-01/dwi/sub-001_ses-01_dir-AP_run-001_space-T1w_desc-preproc_dwi.bval
bvec_file = /input/qsiprep/sub-001/ses-01/dwi/sub-001_ses-01_dir-AP_run-001_space-T1w_desc-preproc_dwi.bvec
dwi_file = /input/qsiprep/sub-001/ses-01/dwi/sub-001_ses-01_dir-AP_run-001_space-T1w_desc-preproc_dwi.nii.gz
itk_file = /input/qsiprep/sub-001/anat/sub-001_from-MNI152NLin2009cAsym_to-T1w_mode-image_xfm.h5
kwargs = {'directions': 'prob', 'max_angle': 30.0, 'sphere': None, 'seed_mask': None, 'seed_threshold': 0, 'n_seeds': 1, 'random_seeds': False, 'rng_seed': None, 'stop_mask': None, 'stop_threshold': 0, 'step_size': 0.5, 'odf_model': 'CSD', 'tracker': 'local', 'nb_points': False, 'nb_streamlines': False, 'seg_algo': 'AFQ', 'clip_edges': False, 'parallel_segmentation': {'n_jobs': -1, 'engine': 'joblib', 'backend': 'loky'}, 'progressive': True, 'greater_than': 50, 'rm_small_clusters': 50, 'model_clust_thr': 1.25, 'reduction_thr': 25, 'refine': False, 'pruning_thr': 12, 'b0_threshold': 50, 'prob_threshold': 0, 'roi_dist_tie_break': False, 'dist_to_waypoint': None, 'rng': None, 'return_idx': False, 'presegment_bundle_dict': None, 'presegment_kwargs': {}, 'filter_by_endpoints': True, 'dist_to_atlas': 4, 'save_intermediates': None, 'n_points': 100, 'clean_rounds': 5, 'distance_threshold': 3, 'length_threshold': 4, 'min_sl': 20, 'stat': 'mean', 'min_bval': None, 'max_bval': None, 'filter_b': True, 'robust_tensor_fitting': False, 'csd_response': None, 'csd_sh_order': None, 'csd_lambda_': 1, 'csd_tau': 0.1, 'gtol': 0.01, 'brain_mask_definition': None, 'bundle_info': None, 'reg_template_spec': 'mni_T1', 'mapping_definition': None, 'reg_subject_spec': 'power_map', 'profile_weights': 'gauss', 'scalars': ['dti_fa', 'dti_md'], 'import_tract': None, 'sbv_lims_bundles': [None, None], 'volume_opacity_bundles': 0.3, 'n_points_bundles': 40, 'sbv_lims_indiv': [None, None], 'volume_opacity_indiv': 0.3, 'n_points_indiv': 40, 'viz_backend_spec': 'plotly_no_gif', 'virtual_frame_buffer': False, 'omp_nthreads': 8}
mask_file = /output/scratch/qsirecon_0_0_wf/sub-001_pyafq_tractometry/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_dwi_specific_anat_wf/resample_mask/sub-001_desc-brain_mask_trans.nii.gz
n_procs = 8
tck_file = <undefined>

Traceback (most recent call last):
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node run_afq.

Traceback:
    Traceback (most recent call last):
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/data.py", line 335, in csd_params
        csdf = csd_fit_model(
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/models/csd.py", line 67, in _fit
        return _model(gtab, data, response, sh_order, csd_fa_thr).fit(
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/models/csd.py", line 56, in _model
        raise CsdNanResponseError
    AFQ.models.csd.CsdNanResponseError

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 397, in run
        runtime = self._run_interface(runtime)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsirecon/interfaces/pyafq.py", line 114, in _run_interface
        myafq.export_all()
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/api/participant.py", line 202, in export_all
        export_all_helper(self, seg_algo, xforms, indiv, viz)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/api/utils.py", line 121, in export_all_helper
        api_afq_object.export("template_xform")
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/api/participant.py", line 155, in export
        return self.wf_dict[section][attr_name]
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
        self._run_node(self.plan.efferents[k])
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
        if not found: res = node(self)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 90, in __call__
        args.append(opts[name])
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 889, in __getitem__
        return self._examine_val(k, ps.PMap.__getitem__(self, k))
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 885, in _examine_val
        val = val()
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1164, in <lambda>
        def curry_choice(k, args): return lambda:choose_fn(k, args)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1133, in _choose_last
        return vs[-1][k]
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
        self._run_node(self.plan.efferents[k])
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
        if not found: res = node(self)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 90, in __call__
        args.append(opts[name])
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 889, in __getitem__
        return self._examine_val(k, ps.PMap.__getitem__(self, k))
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 885, in _examine_val
        val = val()
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1164, in <lambda>
        def curry_choice(k, args): return lambda:choose_fn(k, args)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1133, in _choose_last
        return vs[-1][k]
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
        self._run_node(self.plan.efferents[k])
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
        if not found: res = node(self)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 91, in __call__
        result = self.function(*args)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/mapping.py", line 182, in get_reg_subject
        reg_subject_spec = data_imap[filename_dict[reg_subject_spec]]
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
        self._run_node(self.plan.efferents[k])
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
        if not found: res = node(self)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 90, in __call__
        args.append(opts[name])
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 889, in __getitem__
        return self._examine_val(k, ps.PMap.__getitem__(self, k))
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 885, in _examine_val
        val = val()
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1164, in <lambda>
        def curry_choice(k, args): return lambda:choose_fn(k, args)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/util.py", line 1133, in _choose_last
        return vs[-1][k]
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 470, in __getitem__
        self._run_node(self.plan.efferents[k])
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 534, in _run_node
        if not found: res = node(self)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/pimms/calculation.py", line 91, in __call__
        result = self.function(*args)
      File "<string>", line 2, in wrapper_has_args_func
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/decorators.py", line 149, in wrapper_as_file
        gen, meta = func(*args[:og_arg_count], **kwargs)
      File "<string>", line 2, in wrapper_has_args_func
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/decorators.py", line 220, in wrapper_as_img
        data, meta = func(*args[:og_arg_count], **kwargs)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/AFQ/tasks/data.py", line 342, in csd_params
        raise CsdNanResponseError(
    AFQ.models.csd.CsdNanResponseError: Could not compute CSD response function for file: 
    <class 'nibabel.nifti1.Nifti1Image'>
    data shape (161, 198, 171, 137)
    affine:
    [[ -1.   0.   0.  80.]
     [  0.  -1.   0.  81.]
     [  0.   0.   1. -80.]
     [  0.   0.   0.   1.]]
    metadata:
    <class 'nibabel.nifti1.Nifti1Header'> object, endian='<'
    sizeof_hdr      : 348
    data_type       : b''
    db_name         : b''
    extents         : 0
    session_error   : 0
    regular         : b''
    dim_info        : 0
    dim             : [  4 161 198 171 137   1   1   1]
    intent_p1       : 0.0
    intent_p2       : 0.0
    intent_p3       : 0.0
    intent_code     : none
    datatype        : float64
    bitpix          : 64
    slice_start     : 0
    pixdim          : [1. 1. 1. 1. 1. 1. 1. 1.]
    vox_offset      : 0.0
    scl_slope       : nan
    scl_inter       : nan
    slice_end       : 0
    slice_code      : unknown
    xyzt_units      : 0
    cal_max         : 0.0
    cal_min         : 0.0
    slice_duration  : 0.0
    toffset         : 0.0
    glmax           : 0
    glmin           : 0
    descrip         : b''
    aux_file        : b''
    qform_code      : unknown
    sform_code      : aligned
    quatern_b       : 0.0
    quatern_c       : 0.0
    quatern_d       : 1.0
    qoffset_x       : 80.0
    qoffset_y       : 81.0
    qoffset_z       : -80.0
    srow_x          : [-1.  0.  0. 80.]
    srow_y          : [ 0. -1.  0. 81.]
    srow_z          : [  0.   0.   1. -80.]
    intent_name     : b''
    magic           : b'n+1'
    .
araikes commented 2 days ago

From the terminal output, this immediately precedes the error:

INFO:AFQ:No seed mask given, using FA (or first scalar if none are FA)thresholded to 0.2
INFO:AFQ:No stop mask given, using FA (or first scalar if none are FA)thresholded to 0.2
INFO:AFQ:Saving /xdisk/adamraikes/PD_allo/derivatives/qsirecon_unstable_v2/scratch/qsirecon_0_0_wf/sub-001_pyafq_tractometry/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/pyafq_tractometry/run_afq/study/subject/PYAFQ/sub-001_ses-01_dir-AP_run-001_desc-brainmasktrans_dwi.nii.gz
INFO:AFQ:Saving /xdisk/adamraikes/PD_allo/derivatives/qsirecon_unstable_v2/scratch/qsirecon_0_0_wf/sub-001_pyafq_tractometry/sub_001_ses_02_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/pyafq_tractometry/run_afq/study/subject/PYAFQ/sub-001_ses-02_dir-AP_run-001_desc-brainmasktrans_dwi.nii.gz
WARNING:AFQ:Failed to export warped b0. This could be because your mapping type is only compatible with transformation from template to subject space. The error is: Could not compute CSD response function for file: 
<class 'nibabel.nifti1.Nifti1Image'>
smeisler commented 4 hours ago

I wonder if this is due to an older pyafq version? I remember this, or a similar error, being discussed here: https://github.com/yeatmanlab/pyAFQ/issues/1156

araikes commented 4 hours ago

So possibly related... (and meant to post this on Friday but apparently didn't actually hit the button)...

multishell_scalarfest yields a PeaksReport failure, also related to ODF nans:

Cmdline:
    recon_plot --background_image /input/qsiprep/sub-001/ses-01/dwi/sub-001_ses-01_dir-AP_run-001_space-T1w_dwiref.nii.gz --directions /output/scratch/qsirecon_0_0_wf/sub-001_multishell_scalarfest/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/mapmri_recon/recon_map/sub-001_ses-01_dir-AP_run-001_space-T1w_desc-preproc_dwi_MAPMRI_dirs.npy --mask_file /output/scratch/qsirecon_0_0_wf/sub-001_multishell_scalarfest/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_dwi_specific_anat_wf/resample_mask/sub-001_desc-brain_mask_trans.nii.gz --amplitudes /output/scratch/qsirecon_0_0_wf/sub-001_multishell_scalarfest/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/mapmri_recon/recon_map/sub-001_ses-01_dir-AP_run-001_space-T1w_desc-preproc_dwi_MAPMRI_amp.nii.gz --odfs_image odfs_mosaic.png --odf_rois /output/scratch/qsirecon_0_0_wf/sub-001_multishell_scalarfest/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_dwi_specific_anat_wf/odf_rois/crossing_rois_trans.nii.gz --peaks_image peaks_mosaic.png
Stdout:
    241025-15:42:08,45 nipype.interface INFO:
         loading amplitudes=/output/scratch/qsirecon_0_0_wf/sub-001_multishell_scalarfest/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/mapmri_recon/recon_map/sub-001_ses-01_dir-AP_run-001_space-T1w_desc-preproc_dwi_MAPMRI_amp.nii.gz, directions=/output/scratch/qsirecon_0_0_wf/sub-001_multishell_scalarfest/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/mapmri_recon/recon_map/sub-001_ses-01_dir-AP_run-001_space-T1w_desc-preproc_dwi_MAPMRI_dirs.npy to plot ODF/peaks
    241025-15:42:32,401 nipype.interface INFO:
         saving peaks image to /output/scratch/qsirecon_0_0_wf/sub-001_multishell_scalarfest/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/mapmri_recon/plot_peaks/peaks_mosaic.png
    241025-15:42:32,591 nipype.interface INFO:
         Plotting slice indices {'x': [73, 80, 87], 'y': [90, 99, 107], 'z': [78, 85, 92]}
Stderr:
    Traceback (most recent call last):
      File "/opt/conda/envs/qsiprep/bin/recon_plot", line 8, in <module>
        sys.exit(recon_plot())
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsirecon/cli/recon_plot.py", line 129, in recon_plot
        peak_slice_series(
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsirecon/cli/recon_plot.py", line 262, in peak_slice_series
        plot_peak_slice(
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsirecon/cli/recon_plot.py", line 192, in plot_peak_slice
        peak_dirs, peak_values = peaks_from_odfs(
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsirecon/cli/recon_plot.py", line 335, in peaks_from_odfs
        direction, pk, ind = peak_directions(
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/dipy/direction/peaks.py", line 132, in peak_directions
        values, indices = local_maxima(odf, sphere.edges)
      File "recspeed.pyx", line 249, in dipy.reconst.recspeed.local_maxima
    ValueError: odf can not have nans
Traceback:
    Traceback (most recent call last):
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 453, in aggregate_outputs
        setattr(outputs, key, val)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/traits_extension.py", line 330, in validate
        value = super(File, self).validate(objekt, name, value, return_pathlike=True)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/traits_extension.py", line 135, in validate
        self.error(objekt, name, str(value))
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/traits/base_trait_handler.py", line 74, in error
        raise TraitError(
    traits.trait_errors.TraitError: The 'peak_report' trait of a _ReconPeaksReportOutputSpec instance must be a pathlike object or string representing an existing file, but a value of '/output/scratch/qsirecon_0_0_wf/sub-001_multishell_scalarfest/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/mapmri_recon/plot_peaks/peaks_mosaic.png' <class 'str'> was specified.

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 400, in run
        outputs = self.aggregate_outputs(runtime)
      File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 460, in aggregate_outputs
        raise FileNotFoundError(msg)
    FileNotFoundError: No such file or directory '/output/scratch/qsirecon_0_0_wf/sub-001_multishell_scalarfest/sub_001_ses_01_dir_AP_run_001_space_T1w_desc_preproc_recon_wf/mapmri_recon/plot_peaks/peaks_mosaic.png' for output 'peak_report' of a CLIReconPeaksReport interface

So PyAFQ and MAPMRI, which both run DIPY to generate the ODFs, fail due to nans. By contrast, MRtrix has no issue estimating FODs for either the connectivity pipelines or PyAFQ.