search

alixlam / fmristroke

Adding quality checks and confounds computation steps to fmriprep for stroke data.
Apache License 2.0
4 stars 0 forks source link

Potential bug: not all expectations of fMRIPrep were met #17

Open SRSteinkamp opened 1 month ago

SRSteinkamp commented 1 month ago

Hi,

I hope you can help me trouble shoot the following issue. In both using the data @alixlam provided on OSF and in using some data of my own with a fake lesion mask I am getting this error.

It works, however, using the test data, that is somewhere stored in the installation. An as far as I can see there are no obvious differences between the OSF data and the example data. Except that the osf data also contains T2w images.

Attempted to access pre-existing anatomical derivatives at         <PATH/osfstorage-archive/ARC - derivatives>, however not all expectations of fMRIPrep         were met (for participant <M2112>, spaces <MNI152NLin2009cAsym>,         >).
Process Process-2:
Traceback (most recent call last):
  File "~/anaconda3/envs/joss/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/~/anaconda3/envs/joss/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "~/anaconda3/envs/joss/lib/python3.9/site-packages/fmristroke/cli/workflow.py", line 73, in build_workflow
    retval["workflow"] = init_fmristroke_wf()
  File "~/anaconda3/envs/joss/lib/python3.9/site-packages/fmristroke/workflows/final.py", line 42, in init_fmristroke_wf
    single_subject_wf = init_single_subject_wf(subject_id)
  File "~/anaconda3/envs/joss/lib/python3.9/site-packages/fmristroke/workflows/final.py", line 146, in init_single_subject_wf
    BIDSDerivativeDataGrabber(
  File "/~/anaconda3/envs/joss/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 563, in __init__
    super(SimpleInterface, self).__init__(
  File "~/anaconda3/envs/joss/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 202, in __init__
    self.inputs.trait_set(**inputs)
  File "~/anaconda3/envs/joss/lib/python3.9/site-packages/traits/has_traits.py", line 1520, in trait_set
    setattr(self, name, value)
  File "~/anaconda3/envs/joss/lib/python3.9/site-packages/traits/trait_types.py", line 3101, in validate
    self.error(object, name, value)
  File "~/anaconda3/envs/joss/lib/python3.9/site-packages/traits/base_trait_handler.py", line 74, in error
    raise TraitError(
traits.trait_errors.TraitError: The 'anat_derivatives' trait of a _BIDSDerivativeDataGrabberInputSpec instance must be a dictionary with keys which are a string and with values which are any value, but a value of None <class 'NoneType'> was specified.

Do you have any idea where this issues might be coming from? In any case, a slightly more verbose error message mit be helpful at this stage, cause I am not sure what exactly is missing from fmriprep.

Cheers

PS: reviewing for JOSS (https://github.com/openjournals/joss-reviews/issues/6636)

alixlam commented 1 month ago

hi @SRSteinkamp ! Thank you for your message ! I just found the error the sub-M2112 folder was missing 1 file ... I just added it ("sub-M2112_space-MNI152NLin2009cAsym_label-WM_probseg.nii.gz"). On your data I am not sure, have you run fmriprep with the freesurfer option ? I will work on something to add verbose in the error msg .

SRSteinkamp commented 1 month ago

Thank you!

Regarding my own data, can we maybe check, what fmriprep version you would recommend and which options? Afaik I ran using freesurfer, but that also has a few options ;)

So I ran the code using default settings on the example data and test data. Both terminate with errors.

On the example data the toolbox now terminates with an error related to rapitide (I guess)

240807-18:35:07,777 nipype.workflow CRITICAL:
     fMRIStroke failed: Traceback (most recent call last):
  File "~/anaconda3/envs/joss/lib/python3.9/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "~/anaconda3/envs/joss/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "~/anaconda3/envs/joss/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "~/anaconda3/envs/joss/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node rapidtide.

Cmdline:
    rapidtide PATH/joss_review/work/fmristroke_0_1_wf/single_subject_M2121_wf/lesion_conn_ses_258_task_rest_run_17_space_T1w_desc_preproc_wf/concat_runs_wf/concat_bold_denoised_t1/concat_bold_pipe_3_space_0.nii.gz PATH/joss_review/work/fmristroke_0_1_wf/single_subject_M2121_wf/lesion_conn_ses_258_task_rest_run_17_space_T1w_desc_preproc_wf/hemodynamic_wf/rapidtide/concat --corrmask PATH/joss_review/work/fmristroke_0_1_wf/single_subject_M2121_wf/lesion_conn_ses_258_task_rest_run_17_space_T1w_desc_preproc_wf/hemodynamic_wf/intersect_mask/mask_inter.nii.gz --filterband None --globalmeanexclude PATH/joss_review/work/fmristroke_0_1_wf/single_subject_M2121_wf/lesion_conn_ses_258_task_rest_run_17_space_T1w_desc_preproc_wf/hemodynamic_wf/roi_resamp/mask_resamp.nii.gz --globalmeaninclude PATH/joss_review/work/fmristroke_0_1_wf/single_subject_M2121_wf/lesion_conn_ses_258_task_rest_run_17_space_T1w_desc_preproc_wf/hemodynamic_wf/intersect_mask/mask_inter.nii.gz --motpos --noglm --searchrange -10 10 --spatialfilt 3.000000
Stdout:
    memprofiler does not exist
    no aggressive optimization
    will not use numba even if present
    startpoint set to minimum, (0)
    endppoint set to maximum, ( 9 )
Stderr:
    starting rapidtide 2.6.5
    running single process - disabled shared memory use
    setting internal precision to double
    setting output precision to single
    input file is NIFTI
    oversample factor set to 2
    fmri data: 10 timepoints, tr = 1.0, oversamptr = 0.5
    276696 spatial locations, 10 timepoints
    Traceback (most recent call last):
      File "~/anaconda3/envs/joss/bin/rapidtide", line 23, in <module>
        rapidtide_workflow.rapidtide_main(rapidtide_parser.process_args(inputargs=None))
      File "~/anaconda3/envs/joss/lib/python3.9/site-packages/rapidtide/workflows/rapidtide.py", line 425, in rapidtide_main
        raise ValueError(
    ValueError: magnitude of lagmin exceeds 5.0 - invalid
Traceback:
    Traceback (most recent call last):
      File "~/anaconda3/envs/joss/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 453, in aggregate_outputs
        setattr(outputs, key, val)
      File "~/anaconda3/envs/joss/lib/python3.9/site-packages/nipype/interfaces/base/traits_extension.py", line 330, in validate
        value = super(File, self).validate(objekt, name, value, return_pathlike=True)
      File "~/anaconda3/envs/joss/lib/python3.9/site-packages/nipype/interfaces/base/traits_extension.py", line 135, in validate
        self.error(objekt, name, str(value))
      File "~/anaconda3/envs/joss/lib/python3.9/site-packages/traits/base_trait_handler.py", line 74, in error
        raise TraitError(
    traits.trait_errors.TraitError: The 'output_corrfit_mask' trait of a RapidTideOutputSpec instance must be a pathlike object or string representing an existing file, but a value of 'PATH/joss_review/work/fmristroke_0_1_wf/single_subject_M2121_wf/lesion_conn_ses_258_task_rest_run_17_space_T1w_desc_preproc_wf/hemodynamic_wf/rapidtide/concat_desc-corrfit_mask.nii.gz' <class 'str'> was specified.

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "~/anaconda3/envs/joss/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 400, in run
        outputs = self.aggregate_outputs(runtime)
      File "~/anaconda3/envs/joss/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 460, in aggregate_outputs
        raise FileNotFoundError(msg)
    FileNotFoundError: No such file or directory 'PATH/joss_review/work/fmristroke_0_1_wf/single_subject_M2121_wf/lesion_conn_ses_258_task_rest_run_17_space_T1w_desc_preproc_wf/hemodynamic_wf/rapidtide/concat_desc-corrfit_mask.nii.gz' for output 'output_corrfit_mask' of a RapidTide interface

And running on the data fails due to a bunch of error I think related to ICA and empty masks: 

nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node acc_masks.

Traceback:
    Traceback (most recent call last):
      File "~/anaconda3/envs/joss/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 397, in run
        runtime = self._run_interface(runtime)
      File "~/anaconda3/envs/joss/lib/python3.9/site-packages/fmriprep/interfaces/confounds.py", line 83, in _run_interface
        self._results["out_masks"] = acompcor_masks(
      File "~/anaconda3/envs/joss/lib/python3.9/site-packages/fmriprep/utils/confounds.py", line 133, in acompcor_masks
        csf_file = mask2vf(
      File "~/anaconda3/envs/joss/lib/python3.9/site-packages/fmriprep/utils/confounds.py", line 48, in mask2vf
        max_data = np.percentile(data[data > 0], 99)
      File "~/anaconda3/envs/joss/lib/python3.9/site-packages/numpy/lib/function_base.py", line 4283, in percentile
        return _quantile_unchecked(
      File "~/anaconda3/envs/joss/lib/python3.9/site-packages/numpy/lib/function_base.py", line 4555, in _quantile_unchecked
        return _ureduce(a,
      File "~/anaconda3/envs/joss/lib/python3.9/site-packages/numpy/lib/function_base.py", line 3823, in _ureduce
        r = func(a, **kwargs)
      File "~/anaconda3/envs/joss/lib/python3.9/site-packages/numpy/lib/function_base.py", line 4721, in _quantile_ureduce_func
        result = _quantile(arr,
      File "~/anaconda3/envs/joss/lib/python3.9/site-packages/numpy/lib/function_base.py", line 4830, in _quantile
        slices_having_nans = np.isnan(arr[-1, ...])
    IndexError: index -1 is out of bounds for axis 0 with size 0
alixlam commented 3 weeks ago

hello again @SRSteinkamp ! Sorry for the late reply.

Concerning the fmriprep version used, I use the one that is in the requirements file (23.0.x). I believe the derivatives should be the same for the other versions though ... but will figure something out to be able to know exactly which file is missing. For FreeSurfer I used the default in fmriprep without changing anything.

For the first error, it is another "problem" in the data I provided ... The run 17 of patient 2121 is actually too short (only 10 volumes) for the rapid tide part. When I tested the data, I thought I used the default parameters but I only tested with --session-level option which concatenates the runs of the same session and task making a new run that is long enough. I will add something to warn users about this limit in size of functional runs.

For the second error, you probably are right. On which data are you trying to run the package ?

SRSteinkamp commented 3 weeks ago

I think the second error was on the test data (test data from within the python installation).

Good to know, will try to give it a new run on my own data using fmriprep 23.0.x, and using --session-level for the OSF data.

SRSteinkamp commented 1 week ago

Hey, thanks for all the work.

I finally got it to run on my own data :) ... The issue seems to be, that fmristroke is not looking for anat in session folders.

I.e. I had my data ordered as:

sub-001
    ses-01
        anat
        func

and fmristroke seems to build fixed paths assuming no session identifier (or other entities) in the name or path.

As this would essentially mean changing the whole data loading process, I am not sure if you want to go that far. But there should be a note regarding data organization, that fmristroke expects some anat_T1w.nii.gz and it's derivatives in an anat folder in the top-directory.

I think it should be a feature in the future, however, to change data-loading to a more flexible structure, adding in multiple entities, sessions and folder structures.

And a last note: The method section in my reports show the error: "Failed to generate boilerplate". Is this intended?

alixlam commented 3 days ago

Thank you for your feedback and your patience ! :) Yes indeed as soon as there are more than one session fmriprep outputs 1 preprocessed T1w in the main folder. Should be a quick fix to add the case when only one session is given when searching for the derivatives. (will do a commit soon for that)

Concerning the last note, it is indeed something I am trying to change, but if you give a different output folder than the fmriprep derivatives folder, the boilerplate is not found. I will definitely work on this in the future.