search

nipreps / mriqc

Automated Quality Control and visual reports for Quality Assessment of structural (T1w, T2w) and functional MRI of the brain
http://mriqc.readthedocs.io
Apache License 2.0
300 stars 132 forks source link

MRIQC IndexError - 'Cannot do a non-empty take from an empty axes' #1048

Closed demidenm closed 1 year ago

demidenm commented 1 year ago

Running MRIQC (singularity: nipreps/mriqc:22.0.6) and have an error that @effigies described as, "interface is not checking for empty masks before masking." Out of on the ~15706 session data, this has occurred for 27 subjects. It appears as though this is an issue that has come up in older version, see issue #868. I cleared the working scratch folders and retried on a subject to no avail.

Curious how I may troubleshoot this error.

The MRIQC command:

singularity run --cleanenv \
-B ${data_dir}/bids_dir/sub-${subj_id}_ses-${ses_id}:/bids_dir \
-B ${data_dir}/processed/mriqc_v22_0_6/sub-${subj_id}_ses-${ses_id}:/output_dir \
-B ${data_dir}/work_dir/mriqc_v22_0_6/sub-${subj_id}_ses-${ses_id}:/wd \
${sif_img} \
/bids_dir /output_dir participant \
--ants-nthreads 8 \
--nprocs 12 \
--mem_gb 30 \
-vv \
--verbose-reports \
-w /wd \
-m bold T1w T2w

Error log:

[Node] Error on "mriqc_wf.anatMRIQC.ComputeIQMs.measures" (/wd/mriqc_wf/anatMRIQC/ComputeIQMs/_in_file_..bids_dir..sub-[redacted]..[redacted].ses-[redacted]..anat..sub-[redacted]_ses-[redacted]_T2w.nii.gz/measures)
230128-17:15:57,757 nipype.workflow ERROR:
     Node measures.a1 failed to run on host cn1071.
230128-17:15:57,760 nipype.workflow ERROR:
     Saving crash info to /output_dir/logs/crash-20230128-171557-mdemiden-measures.a1-8caeb971-97eb-4c30-80f8-8dede7fa7627.txt
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/mriqc/engine/plugin.py", line 60, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 524, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 642, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 750, in _run_command
    raise NodeExecutionError(
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node measures.

Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 398, in run
    runtime = self._run_interface(runtime)
  File "/opt/conda/lib/python3.9/site-packages/mriqc/interfaces/anatomical.py", line 164, in _run_interface
    stats = summary_stats(inudata, pvmdata, airdata, erode=erode)
  File "/opt/conda/lib/python3.9/site-packages/mriqc/qc/anatomical.py", line 637, in summary_stats
    "p95": float(np.percentile(img[mask == 1], 95)),
  File "<__array_function__ internals>", line 180, in percentile
  File "/opt/conda/lib/python3.9/site-packages/numpy/lib/function_base.py", line 4134, in percentile
    return _quantile_unchecked(
  File "/opt/conda/lib/python3.9/site-packages/numpy/lib/function_base.py", line 4383, in _quantile_unchecked
    r, k = _ureduce(a,
  File "/opt/conda/lib/python3.9/site-packages/numpy/lib/function_base.py", line 3702, in _ureduce
    r = func(a, **kwargs)
  File "/opt/conda/lib/python3.9/site-packages/numpy/lib/function_base.py", line 4552, in _quantile_ureduce_func
    result = _quantile(arr,
  File "/opt/conda/lib/python3.9/site-packages/numpy/lib/function_base.py", line 4658, in _quantile
    take(arr, indices=-1, axis=DATA_AXIS)
  File "<__array_function__ internals>", line 180, in take
  File "/opt/conda/lib/python3.9/site-packages/numpy/core/fromnumeric.py", line 190, in take
    return _wrapfunc(a, 'take', indices, axis=axis, out=out, mode=mode)
  File "/opt/conda/lib/python3.9/site-packages/numpy/core/fromnumeric.py", line 57, in _wrapfunc
    return bound(*args, **kwds)
IndexError: cannot do a non-empty take from an empty axes.

Crash log:

`Node: mriqc_wf.anatMRIQC.ComputeIQMs.measures
Working directory: /wd/mriqc_wf/anatMRIQC/ComputeIQMs/_in_file_..bids_dir..[redacted]..ses-[redacted]..ses-[redacted]..anat..[redacted]_ses-[redacted]_T2w.nii.gz/measures

Node inputs:

air_msk = <undefined>
artifact_msk = <undefined>
head_msk = <undefined>
human = True
in_bias = <undefined>
in_file = <undefined>
in_fwhm = <undefined>
in_noinu = <undefined>
in_pvms = <undefined>
in_segm = <undefined>
in_tpms = <undefined>
mni_tpms = <undefined>
rot_msk = <undefined>

Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/mriqc/engine/plugin.py", line 60, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 524, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 642, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 750, in _run_command
    raise NodeExecutionError(
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node measures.

Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 398, in run
    runtime = self._run_interface(runtime)
  File "/opt/conda/lib/python3.9/site-packages/mriqc/interfaces/anatomical.py", line 164, in _run_interface
    stats = summary_stats(inudata, pvmdata, airdata, erode=erode)
  File "/opt/conda/lib/python3.9/site-packages/mriqc/qc/anatomical.py", line 637, in summary_stats
    "p95": float(np.percentile(img[mask == 1], 95)),
  File "<__array_function__ internals>", line 180, in percentile
  File "/opt/conda/lib/python3.9/site-packages/numpy/lib/function_base.py", line 4134, in percentile
    return _quantile_unchecked(
  File "/opt/conda/lib/python3.9/site-packages/numpy/lib/function_base.py", line 4383, in _quantile_unchecked
    r, k = _ureduce(a,
  File "/opt/conda/lib/python3.9/site-packages/numpy/lib/function_base.py", line 3702, in _ureduce
    r = func(a, **kwargs)
  File "/opt/conda/lib/python3.9/site-packages/numpy/lib/function_base.py", line 4552, in _quantile_ureduce_func
    result = _quantile(arr,
  File "/opt/conda/lib/python3.9/site-packages/numpy/lib/function_base.py", line 4658, in _quantile
    take(arr, indices=-1, axis=DATA_AXIS)
  File "<__array_function__ internals>", line 180, in take
  File "/opt/conda/lib/python3.9/site-packages/numpy/core/fromnumeric.py", line 190, in take
    return _wrapfunc(a, 'take', indices, axis=axis, out=out, mode=mode)
  File "/opt/conda/lib/python3.9/site-packages/numpy/core/fromnumeric.py", line 57, in _wrapfunc
    return bound(*args, **kwds)
IndexError: cannot do a non-empty take from an empty axes.`
effigies commented 1 year ago

The issue here is that this function:

https://github.com/nipreps/mriqc/blob/1ffd4c8d1a20b44ebfea648a7b12bb32a425d4ec/mriqc/qc/anatomical.py#L560-L645

Needs to decide what to do if a label contains no voxels. Should it be dropped, or should the results all be None or nan, or is raising an exception appropriate? If raising an exception is appropriate, all interfaces that call it should check for that exception and decide how to handle this case, as interfaces should not raise exceptions for empty ROIs.

effigies commented 1 year ago

@mckenziephagen or @celprov could I ask one of you to make the call on this? My guess is that either the ROI should be dropped or filled with nans, and then any calling code should verify that the result isn't empty or that nans are properly handled. But you are much more in the weeds of how MRIQC works/should work than I am...

@demidenm Would you be interested in submitting a PR once a decision is made?

demidenm commented 1 year ago

@effigies - Thanks for the follow up, Chris. Do you suggest that the issue is with the final values so updating to something such as? 

if not img:
    output = {
        "mean": np.nan,
        "stdv": np.nan,
        "median": np.nan,
        "mad": np.nan,
        "p95": np.nan,
        "p05": np.nan,
        "k": np.nan,
        "n": np.nan,
    }
else:
    output = { 
        "mean": float(img[mask == 1].mean()), 
        "stdv": float(img[mask == 1].std()), 
        "median": float(np.median(img[mask == 1])), 
        "mad": float(mad(img[mask == 1])), 
        "p95": float(np.percentile(img[mask == 1], 95)), 
        "p05": float(np.percentile(img[mask == 1], 5)), 
        "k": float(kurtosis(img[mask == 1])), 
        "n": nvox, 
    }
effigies commented 1 year ago

I had a very quick look through earlier, but looking at it again, it seems a solution is already there for one ROI, we just need to make it accessible to other ROIs:

https://github.com/nipreps/mriqc/blob/1ffd4c8d1a20b44ebfea648a7b12bb32a425d4ec/mriqc/qc/anatomical.py#L621-L632

Why not just:

-            if k == "bg":
-                output["bg"] = {
+            if k == "bg" or nvox == 0:
+                output[k] = {
demidenm commented 1 year ago

@effigies That makes more sense! Incorrectly misread the no voxels as empty img 🤦‍♂️