Number of DWIs, b-vals and b-vecs mismatch

[ta4218]

Hello, I am running the dwi-preprocessing-using-t1 pipeline on some ADNI data. I have been receiving the following error which causes the pipeline to crash: "OSError: Number of DWIs, b-vals and b-vecs mismatch (# DWI = 34, # B-vec = 32, #B-val = 32)"

I can remove the specific image from my .tsv file but then a new image error occurs with a new mismatch. Is there a way to identify all images that have mismatches prior to running, or to ignore this error and run without the pipeline crashing.

Thanks, Tom

[NicolasGensollen]

Hi @ta4218

Thanks for reporting. This error means that, at some point, you are trying to associate a DWI image with 34 directions with a B-vector and a B-value having only 32 elements. It is very difficult to tell you more without more details about your setup. What version of Clinica are you using ? Which OS (name and version) are you working on ? What command did you ran ? What does your input data look like ? What is the full error traceback that you get when running the command ?

Best, Nicolas

[ta4218]

Hi Nicolas,

I am currently using Clinica 0.7.7. My OS is Red Hat Enterprise Linux 8.5 (Ootpa). As for the input data, it is in BIDS format; could you clarify what you mean by what the input data looks like?

The command line is: clinica run dwi-preprocessing-using-t1 $EPHEMERAL/ADNI234_2/adni_bids $EPHEMERAL/ADNI234_2/adni_caps -tsv $EPHEMERAL/ADNI234_2/batches/dwi_session_batch_0.tsv --n_procs 128 -wd $EPHEMERAL

The error traceback looks like this:

File: /rds/general/user/ta4218/home/crash-20240306-194813-ta4218-0-InitNode.a146-0e568070-f89a-4cd5-9bdf-dbda0d840ffc.pklz
Node: dwi-preprocessing-using-t1.0-InitNode
Working directory: /rds/general/ephemeral/user/ta4218/ephemeral/dwi-preprocessing-using-t1/41c530d5d9250f06c8dc044adf956ca4c023d037/0-InitNode

Node inputs:

bval = /rds/general/ephemeral/user/ta4218/ephemeral/ADNI234_2/adni_bids/sub-ADNI099S2146/ses-M72/dwi/sub-ADNI099S2146_ses-M72_dwi.bval
bvec = /rds/general/ephemeral/user/ta4218/ephemeral/ADNI234_2/adni_bids/sub-ADNI099S2146/ses-M72/dwi/sub-ADNI099S2146_ses-M72_dwi.bvec
dwi = /rds/general/ephemeral/user/ta4218/ephemeral/ADNI234_2/adni_bids/sub-ADNI099S2146/ses-M72/dwi/sub-ADNI099S2146_ses-M72_dwi.nii.gz
dwi_json = /rds/general/ephemeral/user/ta4218/ephemeral/ADNI234_2/adni_bids/sub-ADNI099S2146/ses-M72/dwi/sub-ADNI099S2146_ses-M72_dwi.json
function_str = def init_input_node(t1w, dwi, bvec, bval, dwi_json):
    """Initialize the pipeline."""
    from clinica.utils.dwi import DWIDataset, bids_dir_to_fsl_dir, check_dwi_volume
    from clinica.utils.filemanip import (
        extract_metadata_from_json,
        get_subject_id,
        handle_missing_keys_dwi,
    )
    from clinica.utils.ux import print_begin_image

    image_id = get_subject_id(t1w)
    check_dwi_volume(DWIDataset(dwi=dwi, b_values=bval, b_vectors=bvec))

    [total_readout_time, phase_encoding_direction] = extract_metadata_from_json(
        dwi_json,
        [
            "TotalReadoutTime",
            "PhaseEncodingDirection",
        ],
        handle_missing_keys=handle_missing_keys_dwi,
    )
    phase_encoding_direction = bids_dir_to_fsl_dir(phase_encoding_direction)

    # Print begin message
    print_begin_image(
        image_id,
        ["TotalReadoutTime", "PhaseEncodingDirection"],
        [str(total_readout_time), phase_encoding_direction],
    )

    return (
        image_id,
        t1w,
        dwi,
        bvec,
        bval,
        total_readout_time,
        phase_encoding_direction,
    )

t1w = /rds/general/ephemeral/user/ta4218/ephemeral/ADNI234_2/adni_bids/sub-ADNI099S2146/ses-M72/anat/sub-ADNI099S2146_ses-M72_T1w.nii.gz

Traceback: 
Traceback (most recent call last):
  File "/rds/general/user/ta4218/home/anaconda3/envs/clinicaEnv/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/rds/general/user/ta4218/home/anaconda3/envs/clinicaEnv/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 524, in run
    result = self._run_interface(execute=True)
  File "/rds/general/user/ta4218/home/anaconda3/envs/clinicaEnv/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 642, in _run_interface
    return self._run_command(execute)
  File "/rds/general/user/ta4218/home/anaconda3/envs/clinicaEnv/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 750, in _run_command
    raise NodeExecutionError(
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node 0-InitNode.

Traceback (most recent call last):
  File "/rds/general/user/ta4218/home/anaconda3/envs/clinicaEnv/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 398, in run
    runtime = self._run_interface(runtime)
  File "/rds/general/user/ta4218/home/anaconda3/envs/clinicaEnv/lib/python3.10/site-packages/nipype/interfaces/utility/wrappers.py", line 142, in _run_interface
    out = function_handle(**args)
  File "<string>", line 12, in init_input_node
  File "/rds/general/user/ta4218/home/anaconda3/envs/clinicaEnv/lib/python3.10/site-packages/clinica/utils/dwi.py", line 467, in check_dwi_volume
    raise IOError(
OSError: Number of DWIs, b-vals and b-vecs mismatch (# DWI = 34, # B-vec = 32, #B-val = 32)

Thanks, Tom

[NicolasGensollen]

Thanks for clarifying a bit @ta4218 !

By looking at the traceback, it seems like the pipeline is crashing right at the beginning because of the dimensions mismatch between the DWI image and the b-vectors and b-values.

You can verify these issues by opening the related files and checking the shapes. Here is an example from data that we use in our CI:

>>> import numpy as np
>>> import nibabel as nib
>>> img = nib.load("sub-PREVDEMALS0010025PG_ses-M000_dwi.nii.gz")
>>> img.shape
(96, 96, 59, 139)
>>> bvals = np.loadtxt("sub-PREVDEMALS0010025PG_ses-M000_dwi.bval")
>>> bvals
>>> bvals.shape
(139,)
>>> bvecs = np.loadtxt("sub-PREVDEMALS0010025PG_ses-M000_dwi.bvec")
>>> bvecs.shape
(3, 139)

I'm wondering whether your input dataset contains other subjects or sessions on which you could try running this to see if you get the same problem. It might just be an issue with this specific subject / session.

[ta4218]

I have another example here. I'm not able to confirm at the moment how many more there are. It is peculiar that the shape is 3d and not 4d for the dwi img.

>>> import numpy as np
>>> import nibabel as nib
>>> bval = "/rds/general/ephemeral/user/ta4218/ephemeral/ADNI234_2/adni_bids/sub-ADNI130S4817/ses-M108/dwi/sub-ADNI130S4817_ses-M108_dwi.bval"
>>> bvec = "/rds/general/ephemeral/user/ta4218/ephemeral/ADNI234_2/adni_bids/sub-ADNI130S4817/ses-M108/dwi/sub-ADNI130S4817_ses-M108_dwi.bvec"
>>> dwi = "/rds/general/ephemeral/user/ta4218/ephemeral/ADNI234_2/adni_bids/sub-ADNI130S4817/ses-M108/dwi/sub-ADNI130S4817_ses-M108_dwi.nii.gz"
>>> img = nib.load(dwi)
>>> print(img.shape)
(128, 128, 2711)
>>> bvals = np.loadtxt(bval)
>>> print(bvals.shape)
(33,)
>>> bvecs = np.loadtxt(bvec)
>>> print(bvecs.shape)
(3, 33)

[NicolasGensollen]

@ta4218 Thanks for looking into it. I'd be interested to know if you see other shape mismatch cases like these (and/or cases where things are correct which would be a bit reassuring...). It could be an issue in the ADNI2BIDS converter since these images were outputted by this pipeline. I will need to take a closer look.

[ta4218]

@NicolasGensollen Thanks for your help.

Don't worry, I have gone through a few that are working OK; my initial investigations suggest that this is a subject-specific issue.

Thanks, Tom

[github-actions[bot]]

This issue is considered stale because it has not received further activity for the last 14 days. You may remove the inactive label or add a comment, otherwise it will be closed after the next 14 days.

[github-actions[bot]]

This issue is considered stale because it has not received further activity for the last 14 days. You may remove the inactive label or add a comment, otherwise it will be closed after the next 14 days.