FIX: Conditionally drop derived volumes from DWI sequences

[effigies]

795 cast a wide net, based on Phillips DICOMs that concatenated derived volumes to diffusion series. This causes problems for Siemens sequences where only derived volumes are contained in an image.

This is a pretty minimal fix that changes the rule to "if raw and derived are present, drop derived, otherwise keep whatever we have".

Fixes #1245.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 92.26%. Comparing base (27c2427) to head (3f81a96).

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #1296 +/- ## ======================================= Coverage 92.25% 92.26% ======================================= Files 99 99 Lines 12467 12469 +2 Branches 2563 2564 +1 ======================================= + Hits 11502 11505 +3 Misses 642 642 + Partials 323 322 -1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[effigies]

@yarikoptic @mgxd Would either of you care to have a look?

[effigies]

That happens a couple lines below.

https://github.com/nipy/nibabel/blob/3f81a96b61106d218da51c0453de23c4e6669bf6/nibabel/nicom/dicomwrappers.py#L526-L530

[yarikoptic]

this DICOM handling to just figure out the shape of the data is somewhat of a dark magic for me, so can't really assess on validity. But what I can say that

if running with stock nibabel 5.2.0-3 and pydicom installed on debian I get 4 hits on that addressed bug and 1 other

```shell ❯ /usr/bin/find dcm_* -type d | grep -v '\.git/' | while read d; do dcm=$(/bin/ls $d/*.dcm 2>/dev/null| head -n 1 || echo ); [ -z "$dcm" ] || chronic python -W ignore::UserWarning -c 'import sys; f=sys.argv[1]; print(f, end=" "); from nibabel.nicom import dicomwrappers as didw; print(didw.wrapper_from_file(f).image_shape)' $dcm; done; dcm_qas/dcm_qa_xa30/In/11_DWI_dir80_PA/0001_1.3.12.2.1107.5.2.43.67093.2022071112123417719608653.dcm Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3/dist-packages/nibabel/onetime.py", line 156, in __get__ val = self.getter(obj) ^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/nibabel/nicom/dicomwrappers.py", line 545, in image_shape frame_indices = np.delete(frame_indices, stackid_dim_idx, axis=1) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<__array_function__ internals>", line 200, in delete File "/usr/lib/python3/dist-packages/numpy/lib/function_base.py", line 5136, in delete axis = normalize_axis_index(axis, ndim) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ numpy.AxisError: axis 1 is out of bounds for array of dimension 1 dcm_qas/dcm_qa_xa30/In/12_DWI_dir80_PA/0001_1.3.12.2.1107.5.2.43.67093.2022071112123419375008659.dcm Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3/dist-packages/nibabel/onetime.py", line 156, in __get__ val = self.getter(obj) ^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/nibabel/nicom/dicomwrappers.py", line 545, in image_shape frame_indices = np.delete(frame_indices, stackid_dim_idx, axis=1) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<__array_function__ internals>", line 200, in delete File "/usr/lib/python3/dist-packages/numpy/lib/function_base.py", line 5136, in delete axis = normalize_axis_index(axis, ndim) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ numpy.AxisError: axis 1 is out of bounds for array of dimension 1 dcm_qas/dcm_qa_xa30/In/19_DWI_dir80_AP/0001_1.3.12.2.1107.5.2.43.67093.2022071112140609850612301.dcm Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3/dist-packages/nibabel/onetime.py", line 156, in __get__ val = self.getter(obj) ^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/nibabel/nicom/dicomwrappers.py", line 545, in image_shape frame_indices = np.delete(frame_indices, stackid_dim_idx, axis=1) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<__array_function__ internals>", line 200, in delete File "/usr/lib/python3/dist-packages/numpy/lib/function_base.py", line 5136, in delete axis = normalize_axis_index(axis, ndim) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ numpy.AxisError: axis 1 is out of bounds for array of dimension 1 dcm_qas/dcm_qa_xa30/In/20_DWI_dir80_AP/0001_1.3.12.2.1107.5.2.43.67093.2022071112140611403312307.dcm Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3/dist-packages/nibabel/onetime.py", line 156, in __get__ val = self.getter(obj) ^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/nibabel/nicom/dicomwrappers.py", line 545, in image_shape frame_indices = np.delete(frame_indices, stackid_dim_idx, axis=1) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<__array_function__ internals>", line 200, in delete File "/usr/lib/python3/dist-packages/numpy/lib/function_base.py", line 5136, in delete axis = normalize_axis_index(axis, ndim) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ numpy.AxisError: axis 1 is out of bounds for array of dimension 1 dcm_qas/dcm_qa_fmap/In/Philips/fmap/IM_0027_fMAP.dcm Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3/dist-packages/nibabel/onetime.py", line 156, in __get__ val = self.getter(obj) ^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/nibabel/nicom/dicomwrappers.py", line 566, in image_shape raise WrapperError('Calculated shape does not match number of frames.') nibabel.nicom.dicomwrappers.WrapperError: Calculated shape does not match number of frames. ```

with this patch (with post-installed pydicom) I get only that other `Calculated shape does not match number of frames.

❯ /usr/bin/find dcm_* -type d | grep -v '\.git/' | while read d; do dcm=$(/bin/ls $d/*.dcm 2>/dev/null|  head -n 1 || echo ); [ -z "$dcm" ] || chronic  /home/yoh/proj/nipy/nipy-suite/nibabel/venvs/dev3/bin/python -W ignore::UserWarning -c 'import sys; f=sys.argv[1]; print(f, end=" "); from nibabel.nicom import dicomwrappers as didw; print(didw.wrapper_from_file(f).image_shape)' $dcm; done;
dcm_qas/dcm_qa_fmap/In/Philips/fmap/IM_0027_fMAP.dcm Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/yoh/proj/nipy/nipy-suite/nibabel/venvs/dev3/lib/python3.11/site-packages/nibabel/onetime.py", line 156, in __get__
    val = self.getter(obj)
          ^^^^^^^^^^^^^^^^
  File "/home/yoh/proj/nipy/nipy-suite/nibabel/venvs/dev3/lib/python3.11/site-packages/nibabel/nicom/dicomwrappers.py", line 569, in image_shape
    raise WrapperError('Calculated shape does not match number of frames.')
nibabel.nicom.dicomwrappers.WrapperError: Calculated shape does not match number of frames.

so, not sure if returned dimensionality is right, but it is definitely not hitting the error any more.

[yarikoptic]

my question would be - how feasible would be to get the release with this out? ;) or should we test more?

[effigies]

Could put out a 5.2.1 with this and other bug fixes. Is the shape mismatch thing something we can address?

[yarikoptic]

on that - added a little to its own

• https://github.com/nipy/nibabel/issues/1211