search

nipy / heudiconv

Flexible DICOM conversion into structured directory layouts
https://heudiconv.readthedocs.io
Other
228 stars 123 forks source link

heudiconv crashes when converting a derived ADC map from Siemens scanner #633

Open neurorepro opened 1 year ago

neurorepro commented 1 year ago

Summary

heudiconv crashes when converting a derived ADC map from Siemens scanner with the following trace:

INFO: Running heudiconv version 0.11.6 latest 0.11.6
INFO: Need to process 1 study sessions
INFO: PROCESSING STARTS: {'subject': 'mysubj', 'outdir': '/mnt/projects/data/dcm_bids/bug_dwi/bids_mysubj/', 'session': '01'}
INFO: Processing 7 dicoms
INFO: Analyzing 7 dicoms
Traceback (most recent call last):
  File "/opt/conda/shared/envs/ni38/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/shared/envs/ni38/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/me/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/home/me/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/home/me/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/home/me/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/home/me/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/me/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/mnt/projects/bids/golestani_data/test_numpy_error/test_heudiconv.py", line 20, in <module>
    runner(args)
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/heudiconv/cli/run.py", line 24, in main
    workflow(**kwargs)
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/heudiconv/main.py", line 379, in workflow
    prep_conversion(sid,
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/heudiconv/convert.py", line 172, in prep_conversion
    seqinfo = group_dicoms_into_seqinfos(
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/heudiconv/dicoms.py", line 202, in group_dicoms_into_seqinfos
    mwinfo = validate_dicom(filename, dcmfilter)
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/heudiconv/dicoms.py", line 106, in validate_dicom
    del mw.series_signature[sig]
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/nibabel/onetime.py", line 142, in __get__
    val = self.getter(obj)
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/nibabel/nicom/dicomwrappers.py", line 642, in series_signature
    signature['image_shape'] = (self.image_shape, eq)
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/nibabel/onetime.py", line 142, in __get__
    val = self.getter(obj)
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/nibabel/nicom/dicomwrappers.py", line 516, in image_shape
    first_frame = self.frames[0]
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/pydicom/multival.py", line 102, in __getitem__
    return self._list[index]
IndexError: list index out of range

This is generated by frame_indices = np.delete(frame_indices, stackid_dim_idx, axis=1) in the if-block:

if stackid_tag in dim_seq:
    stackid_dim_idx = dim_seq.index(stackid_tag)
    frame_indices = np.delete(frame_indices, stackid_dim_idx, axis=1)
    dim_seq.pop(stackid_dim_idx)

Looks like this is because frame_indices is empty (so no second axis), which should be expected for a derived map though, since earlier there is:

frame_indices = np.array(
    [frame.FrameContentSequence[0].DimensionIndexValues
     for frame in self.frames])

with

self.frames = pydicom.Sequence(
    frame for frame in self.frames if
    frame.MRDiffusionSequence[0].DiffusionDirectionality
    != 'ISOTROPIC'
    )

So for a derived map self.frames will become empty, so frame_indices will be so as well, and so the if-block will bring hell.

The thing i don't understand is why we are running the if-block in this case, and why the condition evaluates to true in my case where stackid_tag=2134102 and dim_seq[0] = (0020, 9056). But i have to admit i didn't look into the __int__ function of the dim_seq items.

Just in case this is useful, here is the metadata for this single problematic dcm file:

{
    "Modality": "MR",
    "MagneticFieldStrength": 3,
    "ImagingFrequency": 123.226,
    "Manufacturer": "Siemens",
    "ManufacturersModelName": "MAGNETOM Prisma",
    "InstitutionName": "Neurocenter",
    "InstitutionAddress": "Brain avenue",
    "DeviceSerialNumber": "42",
    "StationName": "AWP42",
    "BodyPartExamined": "BRAIN",
    "PatientPosition": "HFS",
    "ProcedureStepDescription": "Not trying to crash",
    "SoftwareVersions": "syngo MR XA30",
    "MRAcquisitionType": "2D",
    "SeriesDescription": "dwi_acq-B2850x1D5MMISO_afid-beta_ADC",
    "ProtocolName": "dwi_acq-B2850x1D5MMISO_afid-beta",
    "ScanningSequence": "EP",
    "SequenceVariant": "SK\\SP",
    "ScanOptions": "PFP\\FS",
    "PulseSequenceName": "*epse2d1_150",
    "ImageType": ["DERIVED", "PRIMARY", "DIFFUSION", "ADC"],
    "ImageTypeText": ["DERIVED", "PRIMARY", "DIFFUSION", "ADC", "ND"],
    "NonlinearGradientCorrection": false,
    "RawImage": false,
    "SeriesNumber": 51,
    "AcquisitionTime": "17:11:9.695000",
    "SliceThickness": 1.5,
    "SpacingBetweenSlices": 1.5,
    "EchoTime": 0.074,
    "RepetitionTime": 6.7,
    "MTState": false,
    "FlipAngle": 90,
    "PartialFourier": 0.875,
    "BaseResolution": 150,
    "ShimSetting": [
        -1273,
        -11693,
        6264,
        288,
        261,
        -78,
        -194,
        57  ],
    "TxRefAmp": 233.778,
    "PhaseResolution": 1,
    "ReceiveCoilName": "HeadNeck_64",
    "ReceiveCoilActiveElements": "HC1-6",
    "CoilString": "HeadNeck_64",
    "PulseSequenceDetails": "%SiemensSeq%\\ep2d_diff",
    "RefLinesPE": 45,
    "CoilCombinationMethod": "Adaptive Combine",
    "MultibandAccelerationFactor": 2,
    "PercentPhaseFOV": 100,
    "PercentSampling": 100,
    "EchoTrainLength": 44,
    "PartialFourierDirection": "PHASE",
    "AcquisitionMatrixPE": 150,
    "ReconMatrixPE": 150,
    "BandwidthPerPixelPhaseEncode": 26.316,
    "ParallelReductionFactorInPlane": 3,
    "ParallelAcquisitionTechnique": "SMS",
    "EffectiveEchoSpacing": 0.000253331,
    "DerivedVendorReportedEchoSpacing": 0.000759994,
    "TotalReadoutTime": 0.0377464,
    "PixelBandwidth": 1515,
    "DwellTime": 2.2e-06,
    "PhaseEncodingDirection": "j-",
    "SliceTiming": [
        6.6225,
        0,
        6.7625,
        0.14,
        6.9025,
        0.2825,
        7.045,
        0.4225,
        7.185,
        0.5625,
        7.325,
        0.705,
        7.4675,
        0.845,
        7.6075,
        0.9875,
        7.7475,
        1.1275,
        7.89,
        1.2675,
        8.03,
        1.41,
        8.17,
        1.55,
        8.3125,
        1.69,
        8.4525,
        1.8325,
        8.595,
        1.9725,
        8.735,
        2.1125,
        8.875,
        2.255,
        9.0175,
        2.395,
        9.1575,
        2.535,
        9.2975,
        2.6775,
        9.44,
        2.8175,
        9.58,
        2.9575,
        9.72,
        3.1,
        9.8625,
        3.24,
        10.0025,
        3.3825,
        10.1425,
        3.5225,
        10.285,
        3.6625,
        10.425,
        3.805,
        10.565,
        3.945,
        10.7075,
        4.085,
        10.8475,
        4.2275,
        10.99,
        4.3675,
        11.13,
        4.5075,
        11.27,
        4.65,
        11.4125,
        4.79,
        11.5525,
        4.93,
        11.6925,
        5.0725,
        11.835,
        5.2125,
        11.975,
        5.3525,
        12.115,
        5.495,
        12.2575,
        5.635,
        12.3975,
        5.7775,
        12.5375,
        5.9175,
        12.68,
        6.0575,
        12.82,
        6.2,
        12.96,
        6.34,
        13.1025,
        6.48    ],
    "ImageOrientationPatientDICOM": [
        1,
        0,
        0,
        0,
        1,
        0   ],
    "InPlanePhaseEncodingDirectionDICOM": "COL",
    "ConversionSoftware": "dcm2niix",
    "ConversionSoftwareVersion": "v1.0.20220720"
}

Platform details:

Choose one:

neurorepro commented 1 year ago

@yarikoptic I got a new error with #634 . This definitely counts as progress:

INFO: Running heudiconv version 0.11.6.post31+g24f4d88 latest 0.11.6
INFO: Need to process 1 study sessions
INFO: PROCESSING STARTS: {'subject': 'mysubj', 'outdir': '/mnt/projects/data/dcm_bids/bids_dwi_min_newheudi/', 'session': '01'}
INFO: Processing 7 dicoms
INFO: Analyzing 7 dicoms
/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/nibabel/nicom/dicomwrappers.py:525: UserWarning: Derived images found and removed
  warnings.warn('Derived images found and removed')
Traceback (most recent call last):
  File "/home/michael/.conda/envs/heudiconv310_dwi/bin/heudiconv", line 8, in <module>
    sys.exit(main())
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/heudiconv/cli/run.py", line 24, in main
    workflow(**kwargs)
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/heudiconv/main.py", line 379, in workflow
    prep_conversion(sid,
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/heudiconv/convert.py", line 172, in prep_conversion
    seqinfo = group_dicoms_into_seqinfos(
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/heudiconv/dicoms.py", line 232, in group_dicoms_into_seqinfos
    if mw.is_same_series(mwgroup[idx]):
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/nibabel/nicom/dicomwrappers.py", line 357, in is_same_series
    my_sig = self.series_signature
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/nibabel/onetime.py", line 142, in __get__
    val = self.getter(obj)
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/nibabel/nicom/dicomwrappers.py", line 625, in series_signature
    signature['image_shape'] = (self.image_shape, eq)
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/nibabel/onetime.py", line 142, in __get__
    val = self.getter(obj)
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/nibabel/nicom/dicomwrappers.py", line 505, in image_shape
    first_frame = self.frames[0]
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/pydicom/multival.py", line 104, in __getitem__
    return self._list[index]
IndexError: list index out of range
neurorepro commented 1 year ago

@yarikoptic looking at the conversion of other metrics with version 0.11.6 of heudiconv (not the PR), i also got the error for TRACEW, but not for FA and ColFA (which stands for something like colored FA).

Serendipitously i noticed that the third dimension of ColFA is not properly retrieved by heudiconv which shows dim3 to be 1 instead of 94. This may be due to the image datatype (RGB24) and Siemens colored FA volumes are probably not on top of heudiconv backlog.

yarikoptic commented 1 year ago

given that we get through nibabel/nicom/dicomwrappers.py", line 357, in is_same_series -- the issue might even be in nibabel , and worth checking/filing there.

yarikoptic commented 11 months ago

also happens upon the same del mw.series_signature[sig] for

yarikoptic commented 4 months ago

@neurorepro could you try with current fresh nibabel? current heudiconv image has it: https://github.com/nipy/heudiconv/issues/670#issuecomment-1969286655