Issue with Generating a Single Segmentation Object for Multiple DICOM Series in `highdicom`

[Rishabhpanacea]

According to the DICOM standards, a single segmentation (SEG) object can reference multiple DICOM series. However, the highdicom library currently does not provide a direct method to generate a single segmentation object that refers to multiple series. I attempted to achieve this by combining datasets from two series using the following code:

seg_dataset = hd.seg.Segmentation(
    source_images=image_datasets,
    pixel_array=mask,
    segmentation_type=hd.seg.SegmentationTypeValues.BINARY,
    segment_descriptions=[description_segment_1],
    series_instance_uid=hd.UID(),
    series_number=2,
    sop_instance_uid=hd.UID(),
    instance_number=1,
    manufacturer='Manufacturer',
    manufacturer_model_name='Model',
    software_versions='v1',
    device_serial_number='Device XYZ',
)

Where image_datasets is image_datasets_series1 + image_datasets_series2, and image_datasets_series1 and image_datasets_series2 correspond to the datasets from two different DICOM series.

However, I encountered the following error:

ValueError: Source images must all be part of the same series and must have the same image dimensions (number of rows/columns).

The error originated from the following line in the highdicom library:

if len(uniqueness_criteria) > 1:
    raise ValueError(
        'Source images must all be part of the same series and must '
        'have the same image dimensions (number of rows/columns).'
    )

Relevant DICOM Standard Information

According to the DICOM Standard documentation, specifically Part 3: Information Object Definitions, the Segmentation Object and its attributes are defined. On page 211, Table 10.37-1 "Related Information Entities Macro Attributes" includes the following:

• Attribute Name: Referenced Series Sequence
• Tag: (0008,1115)
• Description: Series that are relevant for the invocation context.
  One or more items are permitted in this sequence.

This indicates that it is permissible in DICOM to reference multiple series within a segmentation object.

Problem and Proposed Solution

The highdicom library currently enforces that all source images for segmentation must belong to the same series and have identical image dimensions. This constraint conflicts with the DICOM standard, which allows a segmentation object to reference multiple series. To resolve this issue, the highdicom library would need to be updated to support the creation of segmentation objects that can reference multiple series, as permitted by the DICOM standard.

[CPBridge]

Hi @Rishabhpanacea, thanks for the feedback.

You are right that Segmentations can reference multiple series. We chose to limit this initially to simplify things. Unfortunately we cannot possibly hope to implement every possible permutation that is valid according to the standard but have to prioritize, and I rather disagree with your phrasing that this "conflicts" with the standard, it just implements a subset of it (as anyone can only ever do).

That said I have always wanted to relax this too because in reality segmentations often reference multiple images that show different tissue characteristics. In fact we had a pull request on it a while ago that got abandoned due to other priorities (and is now probably rather out of date): see #199

Unfortunately it's more complicated than it may initially appear. It's not simply a matter of populating the ReferencedSeriesSequence. This attribute at the top level of the data just lists every instance that is referenced. However in the case that the segmentations are spatially aligned with the source images, the references should also be included on a per-frame basis within the PerFrameFunctionalGroupsSequence (and below that within the DerivationImageSequence), and this in turn requires various checks on the geometry of all source images passed in by the user. Also somewhat tricky is how to have users pass multiple images in a backwards compatible way that also conveys which image frames are aligned with which segmentation frames. Previously we settled on allowing a list of datasets (as currently) or a list of list of datasets (as a new feature).

As there is clearly demand, I will try to implement this in release 0.24.0 (the release after next). There are some branches under development right now that include a very significant refactor of the segmentation codebase to reduce code duplication with other objects (such as ParametricMap) and it doesn't make sense to work on this until that is complete.

In the meantime, as a workaround, if you just want to convey that the further source images were used to create the segmentation, without establishing any frame-level correspondences, you can fairly easily just add items to the references series sequence yourself after construction.

[CPBridge]

Having thought about this a bit further, I think that we will ultimately need an ability to pass further images via a new parameter, so I have added a further_source_images parameter to the Segmentation constructor in #304. Images passed here will not for now be included as source images within the per frame functional groups sequence. I will leave this issue open until that can be implemented at a later date.