Wrong Segmentation Position when opening with CornerstoneTools

[AlexandrineRibeiro]

I'm writing a script to attach AI made lesion segmentations to existing DICOM files (It's my first time that I dive so deep into DICOM, so I might have missed something).

I took an existing DICOM file (of a CT scan) that contains both a scan and a segmentation, converted the segmentation to np.array, and used is as the mask input, while the scan folder is the source_images.

When I try to load the resulting segmentation with Cornerstone Segmentation Tools, my segmentation appears in the right order but always placed in the first slices. For example, if it is the liver segmentation, it will appear in the first slices and not in the liver area.

My env:

• python 3.6.2
• pydicom 2.3.1
• pydicom-seg 0.3.0
• highdicom 0.20.0

The code I use to store the segmentations:

# Read Image data sets
image_datasets = [dcmread(str(f)) for f in image_files]

segs_descriptions = []
for label in np.unique(y_pred_all):
    if label != 0:
        # Describe the segment
        description_segment = hd.seg.SegmentDescription(
            segment_number=int(label),
            segment_label=f"Segment {int(label)}",
            segmented_property_category=codes.cid7150.Tissue,
            segmented_property_type=codes.cid7166.ConnectiveTissue,
            algorithm_type=hd.seg.SegmentAlgorithmTypeValues.MANUAL,
        )
        segs_descriptions.append(description_segment)

# Create the Segmentation instance
seg_dataset = hd.seg.Segmentation(
    source_images=image_datasets,
    pixel_array=np.uint8(y_pred_all),
    segmentation_type=hd.seg.SegmentationTypeValues.FRACTIONAL,
    segment_descriptions=segs_descriptions,
    series_instance_uid=image_datasets[0].SeriesInstanceUID,
    series_number=99,
    sop_instance_uid=image_datasets[0].SOPInstanceUID,
    instance_number=1,
    max_fractional_value=255,
    transfer_syntax_uid="1.2.840.10008.1.2.5",
    manufacturer="Manufacturer",
    manufacturer_model_name="Model",
    software_versions="v1",
    device_serial_number="Device XYZ",
    omit_empty_frames=False,
)

seg_dataset.save_as(output_dcm)

[CPBridge]

HI @AlexandrineRibeiro thanks for using the library and reporting this issue!

Generally your code looks good but there are a couple of comments unrelated to the issue:

• The sop_instance_uid and series_instance_uid that you pass to the segmentation should not be the uids taken from the input datasets, since the segmentation is a new instance in a new series and therefore needs its own instance/series UIDs. In nearly all cases you just want to generate two new uids, one for each using hd.UID(). Early on we took the decision to make users pass the UIDs themselves in case they want more control over this, which I regret because a lot of people make the same mistake that you do... We should probably improve the docs here.
• If your segments are lesions, "Connective Tissue" is probably not the best code to use to describe them

But to your actual question: I suspect that this is neither an issue with highdicom or your code. There have been issues with the way cornerstone/ohif/dcmjs display segmentations for a while and many people have reported them. We believe that this is a problem in dcmjs, not highdicom. However, it is still worth checking a few things:

• Ensure that before you are creating the segmentation dataset, the order of the source images image_datasets matches the order of slices of the segmentation mask down the 0th dimension of the numpy array.
• Check (using some python visualization such as matplotlib) that if you reconstruct the numpy array using the following method, the mask looks like it should:

reconstructed_mask = seg_dataset.get_pixels_by_source_instance(
    source_sop_instance_uids=[ds.SOPInstanceUID for ds in image_datasets],
    combine_segments=True
)

If it does, then probably everything is fine and the issue is in the viewer, though there is a small chance the bug is in highdicom. Please report this issue to cornerstone or dcmjs, and even better if you can provide example segmentations and screenshots. You may wish to refer to this issue on OHIF (built on cornerstone): https://github.com/OHIF/Viewers/issues/2833

[CPBridge]

P.s. better docs for segs generally are coming soon: you may wish to read the docs on this branch and in particular this file

[pieper]

You can also check your segmentations with Slicer using the Quantitative Reporting extension. If you find that a source image and segmentation pair renders differently in OHIF vs Slicer please post the data and hopefully the difference can be reconciled.

[AlexandrineRibeiro]

@CPBridge @pieper Thank you for your prompt response. I agree with you, the problem appears to be with cornerstone or dcmjs.