Segmentations are misplaced in v3.7.0, it worked fine in v3.6.0

[pavel-kaiko]

Describe the Bug

Hello,

We faced the bug when segmentations weren't placed where they should be. We tried to use a different slice order, but it didn't help. In the provided example, there are two segmentations with regular and reversed order.

We create the segments based on the Segmentation class from highdicom library, so if there is a bug on ReferencedSeriesSequence, it would be there. This OHIF issue regarding the misplacement also mentions this library.

Example data: 2023.10.27 - Segmentation alignment issue files

To check that it worked as expected before.

1. Open the Viewer v3.6.0
2. Load the slides and segmentations
3. Open them in Basic Viewer and apply the regular segmentation.

Steps to Reproduce

1. Upload the provided data to OHIF Viewer v 3.7.0+. It can even be https://viewer-dev.ohif.org/local. There are two versions of segmentation in the dicom_seg_cielab folder – regular and reversed.
2. Apply one of the versions in Basic Viewer or Segmentation mode.

The current behavior

The segments will be applied to the improper slides.

The expected behavior

1. Open the Viewer v3.7.0+
2. Load the slides and segmentations
3. Open them in Basic Viewer/Segmentation mode and apply the regular segmentation.
4. Segmentation should be applied properly

Expected result:

OS

macOS 14.1

Node version

v18.18.2

Browser

Version 118.0.5993.117 (Official Build) (arm64)

[Vlad-C-Crisan]

Colleague of @pavel-kaiko here. Following a recommendation and follow-up questions we received from previous office hours, we tried to use THIS SCRIPT for converting the files to DICOM-SEG.

HERE you will find both the input data and the script used for conversion.

To reproduce, download the data locally and run the notebook to recreate the segmentation file. The result can be displayed correctly in v3.6.0, but in v3.7.0 the order is broken.

[03shek]

Thought I'd chime in while I have 3D Slicer open. The example data Pavel provided seems to load properly for "reversed_order" in 3D Slicer. The "normal_order" did not in 3D Slicer. In OHIF v3.7.0+, the SEGs still look wrong indeed for both orders.

1. [LOOKS RIGHT] reversed_order in 3D Slicer:

2. [LOOKS WRONG] reversed_order in OHIF v3.7.0+:

I would've expected OHIF to match the 3D Slicer view

[fedorov]

@CPBridge FYI

You could also try converting using dcmqi: https://github.com/QIICR/dcmqi.

[CPBridge]

Highdicom author here. I am open of course to there being a bug in highdicom though I will say that I find that possibility increasingly unlikely: I have spent a lot of time in that part of the codebase recently and checked the logic many many times.

Couple of thoughts from me:

• because you pass a numpy array with no spatial information, highdicom relies on you to ensure that the order of frames in the segmentation array you pass is such that it corresponds exactly to the order of the source images. It cannot detect if you get this wrong, and so this is the most likely way for a logical error to have crept into the conversion script that could explain this behaviour. Another way to check this is to use highdicom to get the segmentation numpy array out of the DICOM seg again with the get_pixels_by_source_instance method for any arbitrary group of source instances (see here also for more information). If that matches as expected, I think the likelihood of an error in the seg file is very low
• I am a little concerned by the mention of the ReferencedSeriesSequence in one of the comments above. I hope that OHIF does not use the information therein to order the frames. According to my reading of the standard, nothing can be assumed from the ordering of items within that sequence, and instead the information in the PerFramesFunctionalGroupsSequence (such as the DerivationImageSequence within it) must be used

[sedghi]

Adding another point to the awesome explanation by Chris. I remember back in the day, the simpleITK toolings (potentially other toolings too) had the axis reversed. Is this happening for you?

See these

• https://discourse.itk.org/t/why-to-invert-the-order-of-the-indices-between-numpy-and-simpleitk/809
• https://discourse.itk.org/t/importing-image-from-array-and-axis-reorder/1192

@CPBridge

I hope that OHIF does not use the information therein to order the frames.

We don't rely on ReferencedSeriesSequence really, in fact we throw an error if it is not available (shouldn't we? since it is Type 1) But during reading we just use PerFrameFunctionalGroupSequence

[Vlad-C-Crisan]

@sedghi we shared in the above comment a script that creates the DICOM segmentations, based on highdicom. As mentioned in that comment and also illustrated by @03shek

• the segmentations produced look as expected in Slicer

• in OHIF 3.7.0+ we can't load them due to an error:

• after fixing the above error, the segmentations can be displayed, but they look wrong in version 3.7.0+:

• the images look correct in version 3.6.0:

[sedghi]

Seems like the segmentation appears on the first slice and it should be on the lung as we saw in the office hours. So it might not be flipping but matching series for filling in

[pavel-kaiko]

Hi @sedghi. As you recommended, we did git bisect to find where breaking changes were introduced. This is this PR: https://github.com/OHIF/Viewers/pull/3577 ~But what is more interesting is that breaking changes are already present on the Netlify deployment preview: https://deploy-preview-3576--ohif-dev.netlify.app.~

[pavel-kaiko]

One more addition. In v 3.7.0+, segmentation is reversed and starts from the beginning or the end of the study (depending on the order), ignoring the original offset (in comparison to 3.6.0 or Slicer3D)

[sedghi]

I found the issue which is we assumed wrongfully that the DICOM SEG always have all the slices of the referenced series, but it is a wrong assumption. I fixed it but it has some performance issues, which i will try to address and create a PR

[pavel-kaiko]

Thank you, @sedghi, for resolving the issue and for your excellent work. I upgraded the Cornerstone library, and it is now functioning as expected.