Bulk Annotation Mismatch in shapes

[garhiaNfer]

HI Markus,

I am trying to annotate certain regions with RECTANGLE, POLYGON etc. in the viewer on the right side it's clearly showing the Graphic Type of annotation as RECTANGLE. But on the biopsy (panorama) it's coming out as POINT. Not able to see the exact geometrical shape on the panorama.

As of now only POINT (geometrical shape) is working fine and coming out as correct for us.

I have tried generating these annotations with highdicom lib, but still facing the same issue.

[hackermd]

@garhiaNfer thank you for your interest in Slim and for reporting the issue. Unfortunately, I do not fully understand the problem you are experiencing. Could you kindly elaborate on what you mean by "biopsy (panorama)" and describe in more detail what you tried to do? Could you share any screenshots (ideally using a public data set from an IDC collection) that would allow us to reproduce the issue?

[garhiaNfer]

@hackermd from panorama or biopsy, I meant the the digitised slide, basically the images we are viewing on the viewer. What I have tried is to create one rectangle and polygon using Bulk Annotations. The problem which I am facing right now is it's not showing up on the viewport. But when I change that annotation type from rectangle or polygon to point it's showing up.

Also, what I am seeing now is on the right in Annotation Group section it's clearly saying the Graphic Type is Rectangle, but on the viewport it's drawing just a point and when I am clicking on the annotation it's showing up as POINT.

Attaching the screenshot regarding the same.

[hackermd]

Ah..I see. Thank you for the clarification.

For complex geometries (RECTANGLE, POLYGON, etc.), we compute the centroid of the geometry and render the individual point. So, the fact that only points are displayed is expected (but undocumented and potentially surprising) behavior. That the graphic type POINT is displayed in the overlay is a bug. We will need to make sure that the original graphic type is propagated and displayed correctly.

[garhiaNfer]

@hackermd just curious why do you calculate the centroid for complex geometries and then render the individual points?

[fedorov]

@garhiaNfer we need to have this issue resolved for IDC as well, and will try to find resources to address it. But I cannot give any timeline at this point.

[igoroctaviano]

@garhiaNfer can you share your SLIM config with the annotations + steps to reproduce? I'll take a look at it asap.

[fedorov]

@igoroctaviano sorry it took us a while to get back to you. There are many unknowns and limitations we had to sort out (you can see some notes here, if interested: https://projectweek.na-mic.org/PW39_2023_Montreal/Projects/TutorialsOnWorkingWithDicomAnnotationsInPathologyWholeSlideImages/).

Here is the sample dataset that was put together by @DanielaSchacherer and @CPBridge:

• ANN: https://console.cloud.google.com/storage/browser/pw39-2d-3d-point-polygon-dicom-annotations/bulk_ann_display_tests/3d_polygon/gbm_polygon
• corresponding SM: https://console.cloud.google.com/storage/browser/pw39-2d-3d-point-polygon-dicom-annotations/bulk_ann_display_tests/original_wsi

The ANN above contains 3D polygon annotations, which we would like to see rendered as polygons in Slim. Currently, those are not rendered at all. Please note that you will need to use dcm4chee backend for this development, since it is a known limitation of Google Healthcare API to not support bulkdata elements, which is needed to support this functionality.

[fedorov]

The ANN above contains 3D polygon annotations, which we would like to see rendered as polygons in Slim. Currently, those are not rendered at all.

@DanielaSchacherer: Igor is reporting that those are indeed rendered as points. Can you please confirm you are observing the same? I do not have the setup to test this myself.

[DanielaSchacherer]

Sorry for the late reply. I am sorry, I can not confirm. I am still only able to visualize 3D point annotations, but not 3D polygon annotations.

[igoroctaviano]

Update: I updated dicom-microscopy-viewer to render the full polygons:

The next step now is to restrict that to high zoom levels and only render the ones that are in the current view (that may require changing the OpenLayers data loader logic). The default will be rendering points at lower zoom levels because the individual edges and vertices would clutter the view and become too much for the browser

[hackermd]

@igoroctaviano that's awesome! Thank you for your efforts.

To @garhiaNfer's point, I suggest ultimately always keeping Graphic Type POLYGON, even if the graphic data are rendered as points at lower resolutions for efficiency.

[fedorov]

This is great work @igoroctaviano!

@CPBridge @DanielaSchacherer please see the update above if you have not noticed it already!

[fedorov]

This is being addressed in https://github.com/ImagingDataCommons/slim/pull/170

[fedorov]

Agreed to support visualization of bulk annotations at the highest resolution of the pyramid for now. Will review clusters, but it is fine to just merge the highest res visualization feature.