[3.9-3D] Enable 3D Visualization of Label Maps: Convert and render label map segmentations as 3D surfaces

sedghi commented 6 months ago

What feature or change would you like to see made?

Implement the ability to convert label map segmentations into 3D surface representations
Leverage existing conversion utilities or algorithms to facilitate the transformation of label maps into 3D surfaces
Integrate the rendering of these 3D surface representations within OHIF's 3D viewport
Explore potential visualization enhancements, such as surface coloring, transparency, or lighting options, for improved 3D rendering of label maps

Why should we prioritize this feature?

Three-dimensional visualization of label maps can provide valuable spatial context and improved understanding of segmented structures, particularly in medical imaging applications involving complex anatomical structures or pathologies. By enabling 3D rendering of label maps as surfaces, OHIF can offer users a more intuitive and immersive experience when working with segmentation data.

robtoth commented 6 months ago

This seems like a great feature!

This type of surface rendering seems already available in cornerstone3d:

Cornerstone Demo: https://www.cornerstonejs.org/live-examples/surfacerendering

Screenshot_20240520_100022

Code in cornerstone that convert labelmap to surface (vtk polydata): https://github.com/cornerstonejs/cornerstone3D/blob/2c4efae4c977bf6df9ca7399b6c6e715ab3f899d/packages/tools/src/stateManagement/segmentation/polySeg/Surface/surfaceComputationStrategies.ts#L81

It seems to use the polyseg implementation of the Flying Edges algorithm: https://github.com/cornerstonejs/cornerstone3D/blob/2c4efae4c977bf6df9ca7399b6c6e715ab3f899d/packages/tools/src/stateManagement/segmentation/polySeg/Surface/convertContourToSurface.ts#L44

Specifically calling ICRPolySeg functionality: https://github.com/cornerstonejs/cornerstone3D/blob/2c4efae4c977bf6df9ca7399b6c6e715ab3f899d/packages/tools/src/workers/polySegConverters.js#L56

from the polyseg-wasm library: https://github.com/cornerstonejs/cornerstone3D/blob/2c4efae4c977bf6df9ca7399b6c6e715ab3f899d/packages/tools/src/workers/polySegConverters.js#L4

This uses vtkDiscreteFlyingEdges3D (a faster implementation of the Marching Cubes algorithm) under the hood via the convertLabelmapToSurface function:

https://bitbucket.org/icrimaginginformatics/polyseg-wasm/src/0fdf4f40e089f57e5ff9927a099a58465891ea90/src/ICRPolySeg.cxx#lines-384

OHIF / Viewers

[3.9-3D] Enable 3D Visualization of Label Maps: Convert and render label map segmentations as 3D surfaces #4109

What feature or change would you like to see made?

Why should we prioritize this feature?