search

Slicer / Slicer

Multi-platform, free open source software for visualization and image computing.
https://www.slicer.org
Other
1.62k stars 539 forks source link

Segmentation closed surface representation generation fails for float voxels #6941

Open lassoan opened 1 year ago

lassoan commented 1 year ago

Summary

When a seg.nrrd file uses type: float voxels then closed surface representation is not generated (nothing appears or only a single segment's 3D representation appears).

See full error report here: https://discourse.slicer.org/t/only-one-of-28-segments-is-showing-in-3d/28934/3

Steps to reproduce

Expected behavior

All segments should appear in 3D

Environment

lassoan commented 1 year ago

Using float voxel type for segmentation does not make much sense (it uses more memory and there are potential issues with comparison of floating-point values), so we don't need to support this. However, we should then display a specific error message to the user when he attempts to load such image and when float/double image is attempted to be used programmatically then we should log an error.

pieper commented 1 year ago

Perhaps if someone tries to load float as segmentation we should offer the option of automatically rounding it and casting to the best corresponding datatype based on the data range. On the other hand I don't think this comes up very often and it's not worth investing much effort.

lassoan commented 1 year ago

Good point, logging a warning and automatically casting would be probably a better solution (need to check the scalar range to choose the correct integer type and the image may need to be rejected if does not fit into any integer range).

lassoan commented 6 months ago

The situation is even worse for ARM macOS systems, as the ARM processors make the application crash with some floating-point exception when working with a segmentation with float/double value.

@Sunderlandkyl do you think you could implement a fix (log a warning and convert the image into the most suitable integer type; return with an error if it is not possible to convert) in the next few weeks?

Sunderlandkyl commented 6 months ago

Sure, I can do that. Still not sure how those segmentations are being generated.

lassoan commented 6 months ago

This is getting more and more common. See for example: https://discourse.slicer.org/t/segmentation-issue-invisible-labels/34460/2

It seems that problems come from that researchers use an AI model to compute the segmentation. The inference result is float, then they use thresholding or sigmoid function to binarize it. But then they forget to cast the result to an integer type in the end.

Sunderlandkyl commented 6 months ago

I have a function that will convert any vtkImageData to the smallest possible data type that will include the entire scalar range, excluding scalar point representations.

I have updated vtkMRMLSegmentationStorageNode to automatically log a warning and cast the image when a floating point segmentation file is read.

At what other points do we want to trigger an automatic warning + conversion of the image?

What about in the (unlikely) case that a floating point image is detected, but the values are outside the range of representable integer values?

pieper commented 6 months ago

I would say any image with non-positive non-integer values or any above what can be represented by our largest integer type should be rejected with an error message indicating that the data is not compatible with a segmentation (ideally with some hints about what is wrong). This should happen any time data is presented as a possible segment/segmentation, like reading, importing, etc.