kreshuklab / plant-seg

A tool for cell instance aware segmentation in densely packed 3D volumetric images
https://kreshuklab.github.io/plant-seg/
MIT License
88 stars 31 forks source link

Add widget: remove objects using a probability map #202

Closed qin-yu closed 6 months ago

qin-yu commented 6 months ago

A generic pmap-guided object removal widget

Instead of adding an option to each "boundary -> segmentation" algorithm as discussed in #174, I made a separate widget for Napari GUI pipeline for generic usage. This independent design avoids bundled computation within other functions such as GASP, allows easy trial-and-error strategy, accepts flexible input pmaps/images and threshold, and is useful in unexpected use cases.

I came up with three designs and decided to implement only one. I now add this to Napari, and will think about the best way to do it in the other UI.

  1. Method 1: Threshold foreground to a binary mask and multiply with the segmentation pixel-wise ❌
    1. Pros: very fast; false positive parts will be deleted within objects if foreground quality is high
    2. Cons: depending on the quality of foreground segmentation, details may be lost
  2. Method 2: Delete instances based on the centroids' value in foreground ❌
    1. Pros: more educated guesses than 1, preserves details
    2. Cons: merged instances or merged false-and-true instances will be deleted together; not helpful if instances are not star-convex, e.g. C shaped false positives around objects will persist
  3. Method 3: Delete instances that has, say, 90% overlap with the background (I believe it's the best way ✔)
    1. Pros: considering the foreground probability of each pixel in an instance, the algorithm is well-informed
    2. Cons: slightly slower than 1 & 2 but after careful design and simple tests I believe it handles segmentation/superpixels at a scale of 100k instance in minutes

Example Usage

As a standalone widget it not only helps reduce unnecessary computation but also can be flexibly used between different steps. For example, the following image shows how an instance connected to a false positive would be deleted if we simple use GASP followed by method 3:

image

To have a perfect segmentation, users can apply a method 3 foreground filter before and after GASP like this:

Starting the a watershed output:

image

Apply the new filter:

image

Run GASP:

image

Apply my new filter again:

image

qin-yu commented 6 months ago

It does work

image

qin-yu commented 6 months ago

I think I'm done with the addition of widget, works well:

image

qin-yu commented 6 months ago

A problem of only have foreground filter as a post processing:

image

qin-yu commented 6 months ago

Screenshots

Raw:

image

Foreground prediction:

image

Boundary P\prediction:

image

Watershed superpixels shown on boundary:

image

Watershed superpixels shown on raw:

image

Filtered superpixels on foreground:

image

Filtered superpixels on raw:

image

GASP ran on filtered superpixels:

image

Filtered GASP:

image

qin-yu commented 6 months ago

Hey @lorenzocerrone and @wolny

I added a widget that uses probability maps to filter objects in another image. It is very flexible but only available in Napari now. I believe the other less interactive interfaces don't need this level of flexibility, and I'll think about how to design the filter into the them later. Please have a look at this PR.

wolny commented 5 months ago

this looks great, nice job @qin-yu