Scribble-based segmentation

computational-cell-analytics / micro-sam

Segment Anything for Microscopy

https://computational-cell-analytics.github.io/micro-sam/

MIT License

368 stars 47 forks source link

Scribble-based segmentation #603

Open anwai98 opened 6 months ago

anwai98 commented 6 months ago

This in particular could potentially be a nice feature to segment vessel-like thin (and long) structures using positive scribblles, or eliminate the outside-boundary region in-a-go for parts outside the object using negative scribbles.

constantinpape commented 6 months ago

I agree that this is a good idea. I see two possible ways to implement it:

Derive a mask prompt from the scribbles. This is easy to implement, but I am a bit pessimistic if it will help much. In any case, we can try.
Train a new prompt encoder for it. I think that is in principle more promising, but also more work.

Any other ideas from your end for how to address this?

anwai98 commented 6 months ago

I was thinking in the direction of a simpler version of the first method, however the second one actually sounds more promising.

Also leaving a hint, I think Mehdi has an implementation for scribble-based segmentation with SAM - https://github.com/juglab/napari-sam-labeling-tools

anwai98 commented 1 week ago

I'll add another mention here:

Using weak annotations in the training context (i.e. deriving annotations from weak labels and using them to iteratively see if only weak labels translate well for interactive segmentation without having the pixel-level labels)