biigle / maia

:m: BIIGLE module for the Machine Learning Assisted Image Annotation method
GNU General Public License v3.0
2 stars 3 forks source link

Optional support for instance segmentation #97

Open dlangenk opened 2 years ago

dlangenk commented 2 years ago

If we use Mask R-CNN anyway for detection we could support instance segmentation if the data is present. This would be more easy in connection with the data import #95 (for example dino would be able to generate more fine grained annotation candidates instead of the autoencoder) and the framework change #96 (if we change the framework we have to rewrite some code anyway).

mzur commented 1 year ago

Segment Anything could be used to generate segmentation masks based on the MAIA circles as prompts. This would work even without masks as training data (which was the main argument against segmentation because masks are hard to draw manually).

dlangenk commented 1 year ago

I'm not sure if it doesn't work better the other way, so use SAM to create more or less accurate training data and create accurate prediction masks. In addition the compute load is much less as usually #trainingData << #predictedData

dlangenk commented 1 year ago

A student of mine has developed some heuristics to improve the conversion of a point in his case to a polygon. But I guess most of these are also applicable to circles.