search

Shared-Reality-Lab / IMAGE-server

IMAGE project server components
Other
2 stars 7 forks source link

Semantic Segmentation: investigate segment anything and other newer alternatives #740

Open jeffbl opened 1 year ago

jeffbl commented 1 year ago

Things seem to be moving very quickly with semantic segmentation. @rbazin notes in slack that Segment Anything is now available:

The project from MMSegmentation to implement Segment Anything from Meta has started (you can take a look at sam_image_demo.ipynb and sam/sam_inferencer.py) to know more about what is available). They already made an inference class that can be used to segment images, but as SA has very different capacities than classic segmentation models (notably, it's more interactive), a lot of testing will be needed to assess if we can use it (or how we can use it) in our current pipeline.

Task is to investigate whether this would offer a significant advantage over the current semantic segmentation, and how we'd go about integrating it. By the time someone picks this up, there might also be additional models released, and should be taken into account as well.

AndyBaiMQC commented 1 month ago

Do we have any benchmarking done using the current SS? Would be cool ot have them in place already but otherwise we could do it ourselves as well.

I'll look into SA and its new developments in the meantime.

jeffbl commented 1 month ago

Romain did some comparisons over a year ago. Link to his report.

Note that SAMv2 was just released earlier this week.