search

aim-uofa / Matcher

[ICLR'24] Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching
https://arxiv.org/abs/2305.13310
Other
449 stars 26 forks source link

Semantic segment #4

Closed zzzyzh closed 1 year ago

zzzyzh commented 1 year ago

Thank you for your outstanding work!

The paper mentions that you use SAM as the class-agnostic segmentation model, does this mean that Matcher does not have the ability to recognize semantic information while segmenting?

In the meantime, I'm curious as to when the source code will be released.

Your excellent will be a great help to my research!

yangliu96 commented 1 year ago

Thanks for your interest.

Matcher is a one-shot segmentation framework based on the vision foundation models (DINOv2 and SAM), and the semantic information of the segment results is consistent with the semantic categories provided by the reference. This is similar to in-context learning used by SegGPT (https://github.com/baaivision/Painter/tree/main/SegGPT). The all-purposed visual features extraction model, DINOv2, makes up for SAM's lack of semantic information.

We are organizing the source code. It will be very soon. Please stay tuned.:)