The functional target and report?

[Everloom-129]

I see a baseline cascade connection of gDINO and SAM. But the project is lack of internal logical reasoning to combine them. I will do a series of test on another similar project, and open-source it later.

[Everloom-129]

Thanks for sharing the idea online, it is pity that we lack official document

[mhyeonsoo]

Hi @Everloom-129 ! Thanks for your feedback. I should have written a more detailed information of why I use / combine two methods together.

The main repository I referred to make this task is, https://github.com/IDEA-Research/Grounded-Segment-Anything. In here, it basically uses SAM model and grounding DINO models together to get the segmentation of any images based on the user's query.

Normally we have been facing the trouble labeling the dataset we have, so we just use open dataset or very few of dataset we take.

To solve that issue, I wanted the model to automatically annotate the images that I insert. --> For this, I used SAM model.

But SAM model requires information of 'which area to segment' such as user-defined pixel coordinates or bounding boxes. For this, I used grounding DINO to make the bounding box outputs which will be the input of SAM model.

You can see the detail in the original papers!

• grounding DINO: https://arxiv.org/abs/2303.05499
• SAM: https://ai.facebook.com/research/publications/segment-anything/