search

IDEA-Research / GroundingDINO

[ECCV 2024] Official implementation of the paper "Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection"
https://arxiv.org/abs/2303.05499
Apache License 2.0
6.72k stars 681 forks source link

About referring expression grounding #44

Open pengzhiliang opened 1 year ago

pengzhiliang commented 1 year ago

Thanks for the great work!

When I try the grounding demo file, I find a sentence (eg. a man in blue coat) will be splited into multiple phrases (eg. a man, blue coat) and the model will predict multiple boxes correspondding to the phrase. In fact, the model is expected to generate just one box (a man), just like the Referring Expression Grounding task. It seems this model can't handle this task. Or do I need to adjust some hyper-parameters?

But If this model cuts the given sentence into multiple phrases, how it is tested on RefCOCO dataset and achieves impressive performance.

I'm little confused about it.

Thanks!

SlongLiu commented 1 year ago

We select the box with the highest text similarities as top-1 outputs. The model inputs and outputs can be found here: https://github.com/IDEA-Research/GroundingDINO#star-explanationtips-for-grounding-dino-inputs-and-outputs . I add a paragraph to explain the model.

drilistbox commented 1 year ago

Thanks for your great work, but I am confusing about how to detect "the left dog".