search

jozhang97 / DETA

Detection Transformers with Assignment
Apache License 2.0
241 stars 20 forks source link

Is it possible to use IoU assignment only in the final assignment? #7

Closed xiaomingwang80 closed 1 year ago

xiaomingwang80 commented 1 year ago

Hi,

Many thanks for your novel and interesting work.

In the paper, all experiments follow the two-stage DETR framework(which means the input query of decoder is the first-stage proposal). Have you ever tried vanilla DETR framework(the input query of decoder is learnable) with IoU assignment？

To be more general, can we just relax the one-to-one matching constraint and instead allow one-to-many assignments in the final assignment?

jozhang97 commented 1 year ago

Thanks for your interest!

It is not clear to us how to use IoU assignments in vanilla DETR. IoU assignment assigns positive/negative labels to queries by computing the IoU between the ground truth box and proposal box (NOT the decoded output box as in Hungarian matching). It is not clear what to use for vanilla DETR framework.

However, there are other ways to perform one-to-many assignments. One way is to use the B-matching (Hungarian with multiple positive per GT, see Table 6) and apply NMS.

xiaomingwang80 commented 1 year ago

Got it! Thanks for your reply!