search

IDEA-Research / DINO

[ICLR 2023] Official implementation of the paper "DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection"
Apache License 2.0
2.1k stars 230 forks source link

About 'no object' in the labels. #202

Open TsingWei opened 1 year ago

TsingWei commented 1 year ago

In the code of making one-hot targets for class prediction, the no-object class prediction seems not to be supervised: https://github.com/IDEA-Research/DINO/blob/3ffda400b0f1d4a919fbe2a9cf567e79d210ffde/models/dino/dino.py#L362-L374 Especially, https://github.com/IDEA-Research/DINO/blob/3ffda400b0f1d4a919fbe2a9cf567e79d210ffde/models/dino/dino.py#L372 According to this, if a query is assigned to no-object, the predicted logit of it is encouraged to all zeros(after sigmoid).

Just curious, why not explicitly supervise the linear layer to predict no-object class?

JureHudoklin commented 1 year ago

I was wondering the same thing. And another thing i found strange is the dimmension of the class embeding layer. I thought it should be: num_classes + 1 (ie: COCO-> 91 classes + no_obj_clas). The network does not even predict the no object class.

https://github.com/IDEA-Research/DINO/blob/3ffda400b0f1d4a919fbe2a9cf567e79d210ffde/models/dino/dino.py#L132

Did you maybe find any answers to this problem?

Chrazqee commented 11 months ago

Hey guys, do you find the answers?

JureHudoklin commented 11 months ago

@Chrazqee Yea. They do not use no-object class. They just predict all classes as 0 if no-object is present in the box. That is why if you look activation functions on last layer they use sigmoid and softmax. It took me a while to figure it out :P

Chrazqee commented 11 months ago

Hi，@JureHudoklin Thank you for your explanation. It has confused me quite much time. I just know the class 0 in coco dataset represent 'N/A', it not a real class. So thank you for talking me about this. As the activation function of last layer I have not notice yet, since I focused DETR mainly. I'll check it out. I hope your help can help me to solve class unbalance question! Thanks again!

Best!

Chrazqee commented 11 months ago

Hi, @JureHudoklin , I am so sorry to bother you again. At SetCriterion class in DINO, I have not found the softmax activation function. Could you point the position of activation function for me? I'll appreciate if you can help me at your spare time！Thanks a lot!

Best!

TsingWei commented 11 months ago

There is actually no softmax applied in class prediction. To my best knowledge, for both testing and training, they (DETR-like models) just ignore the no object class.