Sense-X / Co-DETR

[ICCV 2023] DETRs with Collaborative Hybrid Assignments Training
MIT License
1.02k stars 113 forks source link

About the number of positive queries in the auxiliary heads #1

Closed JCZ404 closed 1 year ago

JCZ404 commented 1 year ago

Hi, Thanks for your great work! I'm confused about the number of positive queries in the auxiliary heads. In your paper, you proposed to regard all the proposals assigned by the auxiliary head with one-to-many assignments as positive queries, and there were about 18.7 positive samples for the Faster-RCNN style assignment and 8.8 positive samples for the ATSS style assignment. But how did you deal with these various positive query numbers when you input these queries into the decoder? did you just pad these queries to the maximum positive query number within a batch? Hope you can help me with this little problem. Thanks in advance!

TempleX98 commented 1 year ago

Thanks for your interest. In fact, we have tried two different implementations for our customized positive queries generation:

  1. Just as you said, we pad these queries to the maximum positive query number within a batch. Besides, we introduce group-wise attention mask to avoid information leakage.
  2. Individually process positive queries from different heads. For example, we perform two forward passes through the decoder for K=2.

Both of them bring similar improvements and training cost in our observation.

JCZ404 commented 1 year ago

I got it, thanks for your kind reply!