关于query scoring of DETR的解释问题

[wanghua12]

你好，我想问一下，论文中为什么说detr的query scoring 是存粹基于预测的分类置信度？采用分类置信度作为对所有查询建议进行排序的代理？detr不是根据预测的100个box的类别和边界框的损失，然后计算匈牙利损失矩阵，然后进行匹配，得到相应的匹配结果。根据匹配的结果计算最终所有损失的加权和，反向梯度回传，更新参数。query scoring 具体是指什么？

[wanghua12]

这个地方的query查询分数是在推理阶段所使用的吗？detr在推理阶段限制置信度在0.7以上的进行保留，所以这个地方是相当于将置信度创新，采用置信度与期望IOU的乘积，然后根据阈值保留预测的query吗？

[ymq2017]

Yes, your description of matching is correct.

For the first problem, Hungarian matching already considers localization quality, but this matching has no gradient. Loss is calculated based on the matching result and backpropagated to previous layers. Here the output score only has a classification loss. We want to add explicit supervision of localization quality to the final classification loss, so we also record the matched IoU in Hungarian matching and regress this matched IoU.

For the second problem, you could find an example in fig9 of our paper to show the usage of query re-scoring. Output will low localization quality will be filtered out while those high-quality boxes perserved.