Closed YellowPig-zp closed 2 years ago
Thanks for your issues. It seems like a bug in our implementation.
The code in line 391 should be changed to reference_points_input = reference_points[:, None] \
I think.
Hence better results could be expected if modified.
I will do more experiments and fix them later.
Thank you for the reply! Seems like the bug is only affecting the first decoder layer. The shape of reference_points in the concerned line of code is 300x4 for the first layer, and later layers have that of batchsize x 300 x 4 which would be broadcasted correctly regarding the last dimension as xywh. I guess a simple fix is to unsqueeze and expand/repeat the first reference_points to have a batch dimension :)
Hello, I fixed the bug and reran the dab-deformable-detr models. We achieved a better result 48.7
AP on COCO with a R50 backbone. Thanks for pointing out the problem.
I am having some trouble understanding the role of refpoint_embed under DABDeformableDETR module. Particularly what are the 4 values represent. Do they represent x,y,w,h or correspond to the 4 levels of the input feature maps? Because from line 391 in models/dab_deformanle_detr/deformable_transformer.py(link) the four values seem to be multiplied with valid ratios from four levels and broadcast along the xywh dimension. Also, when they are inputted into deformable attention, the order of dimension indicate that the 4 values correspond to 4 levels. On the other hand, when random_refpoints_xy is used, the first two values seem to represent xy instead? It's a bit confusing.