evaluation with different data_loader_val batch size leads to different results

IDEA-Research / DINO

[ICLR 2023] Official implementation of the paper "DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection"

Apache License 2.0

2.08k stars 228 forks source link

evaluation with different data_loader_val batch size leads to different results #231

Open cdluminate opened 8 months ago

cdluminate commented 8 months ago

At https://github.com/IDEA-Research/DINO/blob/main/main.py#L182 , the batchsize for the validation set is fixed to 1. Using the same ckpt, same hardware and the same environment, increasing the validation batch size will simply lead to worse AP. For instance, with batchsize=1 I may get 49.3, but with batchsize=4 I get 49.0. Any idea on how to fix this issue?