Increasing batch size negatively impacts mAP, is it because of padding ?

[InternetXplorer]

Hello, I have noticed that running evaluations with batch size > 1 leads to much lower mAP, so I was wondering if the reason is because the model (large fl4 with 5scale DINO) was trained with only 1 image per GPU ? It is not specified in focal-dino's README and I would like to make sure this is indeed the reason.

And as an additional question, does someone know why increasing the batch size does not improve the inference speed / image ? I just know that it is not because of focalnet backbone, because I have observed the same effect with resnet50 and swin backbones.

[jwyang]

Yes, it is mainly because of the padding for different images of different sizes. For inference, please always use bs=1 per GPU to ensure the evaluation is correct.

It might be attributed to that bs=1 already uses up all the GPU computations on one GPU, as such it is not necessary to improve the inference speed using bs=2 but still one GPU.