Why is the forward latency of 640 even lower than 320's?

[fengyuentau]

So the following is my testing results on widerface val:	model	latency (ms)
slim-320	4.43
slim-640	4.21
RFB-320	5.91
RFB-640	5.64

And even if I change the input scale from the fixed settings to the original scale of images, same happens:	model	latency (ms)
slim-320	4.21
slim-640	4.21
RFB-320	5.61
RFB-640	5.54

From my point of view, if the input size increase while keeping the model architectures the same, the forward latency should also increase, but the results went against. I also checked the code, but found nothing different between model in 320 and 640. Do you have any ideas how to explain this? Following is my testing environment:

• Ubuntu 18.04.4 LTS
• conda 4.8.3
• torch 1.0.0
• torchvision 0.2.1
• CUDA 10.0.130
• cuDNN 7.4.1

[fengyuentau]

By forward latency, I mean the time cost of a single and only forward process of the network.