[SCALED YOLOV4] Inference time is linear respective to batch size

[bobbilichandu]

I am using ScaledYolov4 with tensorrt version 7.2.2 and T4 GPUs. When I am increasing batchsize, inference time is increasing linearly. Is this behaviour expected? FOR BATCH SIZE - 1:

Inference take: 48.5283 ms.
Inference take: 48.518 ms.
Inference take: 40.1897 ms.
Inference take: 40.0713 ms.
Inference take: 38.54 ms.
Inference take: 38.7829 ms.
Inference take: 38.6083 ms.
Inference take: 38.6635 ms.
Inference take: 38.1827 ms.
Inference take: 38.1016 ms

FOR BATCH SIZE - 2:

Inference take: 76.3045 ms.
Inference take: 74.9346 ms.
Inference take: 73.3341 ms.
Inference take: 73.9554 ms.
Inference take: 73.4185 ms.
Inference take: 75.4546 ms.
Inference take: 77.7809 ms.
Inference take: 78.3289 ms.
Inference take: 79.5533 ms.
Inference take: 79.0556 ms.
Inference take: 79.2939 ms.
Inference take: 77.214 ms.

FOR BATCH SIZE - 4:

Inference take: 158.327 ms.
Inference take: 157.001 ms.
Inference take: 157.107 ms.
Inference take: 154.237 ms.
Inference take: 155.899 ms.
Inference take: 157.408 ms.
Inference take: 155.758 ms.
Inference take: 155.906 ms.

I expected batch size not to have this proportionality. Can anything done to improve the inference time using batching? TIY.