Hi all, I need to deploy the model with dynamic batching on DS-Triton, but the YOLOV4 example in DeepStream says Following properties are always recommended: # batch-size(Default=1)
I run a test with yolov3 comparing BS=8 vs BS=1 and it exposed poor performance 0.24X , running TensorRT engine with DeepStream 5.1:
Is this multi-batch poor performance related to the statement "Note that for multi-batch, overhead is large owing to NMS is not used."?, what does it mean?.
What do you recommend to work with BS>1?
Hi all, I need to deploy the model with dynamic batching on DS-Triton, but the YOLOV4 example in DeepStream says Following properties are always recommended: # batch-size(Default=1)
I run a test with yolov3 comparing BS=8 vs BS=1 and it exposed poor performance 0.24X , running TensorRT engine with DeepStream 5.1:
Throughput FPS (avg) | INT8
BS =1 → PERF: 246.29 (245.98) BS =8 → PERF: 60.31 (60.63)
Is this multi-batch poor performance related to the statement "Note that for multi-batch, overhead is large owing to NMS is not used."?, what does it mean?. What do you recommend to work with BS>1?