search

isarsoft / yolov4-triton-tensorrt

This repository deploys YOLOv4 as an optimized TensorRT engine to Triton Inference Server
http://www.isarsoft.com
Other
276 stars 63 forks source link

How to use batch? #40

Closed dreambit closed 3 years ago

dreambit commented 3 years ago

Hi How to use batch in order to improve inference throughput? Thanks.

philipp-schmidt commented 3 years ago

Hi, you can set your maximum batch size here:

https://github.com/isarsoft/yolov4-triton-tensorrt/blob/ba0d4fd1c7f0001d2e77c44f5e32b04c4acb7c7f/main.cpp#L19