Closed dreambit closed 3 years ago
Hi How to use batch in order to improve inference throughput? Thanks.
Hi, you can set your maximum batch size here:
https://github.com/isarsoft/yolov4-triton-tensorrt/blob/ba0d4fd1c7f0001d2e77c44f5e32b04c4acb7c7f/main.cpp#L19
Hi How to use batch in order to improve inference throughput? Thanks.