search

itsnine / yolov5-onnxruntime

YOLOv5 ONNX Runtime C++ inference code.
241 stars 58 forks source link

Integrate with TensorRT? #6

Open rtrahms opened 2 years ago

rtrahms commented 2 years ago

I tried out your sample - very cool! I get 110 FPS with a YOLOv5s running CUDA 11.5 on my 1080ti. I am curious what it would take to evaluate performance with TensorRT. Have you tried this? Any pointers? Thanks.

zhiqwang commented 2 years ago

There are two ways to integrate the TensorRT to this project

  1. Use the TensorRT-ExecutionProvider provided by ONNX Runtime, this is simple and can be integrated into this project seamlessly.
  2. Use the vanilla TensorRT-OSS, we can also use ONNX and the TRT Engine exported by YOLOv5.

For the second option, we can also embed the post-processing plugin provided by TRT-OSS with the onnx-graphsurgeon at the end of the exported ONNX graph, we have done some experiments on yolort. And this can reduce the difficulties of deploying. The remaining things that need to do is to implement the pre-processing transformation already available in this project.