Yolov5 tensorRT 推理时间问题

ycdhqzhiai commented 3 years ago

请教下大家，我这边测试yolov5 模型，推理时间很慢

loading filename from:../yolov5x.trt
[01/22/2021-08:39:50] [W] [TRT] TensorRT was linked against cuDNN 8.0.5 but loaded cuDNN 8.0.4
deserialize done
[01/22/2021-08:39:52] [W] [TRT] TensorRT was linked against cuDNN 8.0.5 but loaded cuDNN 8.0.4
binding0: 1277952
binding1: 340704
Processing: ../samples//000015.jpg
prepareImage
prepare image take: 7.17282 ms.
host2device
execute
Inference take: 1114.77 ms.
execute success
device2host
post process
Post process take: 0.766905 ms.
../samples//000015_.jpg
Average processing time is 1122.71ms

一帧耗时很不稳定，从500ms-2s都有，pt模型是yolov5原始ymal d=0.1 w=0.25（比v5s还小） cuda11.1-trt7.2.2.3 tensorrt新手，是我姿势不对吗？求解答

linghu8812 commented 3 years ago

用的什么GPU啊？pytorch模型用时多长时间？

ycdhqzhiai commented 3 years ago

3090服务器，没在服务器上跑过，在本机PC 2070上pth模型测试0.008s

ycdhqzhiai commented 3 years ago

256*416输入

DaChaoXc commented 3 years ago

你看结果了吗？1000多ms可能结果根本不对，有n个框

ycdhqzhiai commented 3 years ago

@DaChaoXc 结果没问题，同一个pth模型，按照uv5原始方式转换onnx, 照抄tensorrt sample里面写，推理耗时2ms左右

hengyanchen commented 2 years ago

解决了吗？我也遇到类似的问题，20到50ms波动，关键是推理同一张图片

linghu8812 / tensorrt_inference

Yolov5 tensorRT 推理时间问题 #35