the speed after using tensorRT is not faster than pytorch

[yilawu]

I have run main.py and get the result like this

tensorRT inference time: avg GPU time: 0.0072007338205973305 avg GPU compute time: 0.0065811554590861 avg pre time: 0.0005954186121622722 avg post time: 2.4159749348958332e-05 pytorch inference time: avg GPU time: 0.007817573017544217 avg GPU compute time: 0.006660024325052897 avg pre time: 0.0005226506127251519 avg post time: 0.0006348980797661676 trt_result ============================= [array([[[ 415.59, 240.87, 506.79, 545.85, 0.83679, 0], [ 49.603, 242.71, 167.21, 552.12, 0.81033, 0], [ 18.173, 137.37, 515.33, 463.03, 0.71066, 5], [ 149.82, 246.16, 230.95, 529.5, 0.69774, 0], [ 16.668, 346.08, 62.551, 542.5, 0.45844, 0]]], dtype=float32)] pytorch_result ============================== [array([[ 415.47, 231.19, 495.51, 524.26, 0.8785, 0], [ 147.73, 239.38, 221.98, 510.14, 0.86237, 0], [ 46.126, 236.25, 158, 531.58, 0.84014, 0], [ 44.223, 136.36, 494.82, 450.86, 0.63891, 5], [ 16.669, 331.73, 57.432, 517.17, 0.52587, 0]], dtype=float32)]

tensorRT avg GPU time: 0.0072007338205973305 pytorch avg GPU time: 0.007817573017544217 which looks like it speed up just 7.7％. while readme claims that it can speed up 4x

[TrojanXu]

Currently it depends on your GPU. On V100, it delivers a 4x speedup, while on other gpus, especially on pascal GPUs, it doesn't accelerate that much.

[yilawu]

Currently it depends on your GPU. On V100, it delivers a 4x speedup, while on other gpus, especially on pascal GPUs, it doesn't accelerate that much.

my gpu is rtx2070s

[makaveli10]

@wurongyuan can you share the working tensorrt code! the current repo seems to throw some onnx parsing error. Its kind of urgent. Thanks in advance.

[yilawu]

@wurongyuan can you share the working tensorrt code! the current repo seems to throw some onnx parsing error. Its kind of urgent. Thanks in advance.

my code is just from this repo