why tensorrt do inference slower than pytorch

DerryHub / BEVFormer_tensorrt

BEVFormer inference on TensorRT, including INT8 Quantization and Custom TensorRT Plugins (float/half/half2/int8).

Apache License 2.0

432 stars 71 forks source link

Open gofreelee opened 10 months ago

gofreelee commented 10 months ago

Device: NVIDIA A100 80G tensorrt fps: 2.61 pytorch fps: 2.94 Why is tensorrt's performance even lower