RT-DETR-R18 onnx2tensorrt succeeded in int8 but failed in fp16

[shuchang0714]

I tried to convert RT-DETR-R18 from onnx to tensorrt, and I succeeded in int8, failed in fp16.

torch2onnx in STATIC: python tools/export_onnx.py onnx2trt: ./trtexec --onnx=rtdetr.onnx --saveEngine=rtdetr.trt --{fp16 or best or in8}

result on COCO:

Anyone met the same problem?

My Environment: os == Linux GPU == A100 cuda == 11.7 torch == 2.0.1 torchvision == 0.15.2 onnx == 1.14.0 Opset version == 16 TensorRT == 8.6.0.1

[xiaochus]

I have the same problem，RT-DETR-R18 FP16 have correct result on V100 and weird result on A10 （TensorRT 8.6.1）.

[chinakook]

+1