[graphOptimizer.cpp::fusePattern::1777] Error Code 2: Internal Error (Assertion matchPattern(context, first) && matchBackend(first) failed. )

[HYLcool]

Hi, when I convert a quantized onnx model to trt engine using trtexec --fp16 --int8 --onnx=xxx.onnx, I met a error like below:

......
[03/01/2023-16:52:23] [TRT] [V] QuantizeDoubleInputNodes: fusing 510_QuantizeLinear into Conv_118 + Relu_119
[03/01/2023-16:52:23] [TRT] [V] QuantizeDoubleInputNodes: fusing (506_DequantizeLinear and 904_DequantizeLinear) into Conv_118 + Relu_119
[03/01/2023-16:52:23] [TRT] [V] Removing 510_QuantizeLinear
[03/01/2023-16:52:23] [TRT] [V] Removing 506_DequantizeLinear
[03/01/2023-16:52:23] [TRT] [V] Removing 904_DequantizeLinear
[03/01/2023-16:52:23] [TRT] [V] ConstWeightsFusion: Fusing 904 + 904_QuantizeLinear with Conv_118 + Relu_119
[03/01/2023-16:52:23] [TRT] [E] 2: [graphOptimizer.cpp::fusePattern::1777] Error Code 2: Internal Error (Assertion matchPattern(context, first) && matchBackend(first) failed. )
[03/01/2023-16:52:23] [TRT] [E] 2: [builder.cpp::buildSerializedNetwork::636] Error Code 2: Internal Error (Assertion engine != nullptr failed. )

And the structure of this area is shown below:

This onnx model can be downloaded here

My environ: python 3.7 CUDA 11.4 TensorRT 8.4.1.5

Is it something wrong with the placements of Q/DQ pairs or the structure of this model? Or are there any notices in TensorRT when using trtexec to convert a quantized onnx model to trt engine? Thanks!

[zerollzeng]

TRT 8.6 works for me and it will release soon. I would suggest to try latest 8.5 first, maybe it's already fixed.

&&&& PASSED TensorRT.trtexec [TensorRT v8600] # trtexec --onnx=matchPattern.onnx --int8 --verbose

[ttyio]

closing since no activity for more than 3 weeks, thank you!