posts/pytorch-train-object-detector-yolox-tutorial/ort-tensorrt-ubuntu/

[utterances-bot]

Quantizing YOLOX with ONNX Runtime and TensorRT in Ubuntu – Christian Mills

Learn how to quantize YOLOX models with ONNX Runtime and TensorRT for int8 inference.

https://christianjmills.com/posts/pytorch-train-object-detector-yolox-tutorial/ort-tensorrt-ubuntu/

[Boo2z]

Hi Chriss when i'm trying to build TensorRT Engine i get the following output: "EP Error: [ONNXRuntimeError] : 11 : EP_FAIL : Non-zero status code returned while running TRTKernel_graph_main_graph_10988226158963509813_0 node. Name:'TensorrtExecutionProvider_TRTKernel_graph_main_graph_10988226158963509813_0_0' Status Message: TensorRT EP failed to set INT8 dynamic range. using ['TensorrtExecutionProvider', 'CPUExecutionProvider'] Falling back to ['CPUExecutionProvider'] and retrying. 2024-10-10 15:06:48.006729074 [E:onnxruntime:Default, tensorrt_execution_provider.cc:112 SetDynamicRange] Found unsupported datatype! 2024-10-10 15:06:48.006748572 [E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running TRTKernel_graph_main_graph_10988226158963509813_0 node. Name:'TensorrtExecutionProvider_TRTKernel_graph_main_graph_10988226158963509813_0_0' Status Message: TensorRT EP failed to set INT8 dynamic range. CPU times: user 1.77 s, sys: 21.8 ms, total: 1.79 s Wall time: 644 ms" Googling didn't help much, maybe you can help with this problem?)

[Boo2z]

The problem is solved)

[cj-mills]

@Boo2z Glad you worked it out! Sorry, work has been too hectic these past few weeks for me to look into this.