search

DerryHub / BEVFormer_tensorrt

BEVFormer inference on TensorRT, including INT8 Quantization and Custom TensorRT Plugins (float/half/half2/int8).
Apache License 2.0
432 stars 71 forks source link

ScatterND error when running onnx2trt.sh with TensorRT 8.5.2 #59

Open sun-lingyu opened 1 year ago

sun-lingyu commented 1 year ago

Hi, I met the following error when executing the command sh samples/bevformer/plugin/tiny/onnx2trt.sh -d 0 : 

...(Omitted)
[06/11/2023-18:30:12] [TRT] [V] node_of_onnx::ScatterND_1203 [Reshape] inputs: [onnx::Reshape_1135 -> (4, 1, 6, 2500)[FLOAT]], [onnx::Reshape_1202 -> (5)[INT32]], 
[06/11/2023-18:30:12] [TRT] [V] Registering layer: node_of_onnx::ScatterND_1203 for ONNX node: node_of_onnx::ScatterND_1203
[06/11/2023-18:30:12] [TRT] [E] [graphShapeAnalyzer.cpp::analyzeShapes::1872] Error Code 4: Miscellaneous (IShuffleLayer node_of_onnx::ScatterND_1203: reshape changes volume to multiple of original volume. Reshaping [4,1,6,2500] to [1,1,6,1,1].)
ERROR: Failed to parse the ONNX file.
In node 502 (parseGraph): INVALID_NODE: Invalid Node - node_of_onnx::ScatterND_1203
[graphShapeAnalyzer.cpp::analyzeShapes::1872] Error Code 4: Miscellaneous (IShuffleLayer node_of_onnx::ScatterND_1203: reshape changes volume to multiple of original volume. Reshaping [4,1,6,2500] to [1,1,6,1,1].)

It seems that the ScatterND is trying to falsely reshaping something...

My environment is slightly different with the author's recommendation. I am using TensorRT 8.5.2 instead of 8.5.1 since I am using a Jetson Orin which does not has a 8.5.1 version. My pytorch version is 1.12 and my CUDA version is 11.4.

Do you think this error is related to TensorRT version? I found the following bug fix of TensorRT 8.5.2 in TensorRT release note: The ONNX parser recognizes the allowzero attribute on Reshape operations for opset 5 and higher, even though ONNX spec requires it only for opset 14 and higher. Setting this attribute to 1 can correct networks that are incorrect for empty tensors, and let TensorRT analyze the memory requirements for dynamic shapes more accurately. Could this bug fix be related to my issue?

Any help is much appreciated!

nainaigetuide commented 8 months ago

Has this bug been resolved?

xiaohangimg commented 3 months ago

Hello, I encountered the same issue. I was able to complete the conversion from pth to ONNX to TRT on my x86 PC, but I couldn't do it on the ARM-based Orin. I suspect that this issue might be related to the ARM version of ONNX, which might not be able to export a correct ONNX file. You can try copying the ONNX file exported from the PC to the ARM platform, as ONNX is architecture-independent.