TopK INT32 support problems, even with TensorRT 8.5.2

[chen-harrison]

Description

This comment indicated that a TopK issue where it did not support INT32 was solved in TensorRT 8.5.2. I was having this issue months ago trying to do TensorRT conversion from ONNX on my Jetson Xavier NX, so I used The NGC Docker container with 8.5.2 to try again.

However, upon running I run into an integer issue with TopK still:

[04/20/2023-18:52:48] [E] [TRT] ModelImporter.cpp:726: While parsing node number 892 [TopK -> "/TopK_1_output_0"]:
[04/20/2023-18:52:48] [E] [TRT] ModelImporter.cpp:727: --- Begin node ---
[04/20/2023-18:52:48] [E] [TRT] ModelImporter.cpp:728: input: "/Slice_188_output_0"
input: "/Constant_924_output_0"
output: "/TopK_1_output_0"
output: "/TopK_1_output_1"
name: "/TopK_1"
op_type: "TopK"
attribute {
  name: "axis"
  i: 2
  type: INT
}
attribute {
  name: "largest"
  i: 0
  type: INT
}

[04/20/2023-18:52:48] [E] [TRT] ModelImporter.cpp:729: --- End node ---
[04/20/2023-18:52:48] [E] [TRT] ModelImporter.cpp:731: ERROR: onnx2trt_utils.cpp:23 In function notInvalidType:
[8] Found invalid input type of INT32

I am on Jetpack 5.0.2, which natively runs TensorRT 8.4.1, and when I run it there, it catches on the same layer, but with a different error print, coming from the same builtin_op_importers.cpp file mentioned above:

[04/20/2023-15:14:51] [E] [TRT] ModelImporter.cpp:773: While parsing node number 892 [TopK -> "/TopK_1_output_0"]:
[04/20/2023-15:14:51] [E] [TRT] ModelImporter.cpp:774: --- Begin node ---
[04/20/2023-15:14:51] [E] [TRT] ModelImporter.cpp:775: input: "/Slice_188_output_0"
input: "/Constant_924_output_0"
output: "/TopK_1_output_0"
output: "/TopK_1_output_1"
name: "/TopK_1"
op_type: "TopK"
attribute {
  name: "axis"
  i: 2
  type: INT
}
attribute {
  name: "largest"
  i: 0
  type: INT
}

[04/20/2023-15:14:51] [E] [TRT] ModelImporter.cpp:776: --- End node ---
[04/20/2023-15:14:51] [E] [TRT] ModelImporter.cpp:778: ERROR: builtin_op_importers.cpp:4424 In function importTopK:
[8] Assertion failed: (tensorPtr->getType() != nvinfer1::DataType::kINT32) && "This version of TensorRT does not support INT32 input for the TopK operator."

Is it because INT32 is still listed as an incompatible type with the operator (i.e. in the invalidTypes input into notInvalidType())? Is it because the values were originally int64 that are clamped down to int32?

Layer info, via Netron:

Environment

TensorRT Version: 8.5.2 Docker container, 8.4.1 locally ONNX-TensorRT Version / Branch: N/A GPU Type: Jetson Xavier NX Nvidia Driver Version: ? (couldn't find the number since nvidia-smi doesn't work on Jetson hardware) CUDA Version: 11.4 CUDNN Version: 8.4.1 Operating System + Version: Jetpack 5.0.2 Python Version (if applicable): TensorFlow + TF2ONNX Version (if applicable): PyTorch Version (if applicable): Baremetal or Container (if container which image + tag): l4t-tensorrt, r8.5.2-runtime

Relevant Files

fast_acvnet_plus_generalization_opset16_256x320.zip

[arko-sarkar]

@chen-harrison : Did you solve this issue with any other tensorrt version or method? As I still get the same issue of (TopK Found invalid input type of INT32) in Tensorrt-8.6.1.6-1+cuda12.0

[lix19937]

https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-853/operators/index.html#layers-matrix

TopKLayer only support fp32/fp16