Picodet to ONNX on Xavier NX use onnx_tensorrt.backend

[NoLoPhe]

I use two computers *the first computer only runs on CPU I used CPU to Convert to ONNX Paddle2ONNX= 0.8.2 and ONNX 1.9.0 onnx-simplifier = 0.3.6

I run:

python tools/export_model.py \
-c configs/picodet/picodet_s_320_coco.yml \
-o weights=https://paddlet.bj.bcebos.com/models/picodet_s_320_coco.pdparams \
--output_dir=inference_model

paddle2onnx \
--model_dir output_inference/picodet_s_320_coco/ \
--model_filename model.pdmodel \
--params_filename model.pdiparams \
--opset_version 11 \
--save_file picodet_s_320_coco.onnx

python -m onnxsim picodet_s_320_coco.onnx picodet_s_processed.onnx

I don't get any error

*second computer is xavier NX, when I run ONNX on xavier NX use onnx_tensorrt.backend ONNX 1.8.1 tensorRT 7.1.3 jetpack 4.5.1

I run:

import onnx
import onnx_tensorrt.backend as backend
import numpy as np
import time

model = onnx.load("picodet_s_320_coco.onnx")
engine = backend.prepare(model, device='CUDA:0')
input_data = np.random.random(size=(1,3,320,320)).astype(np.float32)

roop = 1
e = 0.0

for _ in range(roop):
    s = time.time()
    output_data = engine.run(input_data)[0]
    e += (time.time() - s)
print(f'elapsed time: {e/roop*1000}ms')

I got an error

[TensorRT] WARNING: onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
Traceback (most recent call last):
  File "test_1.py", line 7, in <module>
    engine = backend.prepare(model, device='CUDA:0')
  File "/home/tnt/onnx-tensorrt/onnx_tensorrt/backend.py", line 200, in prepare
    return TensorRTBackendRep(model, device, **kwargs)
  File "/home/tnt/onnx-tensorrt/onnx_tensorrt/backend.py", line 86, in _init_
    raise RuntimeError(msg)
RuntimeError: While parsing node number -1:
builtin_op_importers.cpp:2523 In function importResize:
[8] Assertion failed: (mode != "nearest" || nearest_mode == "floor") && "This version of TensorRT only supports floor nearest_mode!"

[NoLoPhe]

I used the Picodet model you released, but still got the same error.

[NoLoPhe]

I have tested and sure onnx_tensorrt.backend has no problem, I have tested the model yolox onnx

[PINTO0309]

This is a TensorRT problem.

Assertion failed: (mode != "nearest" || nearest_mode == "floor") && "This version of TensorRT only supports floor nearest_mode!" https://forums.developer.nvidia.com/t/centernet-mobilenet-v2-from-tfod-failure-when-converting-onnx-to-trt/183972/6

[NoLoPhe]

Thank you for your immediate reply!

[PINTO0309]

TensorRT 8.2. It works normally. I've known for a while that the old TensorRT had that problem. So I recently updated all my environments to TensorRT 8.2.

>>> import onnx
>>> import onnx_tensorrt.backend as backend
>>> import numpy as np
>>> import time
>>> model = onnx.load("picodet_s_320x320.onnx")
>>> engine = backend.prepare(model, device='CUDA:0')
[11/16/2021-11:44:38] [TRT] [W] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
>>> input_data = np.ones((1,3,320,320)).astype(np.float32)
>>> output_data = engine.run(input_data)[0]
>>> print(output_data)
[[[0.00368589 0.00156462 0.00319499 ... 0.00099653 0.00055185 0.00061941]
  [0.00282133 0.00119847 0.00223592 ... 0.00065058 0.00027722 0.00038844]
  [0.00267431 0.00134867 0.00246176 ... 0.00068846 0.00026196 0.00044441]
  ...
  [0.00795203 0.00173521 0.00448856 ... 0.00186968 0.00104919 0.00120309]
  [0.00829846 0.00194225 0.00480601 ... 0.00202399 0.001185   0.00131527]
  [0.00815716 0.0022305  0.00516585 ... 0.00217304 0.00137123 0.00165436]]]

[NoLoPhe]

Installing onnx_tensorrt on Xavier NX is very difficult, I went around from jetpack 4.5 -> 4.6 -> 4.4 -> 4.5. In jetpack 4.6 tensorRT 8.0.1.6 requires library onnx=1.8.0, can only install onnx=1.6.0 (if anyone reading this is on jetpack 4.6, I would love to ask them to install onnx=1.8. 1, is it possible?), besides scipy is also very difficult to install. In jetpack 4.4 version, I often encounter minor errors, unstable operation, or freeze when running the model. I am very satisfied with jetpack 4.5.1.

Following your suggestion on the Nvidia forum the more modern tensorflow code, I hope can solve this problem. Can you help me convert by another pipeline Padlepadle -> TF -> onnx.

I suddenly came up with an idea like that, I don't know if it's possible.

[PINTO0309]

• PP -> ONNX -> OpenVINO -> TF -> ONNX model_float32.onnx.tar.gz

See. https://github.com/PINTO0309/PINTO_model_zoo/issues/150

If the accuracy deteriorates, there is nothing you can do about it.

[NoLoPhe]

Once again many thanks for your help!