[Bug] failed install Model Converter on Jetpack4.5.1 AGX Xavier by mmdeploy 1.2.0

[azuryl]

Checklist

• [X] I have searched related issues but cannot get the expected help.
• [X] 2. I have read the FAQ documentation but cannot get the expected help.
• [X] 3. The bug has not been fixed in the latest version.

Describe the bug

I'm failed to install Jetson AGX Xavier in jetpack 4.5.1 following the Build for Jetson.

git clone -b main --recursive https://github.com/open-mmlab/mmdeploy.git cd mmdeploy export MMDEPLOY_DIR=$(pwd)

build TensorRT custom operators

mkdir -p build && cd build cmake .. -DMMDEPLOY_TARGET_BACKENDS="trt" make -j$(nproc) && make install

install model converter

cd ${MMDEPLOY_DIR} pip install -v -e .

Using pip 21.3.1 from /home/nvidia/archiconda3/envs/mmdeploy/lib/python3.6/site-packages/pip (python 3.6) Obtaining file:///data/azuryl/mmdeploy Running command python setup.py egg_info running egg_info creating /tmp/pip-pip-egg-info-vma11n80/mmdeploy.egg-info writing /tmp/pip-pip-egg-info-vma11n80/mmdeploy.egg-info/PKG-INFO writing dependency_links to /tmp/pip-pip-egg-info-vma11n80/mmdeploy.egg-info/dependency_links.txt writing requirements to /tmp/pip-pip-egg-info-vma11n80/mmdeploy.egg-info/requires.txt writing top-level names to /tmp/pip-pip-egg-info-vma11n80/mmdeploy.egg-info/top_level.txt writing manifest file '/tmp/pip-pip-egg-info-vma11n80/mmdeploy.egg-info/SOURCES.txt' reading manifest file '/tmp/pip-pip-egg-info-vma11n80/mmdeploy.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' /home/nvidia/archiconda3/envs/mmdeploy/lib/python3.6/site-packages/torch/utils/cpp_extension.py:381: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend. warnings.warn(msg.format('we could not find ninja.')) warning: no files found matching 'mmdeploy/backend/ncnn/.so' warning: no files found matching 'mmdeploy/backend/ncnn/.dll' warning: no files found matching 'mmdeploy/backend/ncnn/.pyd' warning: no files found matching 'mmdeploy/backend/ncnn/mmdeploy_onnx2ncnn' warning: no files found matching 'mmdeploy/lib/.dll' warning: no files found matching 'mmdeploy/lib/.pyd' warning: no files found matching 'mmdeploy/backend/torchscript/.so' warning: no files found matching 'mmdeploy/backend/torchscript/.dll' warning: no files found matching 'mmdeploy/backend/torchscript/*.pyd' adding license file 'LICENSE' writing manifest file '/tmp/pip-pip-egg-info-vma11n80/mmdeploy.egg-info/SOURCES.txt' Preparing metadata (setup.py) ... done Collecting aenum Downloading aenum-3.1.15-py3-none-any.whl (137 kB) |████████████████████████████████| 137 kB 2.7 MB/s
Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/53/c2/b891e3e0ff02a784cbe64f1712f539b30d10ec09af6c9405e05ba316a50e/grpcio-1.49.0rc1.tar.gz#sha256=e4096e2a27cac06cdf9971c99c23dba6d88aa65676621aff356556817f1839d7 (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7) Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/56/57/f50c9e3bcde690bd08b18fa1885d6cb578a6b10cf7e492c9e69646688f50/grpcio-1.49.0rc3.tar.gz#sha256=006086cea4a05d70cbd83343e06f9370c64b38570eeb4486c5c7b12030736c98 (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7) Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/44/d0/789c52d384c97118b2dce430e3fcea50be85c5d0f9b6ab986865041bb2a3/grpcio-1.49.0.tar.gz#sha256=90ec337f36db26fbc70a4c032bcbabb2008f950f4194e99385118a12688fdf92 (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7) Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/a3/1c/c42834d4fee45c5cf2d9546e97e879a1cbcdecfd16eb1a12144dcb91edae/grpcio-1.49.1.tar.gz#sha256=d4725fc9ec8e8822906ae26bb26f5546891aa7fbc3443de970cc556d43a5c99f (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7) Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/62/10/3be7fa7ddb13e1b110a4a9e71968587b04daa7f9ae8fae2bb8664a8baceb/grpcio-1.50.0rc1.tar.gz#sha256=bc50887373898f247f749afc70dc8093ed12b6657d7841893a80cb9efe6a1110 (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7) Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/71/87/118ed062bba99c54162f9b3a15cc58f018e157fe2f274edd7bc1a8a5bfe2/grpcio-1.50.0.tar.gz#sha256=12b479839a5e753580b5e6053571de14006157f2ef9b71f38c56dc9b23b95ad6 (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7) Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/fd/cb/bfaa11656cc7fea6f0ccf0132448693e8213d966c760e58574e5c7707131/grpcio-1.51.0rc1.tar.gz#sha256=60b57eeae0c095a68139445345cc564f49b2ce4e37168d5510022e72dd491dec (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7) Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/26/75/30370b9135034d809475cb4ef711079ddd6f0e302a17aef4982d5cee5942/grpcio-1.51.0.tar.gz#sha256=a103811a4b318abc9b1592cd6cc9187d34bfb2e192eb0fe113dfabb38a162284 (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7) Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/c4/8a/61f84aa2f061395a1aa9faaf325fa200da44191c9631082f33d46602efff/grpcio-1.51.1.tar.gz#sha256=e6dfc2b6567b1c261739b43d9c59d201c1b89e017afd9e684d85aa7a186c9f7a (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7)

Reproduction

build TensorRT custom operators

mkdir -p build && cd build cmake .. -DMMDEPLOY_TARGET_BACKENDS="trt" make -j$(nproc) && make install

install model converter

cd ${MMDEPLOY_DIR} pip install -v -e .

Environment

JetPack: 4.5.1
Python: 3.6.9
CUDA: 10.2
torch: 1.10.0
torchvision: 0.11.1
mmdeploy 1.2.0
mmcv:2.0.0
onnx:1.10.0
tensorrt:7.1.3.0

I modified mmcv according 
https://github.com/open-mmlab/mmdeploy/issues/2176
changed  python_requires='>=3.6',://github.com/open-mmlab/mmcv/blob/558742c9cb1fc593996e55522a08568f79ff51e5/setup.py#LL434C1-L434C1

Error traceback

(mmdeploy) nvidia@xavier0:/data/azuryl/mmdeploy$ pip install -v -e .
Using pip 21.3.1 from /home/nvidia/archiconda3/envs/mmdeploy/lib/python3.6/site-packages/pip (python 3.6)
Obtaining file:///data/azuryl/mmdeploy
  Running command python setup.py egg_info
  running egg_info
  creating /tmp/pip-pip-egg-info-vma11n80/mmdeploy.egg-info
  writing /tmp/pip-pip-egg-info-vma11n80/mmdeploy.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-pip-egg-info-vma11n80/mmdeploy.egg-info/dependency_links.txt
  writing requirements to /tmp/pip-pip-egg-info-vma11n80/mmdeploy.egg-info/requires.txt
  writing top-level names to /tmp/pip-pip-egg-info-vma11n80/mmdeploy.egg-info/top_level.txt
  writing manifest file '/tmp/pip-pip-egg-info-vma11n80/mmdeploy.egg-info/SOURCES.txt'
  reading manifest file '/tmp/pip-pip-egg-info-vma11n80/mmdeploy.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  /home/nvidia/archiconda3/envs/mmdeploy/lib/python3.6/site-packages/torch/utils/cpp_extension.py:381: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
    warnings.warn(msg.format('we could not find ninja.'))
  warning: no files found matching 'mmdeploy/backend/ncnn/*.so'
  warning: no files found matching 'mmdeploy/backend/ncnn/*.dll'
  warning: no files found matching 'mmdeploy/backend/ncnn/*.pyd'
  warning: no files found matching 'mmdeploy/backend/ncnn/mmdeploy_onnx2ncnn*'
  warning: no files found matching 'mmdeploy/lib/*.dll'
  warning: no files found matching 'mmdeploy/lib/*.pyd'
  warning: no files found matching 'mmdeploy/backend/torchscript/*.so'
  warning: no files found matching 'mmdeploy/backend/torchscript/*.dll'
  warning: no files found matching 'mmdeploy/backend/torchscript/*.pyd'
  adding license file 'LICENSE'
  writing manifest file '/tmp/pip-pip-egg-info-vma11n80/mmdeploy.egg-info/SOURCES.txt'
  Preparing metadata (setup.py) ... done
Collecting aenum
  Downloading aenum-3.1.15-py3-none-any.whl (137 kB)
     |████████████████████████████████| 137 kB 2.7 MB/s            
  Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/53/c2/b891e3e0ff02a784cbe64f1712f539b30d10ec09af6c9405e05ba316a50e/grpcio-1.49.0rc1.tar.gz#sha256=e4096e2a27cac06cdf9971c99c23dba6d88aa65676621aff356556817f1839d7 (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7)
  Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/56/57/f50c9e3bcde690bd08b18fa1885d6cb578a6b10cf7e492c9e69646688f50/grpcio-1.49.0rc3.tar.gz#sha256=006086cea4a05d70cbd83343e06f9370c64b38570eeb4486c5c7b12030736c98 (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7)
  Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/44/d0/789c52d384c97118b2dce430e3fcea50be85c5d0f9b6ab986865041bb2a3/grpcio-1.49.0.tar.gz#sha256=90ec337f36db26fbc70a4c032bcbabb2008f950f4194e99385118a12688fdf92 (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7)
  Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/a3/1c/c42834d4fee45c5cf2d9546e97e879a1cbcdecfd16eb1a12144dcb91edae/grpcio-1.49.1.tar.gz#sha256=d4725fc9ec8e8822906ae26bb26f5546891aa7fbc3443de970cc556d43a5c99f (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7)
  Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/62/10/3be7fa7ddb13e1b110a4a9e71968587b04daa7f9ae8fae2bb8664a8baceb/grpcio-1.50.0rc1.tar.gz#sha256=bc50887373898f247f749afc70dc8093ed12b6657d7841893a80cb9efe6a1110 (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7)
  Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/71/87/118ed062bba99c54162f9b3a15cc58f018e157fe2f274edd7bc1a8a5bfe2/grpcio-1.50.0.tar.gz#sha256=12b479839a5e753580b5e6053571de14006157f2ef9b71f38c56dc9b23b95ad6 (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7)
  Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/fd/cb/bfaa11656cc7fea6f0ccf0132448693e8213d966c760e58574e5c7707131/grpcio-1.51.0rc1.tar.gz#sha256=60b57eeae0c095a68139445345cc564f49b2ce4e37168d5510022e72dd491dec (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7)
  Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/26/75/30370b9135034d809475cb4ef711079ddd6f0e302a17aef4982d5cee5942/grpcio-1.51.0.tar.gz#sha256=a103811a4b318abc9b1592cd6cc9187d34bfb2e192eb0fe113dfabb38a162284 (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7)
  Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/c4/8a/61f84aa2f061395a1aa9faaf325fa200da44191c9631082f33d46602efff/grpcio-1.51.1.tar.gz#sha256=e6dfc2b6567b1c261739b43d9c59d201c1b89e017afd9e684d85aa7a186c9f7a (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7)
  Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/f9/d1/8e1d6f684e36492476e1d6bbbf0162f8021421d9880fc346620d29f3c5a6/grpcio-1.51.3.tar.gz#sha256=be7b2265b7527bb12109a7727581e274170766d5b3c9258d4e466f4872522d7a (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7)
  Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/ac/e2/6a3212b6d38a81dc7fde9c40b6d3cda7b5837139e3a575272213c8c9ce8a/grpcio-1.52.0rc1.tar.gz#sha256=476fa89a6c24ba47a2ea6aa3b01d86200c5cfd1eaef375b20bea9f96fb43cfe9 (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7)
  Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/ed/b7/6c5cf5c73445b28ac25347c78c6f2d76f2ed9a5d5966bd45b6904b3e7a51/grpcio-1.52.0.tar.gz#sha256=a5d4a83d29fc39af429c10b9b326c174fec49b73398e4a966a1f2a4f30aa4fdb (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7)
  Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/20/b1/5336e1f8c4c52f29c771a0e75b42e140c6a1e9eeb7c9b3465e18fdb36159/grpcio-1.53.0rc2.tar.gz#sha256=57424c9c25f4f976edd0e90f441dd8f600f51e2dba3ca9f35c11e8b17c0c6304 (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7)
  Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/9d/13/f7e3834ba272c0058c66de57272fb3b20dde69003032fdb6b4d7c36eebed/grpcio-1.53.0.tar.gz#sha256=a4952899b4931a6ba12951f9a141ef3e74ff8a6ec9aa2dc602afa40f63595e33 (from https://pypi.org/simple/grpcio/) (requires-python:>=3.7)
  Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/5e/88/ca25eb345ad58691d6660300c0a78f3c6418d0719cb43432578080be4472/grpcio-

  package init file 'onnx/backend/test/data/node/test_flatten_axis1/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_concat_3d_axis_negative_1/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_mean/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_globalaveragepool_precomputed/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_layer_normalization_4d_axis3/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_dynamicquantizelinear_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reversesequence_time/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sequence_map_identity_1_sequence/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_softmax_large_number/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_greater_equal_bcast/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_transpose_all_permutations_3/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_max_default_axes_keepdims_random/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_argmin_negative_axis_keepdims_example/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_gemm_all_attributes/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_min_two_inputs/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_instancenorm_epsilon/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_nllloss_NCd1_weight/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_leakyrelu_example/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_bitwise_xor_i16_3d/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_cumsum_1d/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_convinteger_without_padding/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_NCd1d2d3d4d5_mean_weight_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_xor_bcast3v1d/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_hannwindow_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_cos/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_l1_keep_dims_random/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_bitwise_or_ui64_bcast_3v1d/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_layer_normalization_3d_axis_negative_1_epsilon_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_mean_no_weight_ii_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_resize_downsample_sizes_cubic/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_l2_negative_axes_keep_dims_example_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sequence_insert_at_front/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_clip_default_min_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_layer_normalization_3d_axis_negative_1_epsilon_expanded_ver18/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_l1_default_axes_keepdims_example_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_layer_normalization_2d_axis_negative_1_expanded_ver18/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_flatten_negative_axis1/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_onehot_with_negative_axis/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_layer_normalization_4d_axis1_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_exp_example/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_and_bcast4v2d/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_castlike_FLOAT16_to_DOUBLE_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_mean_keepdims_example/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_layer_normalization_4d_axis_negative_2_expanded_ver18/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_layer_normalization_2d_axis_negative_2/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_none_weights_log_prob_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_tril_square/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_clip_default_int8_inbounds/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_optional_has_element_empty_no_input_optional_input/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_gridsample_reflection_padding/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_resize_upsample_scales_cubic/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_max_int8/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_prelu_broadcast_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_split_equal_parts_default_axis_opset18/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_unique_sorted_without_axis/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_NCd1d2d3d4d5_mean_weight/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_argmin_keepdims_random/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_gridsample_aligncorners_true/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_argmax_no_keepdims_random_select_last_index/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_slice/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_less_equal/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_split_2d_uneven_split_opset18/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sequence_map_extract_shapes_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_log_sum_exp_negative_axes_keepdims_random_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_unsqueeze_three_axes/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_mean_default_axes_keepdims_random/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_l2_default_axes_keepdims_random/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_castlike_FLOAT_to_STRING_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_NCd1d2d3d4d5_none_no_weight_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_nllloss_NCd1d2_with_weight_reduction_mean/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_castlike_FLOAT_to_BFLOAT16_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_castlike_FLOAT_to_FLOAT16_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_softmax_default_axis/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_batchnorm_epsilon_training_mode/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_mean_two_inputs/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_xor4d/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_shape_start_1_end_negative_1/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_acos_example/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_averagepool_2d_ceil/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_where_long_example/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_argmin_no_keepdims_example/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_min_negative_axes_keepdims_random/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_batchnorm_example_training_mode/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_castlike_DOUBLE_to_FLOAT/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_gru_batchwise/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_upsample_nearest/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_shape_end_1/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_eyelike_with_dtype/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_tfidfvectorizer_tf_only_bigrams_skip0/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_NCd1d2d3d4d5_none_no_weight_log_prob/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_batchnorm_epsilon/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sum_example/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_bitwise_and_ui8_bcast_4v3d/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_and_bcast3v1d/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_layer_normalization_3d_axis1_epsilon/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_NCd1_mean_weight_negative_ii/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_spacetodepth/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sequence_map_identity_2_sequences/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sinh/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_loop13_seq/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_and_bcast4v4d/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_onehot_without_axis/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_dynamicquantizelinear_min_adjusted_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_bernoulli_seed_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_softmax_negative_axis_expanded_ver18/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_thresholdedrelu_example_expanded_ver18/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_gather_1/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_sum_default_axes_keepdims_random/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_neg/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_clip_default_inbounds_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_mean_log_prob_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_resize_upsample_scales_nearest/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_averagepool_2d_precomputed_pads/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_maxpool_3d_default/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_resize_upsample_sizes_nearest_not_larger/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_tril_out_pos/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_pow_bcast_array/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_l1_keep_dims_example/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_gridsample/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_castlike_FLOAT_to_FLOAT16/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_scan9_sum/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_thresholdedrelu_expanded_ver18/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_clip_example/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_batchnorm_example/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_min_default_axes_keepdims_example/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_mean_weight_ii_log_prob_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sequence_map_identity_1_sequence_1_tensor_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_tril/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_bitwise_xor_ui8_bcast_4v3d/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_pow_types_int32_float32/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_rnn_seq_length/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_hardswish_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_scatter_elements_with_reduction_max/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_softmax_axis_1_expanded_ver18/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_nllloss_NC_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_sum_square_negative_axes_keepdims_random/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_layer_normalization_4d_axis_negative_2_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_nllloss_NCd1d2d3_none_no_weight_negative_ii/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_asin/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_tfidfvectorizer_tf_batch_onlybigrams_skip0/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_cast_FLOAT_to_STRING/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_cast_DOUBLE_to_FLOAT16/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_matmulinteger/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_nesterov_momentum/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_l2_negative_axes_keep_dims_example/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_none_weights_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_nllloss_NCd1_ii/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_averagepool_2d_same_lower/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_less_bcast/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_flatten_negative_axis4/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_sum_square_negative_axes_keepdims_random_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_hammingwindow/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_nonmaxsuppression_identical_boxes/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_slice_negative_axes/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_sum_do_not_keepdims_example/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_castlike_BFLOAT16_to_FLOAT_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_loop11/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_log_sum_exp_negative_axes_keepdims_example/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_blackmanwindow_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_maxpool_2d_precomputed_strides/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_unsqueeze_negative_axes/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_momentum/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sign/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_min_int64/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_tfidfvectorizer_tf_batch_onlybigrams_skip5/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_maxpool_2d_precomputed_same_upper/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_maxpool_2d_strides/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_prod_default_axes_keepdims_random/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_clip_default_inbounds/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_acosh/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_clip_outbounds_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_celu/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_l2_negative_axes_keep_dims_random/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_greater_equal_bcast_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_prod_do_not_keepdims_example/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_nllloss_NCd1d2d3_sum_weight_high_ii_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_mean_no_weight_ii_4d/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_clip_splitbounds_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_mean_weight_ii_4d_log_prob_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_concat_2d_axis_0/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_max_do_not_keepdims_random/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_logsoftmax_large_number_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_none/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_scatter_with_axis/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_sum_empty_axes_input_noop_example/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_sum_square_default_axes_keepdims_example_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_slice_default_axes/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_range_float_type_positive_delta_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_constantofshape_float_ones/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_pow_types_float32_uint64/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_xor3d/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_bitwise_not_3d/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_nonmaxsuppression_two_batches/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_mean_weight_ii_4d_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_clip_inbounds/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_logsoftmax_example_1/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_bitwise_not_2d/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reshape_reduced_dims/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_layer_normalization_default_axis_expanded_ver18/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_range_int32_type_negative_delta_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_logsoftmax_negative_axis/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_split_variable_parts_default_axis_opset13/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_log_sum_exp_negative_axes_keepdims_example_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_cast_BFLOAT16_to_FLOAT/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_gather_2d_indices/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_center_crop_pad_crop_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_max_float16/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_mul/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_argmax_negative_axis_keepdims_random/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_convtranspose_kernel_shape/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_convtranspose/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_NCd1d2d3_none_no_weight_negative_ii_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_transpose_all_permutations_4/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_unsqueeze_two_axes/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_or_bcast3v1d/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_or_bcast3v2d/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_tfidfvectorizer_tf_batch_uniandbigrams_skip5/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_bitwise_xor_i32_2d/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_max_one_input/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_resize_upsample_scales_nearest_axes_3_2/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_softplus_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_reduce_log_sum_negative_axes/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_expand_dim_unchanged/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sequence_map_add_1_sequence_1_tensor/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_mean_weight_log_prob_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_clip_default_int8_max_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_optional_has_element_empty_no_input_name_optional_input/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_NCd1d2d3d4d5_mean_weight_log_prob_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sce_NCd1d2d3_sum_weight_high_ii/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_shape_start_negative_1/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_slice_start_out_of_bounds/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_sequence_insert_at_back/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_nllloss_NCd1_weight_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_argmin_negative_axis_keepdims_random_select_last_index/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_prelu_broadcast/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_training_dropout_zero_ratio/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_logsoftmax_default_axis/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_layer_normalization_4d_axis0_expanded/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_thresholdedrelu_example/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_globalaveragepool/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_argmin_default_axis_example_select_last_index/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/node/test_quantizelinear_axis/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/real/test_bvlc_alexnet/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/real/test_squeezenet/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/real/test_zfnet512/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/real/test_vgg19/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/real/test_inception_v1/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/real/test_shufflenet/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/real/test_densenet121/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/real/test_resnet50/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/real/test_inception_v2/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_strnorm_model_monday_casesensintive_nochangecase/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_strnorm_model_monday_empty_output/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_single_relu_model/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_expand_shape_model2/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_strnorm_model_monday_insensintive_upper_twodim/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_sequence_model6/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_expand_shape_model4/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_sequence_model4/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_expand_shape_model1/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_sign_model/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_sequence_model7/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_sequence_model1/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_sequence_model3/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_shrink/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_strnorm_model_monday_casesensintive_lower/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_gradient_of_add/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_sequence_model8/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_gradient_of_add_and_mul/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_sequence_model2/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_strnorm_model_monday_casesensintive_upper/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_expand_shape_model3/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_sequence_model5/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_strnorm_model_nostopwords_nochangecase/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_strnorm_model_monday_casesensintive_nochangecase/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_strnorm_model_monday_empty_output/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_single_relu_model/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_expand_shape_model2/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_strnorm_model_monday_insensintive_upper_twodim/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_sequence_model6/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_expand_shape_model4/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_sequence_model4/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_expand_shape_model1/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_sign_model/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_sequence_model7/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_sequence_model1/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_sequence_model3/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_shrink/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_strnorm_model_monday_casesensintive_lower/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_gradient_of_add/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_sequence_model8/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_gradient_of_add_and_mul/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_sequence_model2/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_strnorm_model_monday_casesensintive_upper/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_expand_shape_model3/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_sequence_model5/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/backend/test/data/simple/test_strnorm_model_nostopwords_nochangecase/test_data_set_0/__init__.py' not found (or not a regular file)
  package init file 'onnx/test/cpp/__init__.py' not found (or not a regular file)
  package init file 'onnx/defs/reduction/__init__.py' not found (or not a regular file)
  package init file 'onnx/defs/training/__init__.py' not found (or not a regular file)
  package init file 'onnx/defs/nn/__init__.py' not found (or not a regular file)
  package init file 'onnx/defs/optional/__init__.py' not found (or not a regular file)
  package init file 'onnx/defs/rnn/__init__.py' not found (or not a regular file)
  package init file 'onnx/defs/logical/__init__.py' not found (or not a regular file)
  package init file 'onnx/defs/quantization/__init__.py' not found (or not a regular file)
  package init file 'onnx/defs/controlflow/__init__.py' not found (or not a regular file)
  package init file 'onnx/defs/math/__init__.py' not found (or not a regular file)
  package init file 'onnx/defs/traditionalml/__init__.py' not found (or not a regular file)
  package init file 'onnx/defs/tensor/__init__.py' not found (or not a regular file)
  package init file 'onnx/defs/object_detection/__init__.py' not found (or not a regular file)
  package init file 'onnx/defs/sequence/__init__.py' not found (or not a regular file)
  package init file 'onnx/defs/generator/__init__.py' not found (or not a regular file)
  package init file 'onnx/version_converter/adapters/__init__.py' not found (or not a regular file)
  package init file 'onnx/examples/resources/__init__.py' not found (or not a regular file)
  reading manifest file '/tmp/pip-modern-metadata-se6nmzpo/onnx.egg-info/SOURCES.txt'
  Collecting grpcio
  Using cached grpcio-1.48.2-cp36-cp36m-manylinux_2_17_aarch64.whl (63.7 MB)
Requirement already satisfied: matplotlib in /home/nvidia/archiconda3/envs/mmdeploy/lib/python3.6/site-packages (from mmdeploy==1.2.0) (3.3.4)
Requirement already satisfied: mmengine in /home/nvidia/archiconda3/envs/mmdeploy/lib/python3.6/site-packages (from mmdeploy==1.2.0) (0.4.0)
  Link requires a different Python (3.6.9 not in: '>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*, !=3.6.*'): https://files.pythonhosted.org/packages/4b/7f/55f7dacc3127a918110763998dfbabbec8dd693bfde9a7df9c4e44f4da29/multiprocess-0.70.13.tar.gz#sha256=2e096dd618a84d15aa369a9cf6695815e5539f853dc8fa4f4b9153b11b1d0b32 (from https://pypi.org/simple/multiprocess/) (requires-python:>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*, !=3.6.*)
  Link requires a different Python (3.6.9 not in: '>=3.7'): https://files.pythonhosted.org/packages/e3/ab/9aafc121c6a3d2470ccdf28f99897e88d324c948893b30e46cb359f595e3/multiprocess-0.70.14.tar.gz#sha256=3eddafc12f2260d27ae03fe6069b12570ab4764ab59a75e81624fac453fbf46a (from https://pypi.org/simple/multiprocess/) (requires-python:>=3.7)
Collecting multiprocess
  Using cached multiprocess-0.70.12.2-py36-none-any.whl (106 kB)
Requirement already satisfied: numpy in /home/nvidia/archiconda3/envs/mmdeploy/lib/python3.6/site-packages (from mmdeploy==1.2.0) (1.19.5)
Collecting onnx>=1.13.0
  Using cached onnx-1.14.0.tar.gz (11.3 MB)

  reading manifest template 'MANIFEST.in'
  warning: no files found matching '*.c' under directory 'onnx'
  adding license file 'LICENSE'
  writing manifest file '/tmp/pip-modern-metadata-se6nmzpo/onnx.egg-info/SOURCES.txt'
  creating '/tmp/pip-modern-metadata-se6nmzpo/onnx.dist-info'
  adding license file "LICENSE" (matched pattern "LICENSE")
  Preparing metadata (pyproject.toml) ... done
INFO: pip is looking at multiple versions of mmdeploy to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install mmdeploy and mmdeploy==1.2.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    mmdeploy 1.2.0 depends on protobuf<=3.20.2
    onnx 1.14.0 depends on protobuf>=3.20.2
    mmdeploy 1.2.0 depends on protobuf<=3.20.2
    onnx 1.13.1 depends on protobuf<4 and >=3.20.2
    mmdeploy 1.2.0 depends on protobuf<=3.20.2
    onnx 1.13.0 depends on protobuf<4 and >=3.20.2

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

[azuryl]

https://github.com/open-mmlab/mmdeploy/blob/main/requirements/runtime.txt#L7 onnx>=1.13.0 is conflict with the https://github.com/open-mmlab/mmdeploy/blob/main/docs/en/01-how-to-build/jetsons.md pip install onnx==1.10.0

[RunningLeon]

hi, could install protobuf==3.20.2 at first.

[azuryl]

hi, could install protobuf==3.20.2 at first.

I change https://github.com/open-mmlab/mmdeploy/blob/main/requirements/runtime.txt#L7 onnx>=1.10.0 the error is fixed but another problem occour

/data/azuryl/mmdeploy/csrc/mmdeploy/net/trt/trtnet.cpp:159:13: error: ‘class nvinfer1::IExecutionContext’ has no member named ‘setOptimizationProfileAsync’; did you mean ‘setOptimizationProfile’? context->setOptimizationProfileAsync(0, static_cast(stream_.GetNative())); ^~~~~~~ setOptimizationProfile

I checked https://github.com/open-mmlab/mmdeploy/pull/133 (It seems you not fixed)

my TRT version is 7.1.3.0 I changed to

if NV_TENSORRT_MAJOR > 7

context_->setOptimizationProfileAsync(0, static_cast(stream_.GetNative()));

else

context_->setOptimizationProfile(0)

endif

[azuryl]

hi, could install protobuf==3.20.2 at first.

I change https://github.com/open-mmlab/mmdeploy/blob/main/requirements/runtime.txt#L7 onnx>=1.10.0 the error is fixed but another problem occour

/data/azuryl/mmdeploy/csrc/mmdeploy/net/trt/trtnet.cpp:159:13: error: ‘class nvinfer1::IExecutionContext’ has no member named ‘setOptimizationProfileAsync’; did you mean ‘setOptimizationProfile’? context->setOptimizationProfileAsync(0, static_cast(stream_.GetNative())); ^~~~~~~ setOptimizationProfile

I checked #133 (It seems you not fixed)

my TRT version is 7.1.3.0 I changed to #if NV_TENSORRTMAJOR > 7 context->setOptimizationProfileAsync(0, static_cast(stream.GetNative())); #else context->setOptimizationProfile(0) #end if

after I fixed TRT version there is another problem

(mmdeploy_clone) nvidia@xavier0:/data/azuryl/mmdeploy$ python ./tools/deploy.py configs/mmdet/detection/detection_tensorrt_dynamic-320x320-1344x1344.py /data/azuryl/mmdetection/configs/retinanet/retinanet_r18_fpn_1x_coco.py retinanet_r18_fpn_1x_coco_20220407_171055-614fd399.pth $PATH_TO_MMDET/demo/demo.jpg --work-dir work_dir --show --device cuda:0 --dump-info Traceback (most recent call last): File "./tools/deploy.py", line 335, in main() File "./tools/deploy.py", line 134, in main device=args.device) File "/data/azuryl/mmdeploy/mmdeploy/backend/sdk/export_info.py", line 352, in export2SDK deploy_info = get_deploy(deploy_cfg, model_cfg, work_dir, device) File "/data/azuryl/mmdeploy/mmdeploy/backend/sdk/export_info.py", line 268, in get_deploy deploy_cfg, model_cfg, work_dir=work_dir, device=device) File "/data/azuryl/mmdeploy/mmdeploy/backend/sdk/export_info.py", line 62, in get_model_name_customs model_cfg=model_cfg, deploy_cfg=deploy_cfg, device=device) File "/data/azuryl/mmdeploy/mmdeploy/apis/utils/utils.py", line 46, in build_task_processor import_codebase(codebase_type, custom_module_list) File "/data/azuryl/mmdeploy/mmdeploy/codebase/init.py", line 37, in import_codebase codebase.register_all_modules() File "/data/azuryl/mmdeploy/mmdeploy/codebase/mmdet/deploy/object_detection.py", line 36, in register_all_modules from mmdet.utils.setup_env import register_all_modules File "/data/azuryl/mmdetection/mmdet/init.py", line 23, in f'MMEngine=={mmengine.version} is used but incompatible. ' \ AssertionError: MMEngine==0.4.0 is used but incompatible. Please install mmengine>=0.7.1, <1.0.0.

[azuryl]

anyway it seems your readme is not for jetpack 4.5.1 python3.6.9?

in https://github.com/open-mmlab/mmdetection/releases MMDetection V2.28.0 Release Deprecate the support of Python 3.6

[RunningLeon]

@azuryl hi you could try

1. use mmdet prior to v2.28.0
2. Or modify the setup.py and install from source

[azuryl]

@azuryl hi you could try

1. use mmdet prior to v2.28.0
2. Or modify the setup.py and install from source

method 2 will cause python version conflict error

I just try In mothod 1

https://mmcv.readthedocs.io/en/2.x/get_started/installation.html which have introduced

pip install mmcv==2.0.0rc1 -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.10/index.html

https://github.com/open-mmlab/mmdetection/blob/main/docs/en/notes/faq.md

MMDetection version	MMCV version	MMEngine version
main	mmcv>=2.0.0, <2.1.0	mmengine>=0.7.1, <1.0.0
3.1.0	mmcv>=2.0.0, <2.1.0	mmengine>=0.7.1, <1.0.0
3.0.0	mmcv>=2.0.0, <2.1.0	mmengine>=0.7.1, <1.0.0
3.0.0rc6	mmcv>=2.0.0rc4, <2.1.0	mmengine>=0.6.0, <1.0.0
**3.0.0rc5	mmcv>=2.0.0rc1, <2.1.0	mmengine>=0.3.0, <1.0.0**
3.0.0rc4	mmcv>=2.0.0rc1, <2.1.0	mmengine>=0.3.0, <1.0.0
3.0.0rc3	mmcv>=2.0.0rc1, <2.1.0	mmengine>=0.3.0, <1.0.0
3.0.0rc2	mmcv>=2.0.0rc1, <2.1.0	mmengine>=0.1.0, <1.0.0
3.0.0rc1	mmcv>=2.0.0rc1, <2.1.0	mmengine>=0.1.0, <1.0.0
3.0.0rc0	mmcv>=2.0.0rc1, <2.1.0	mmengine>=0.1.0, <1.0.

MMDetection select 3.0.0rc5 mmdeploy select v1.0.0rc3 they are all support 3.6.9

however error [TensorRT] INFO: Detected 1 inputs and 2 output network tensors. 07/11 16:46:56 - mmengine - INFO - Finish pipeline mmdeploy.apis.utils.utils.to_backend 07/11 16:46:58 - mmengine - INFO - visualize tensorrt model start. 07/11 16:47:11 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. 07/11 16:47:11 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. 07/11 16:47:11 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. 07/11 16:47:11 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "backend_detectors" registry tree. As a workaround, the current "backend_detectors" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized. 07/11 16:47:11 - mmengine - WARNING - The "model" registry in mmdet did not set import location. Fallback to call mmdet.utils.register_all_modules instead. 07/11 16:47:11 - mmengine - INFO - Successfully loaded tensorrt plugins from /data/azuryl/mmdeploy_v1.0.0rc3/mmdeploy/lib/libmmdeploy_tensorrt_ops.so 2023-07-11:16:47:11 - root - ERROR - class End2EndModel in mmdeploy/codebase/mmdet/deploy/object_detection_model.py: TorchAllocator: No constructor defined! Traceback (most recent call last): File "/home/nvidia/archiconda3/envs/mmdeploy/lib/python3.6/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg obj = obj_cls(args) # type: ignore File "/data/azuryl/mmdeploy_v1.0.0rc3/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 54, in init backend=backend, backend_files=backend_files, device=device) File "/data/azuryl/mmdeploy_v1.0.0rc3/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 73, in _init_wrapper deploy_cfg=self.deploy_cfg) File "/data/azuryl/mmdeploy_v1.0.0rc3/mmdeploy/codebase/base/backend_model.py", line 66, in _build_wrapper output_names, deploy_cfg, kwargs) File "/data/azuryl/mmdeploy_v1.0.0rc3/mmdeploy/backend/tensorrt/backend_manager.py", line 34, in build_wrapper return TRTWrapper(engine=backend_files[0], output_names=output_names) File "/data/azuryl/mmdeploy_v1.0.0rc3/mmdeploy/backend/tensorrt/wrapper.py", line 85, in init self.allocator = TorchAllocator(device_id) File "/data/azuryl/mmdeploy_v1.0.0rc3/mmdeploy/backend/tensorrt/torch_allocator.py", line 12, in init super().init() TypeError: TorchAllocator: No constructor defined!

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/data/azuryl/mmdeploy_v1.0.0rc3/mmdeploy/utils/utils.py", line 41, in target_wrapper result = target(args, kwargs) File "/data/azuryl/mmdeploy_v1.0.0rc3/mmdeploy/apis/visualize.py", line 67, in visualize_model data_preprocessor_updater=task_processor. File "/data/azuryl/mmdeploy_v1.0.0rc3/mmdeploy/codebase/mmdet/deploy/object_detection.py", line 162, in build_backend_model data_preprocessor=data_preprocessor) File "/data/azuryl/mmdeploy_v1.0.0rc3/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 890, in build_object_detection_model kwargs)) File "/home/nvidia/archiconda3/envs/mmdeploy/lib/python3.6/site-packages/mmengine/registry/registry.py", line 521, in build return self.build_func(cfg, args, **kwargs, registry=self) File "/home/nvidia/archiconda3/envs/mmdeploy/lib/python3.6/site-packages/mmengine/registry/build_functions.py", line 136, in build_from_cfg f'class {obj_cls.__name__} in ' # type: ignore TypeError: class End2EndModel in mmdeploy/codebase/mmdet/deploy/object_detection_model.py: TorchAllocator: No constructor defined! Exception ignored in: <bound method TorchAllocator.del of <mmdeploy.backend.tensorrt.torch_allocator.TorchAllocator object at 0x7f07ffb990>> Traceback (most recent call last): File "/data/azuryl/mmdeploy_v1.0.0rc3/mmdeploy/backend/tensorrt/torch_allocator.py", line 20, in del mems = self.mems.copy() AttributeError: 'TorchAllocator' object has no attribute 'mems' 07/11 16:47:13 - mmengine - ERROR - ./tools/deploy.py - create_process - 82 - visualize tensorrt model failed.

@RunningLeon it seems have no relationship with mmdet, it is mmdeploy's problem

[azuryl]

hi, could install protobuf==3.20.2 ,at first.

pip insatall protobuf==3.20.2 is not available ,because pip have not that version,you can try it

[RunningLeon]

hi, pls. rerun python tools/check_env.py to show your torch version and other env info.

[RunningLeon]

@azuryl hi, just tested on my machine by changing the python_requires='>=3.6' in setup.py and installing mmcv, mmengine, mmdetection, mmdeploy from source. And the result was OK. Maybe you could align the env to it.

jetson_release

 - Jetson-AGX
   * Jetpack 4.6 [L4T 32.6.1]
   * NV Power Mode: MAXN - Type: 0
   * jetson_stats.service: inactive
 - Libraries:
   * CUDA: 10.2.300
   * cuDNN: 8.2.1.32
   * TensorRT: 8.0.1.6
   * Visionworks: 1.6.0.501
   * OpenCV: 4.1.1 compiled CUDA: NO
   * VPI: 1.1.12
   * Vulkan: 1.2.70

check env

07/12 13:55:35 - mmengine - INFO - 

07/12 13:55:35 - mmengine - INFO - **********Environmental information**********
jet07/12 13:55:39 - mmengine - INFO - sys.platform: linux
07/12 13:55:39 - mmengine - INFO - Python: 3.6.15 | packaged by conda-forge | (default, Dec  3 2021, 19:12:04) [GCC 9.4.0]
07/12 13:55:39 - mmengine - INFO - CUDA available: True
07/12 13:55:39 - mmengine - INFO - numpy_random_seed: 2147483648
07/12 13:55:39 - mmengine - INFO - GPU 0: Xavier
07/12 13:55:39 - mmengine - INFO - CUDA_HOME: /usr/local/cuda-10.2
07/12 13:55:39 - mmengine - INFO - NVCC: Cuda compilation tools, release 10.2, V10.2.300
07/12 13:55:39 - mmengine - INFO - GCC: gcc (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0
07/12 13:55:39 - mmengine - INFO - PyTorch: 1.10.0
07/12 13:55:39 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - GCC 7.5
  - C++ Version: 201402
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: NO AVX
  - CUDA Runtime 10.2
  - NVCC architecture flags: -gencode;arch=compute_53,code=sm_53;-gencode;arch=compute_62,code=sm_62;-gencode;arch=compute_72,code=sm_72
  - CuDNN 8.2.1
    - Built with CuDNN 8.0
  - Build settings: BLAS_INFO=open, BUILD_TYPE=Release, CUDA_VERSION=10.2, CUDNN_VERSION=8.0.0, CXX_COMPILER=/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -DMISSING_ARM_VST1 -DMISSING_ARM_VLD1 -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=open, TORCH_VERSION=1.10.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EIGEN_FOR_BLAS=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=OFF, USE_MKLDNN=OFF, USE_MPI=ON, USE_NCCL=0, USE_NNPACK=ON, USE_OPENMP=ON, 

07/12 13:55:39 - mmengine - INFO - TorchVision: 0.11.0a0+fa347eb
07/12 13:55:39 - mmengine - INFO - OpenCV: 4.7.0
07/12 13:55:39 - mmengine - INFO - MMEngine: 0.8.2
07/12 13:55:39 - mmengine - INFO - MMCV: 2.0.1
07/12 13:55:39 - mmengine - INFO - MMCV Compiler: GCC 7.5
07/12 13:55:39 - mmengine - INFO - MMCV CUDA Compiler: 10.2
07/12 13:55:39 - mmengine - INFO - MMDeploy: 1.0.0rc3+0a8cbe2
07/12 13:55:39 - mmengine - INFO - 

07/12 13:55:39 - mmengine - INFO - **********Backend information**********
07/12 13:55:39 - mmengine - INFO - tensorrt:    8.0.1.6
07/12 13:55:39 - mmengine - INFO - tensorrt custom ops: Available
07/12 13:55:39 - mmengine - INFO - ONNXRuntime: None
07/12 13:55:39 - mmengine - INFO - ONNXRuntime-gpu: 1.11.0
07/12 13:55:39 - mmengine - INFO - ONNXRuntime custom ops:  NotAvailable
07/12 13:55:39 - mmengine - INFO - pplnn:   None
07/12 13:55:39 - mmengine - INFO - ncnn:    None
07/12 13:55:39 - mmengine - INFO - snpe:    None
07/12 13:55:39 - mmengine - INFO - openvino:    None
07/12 13:55:39 - mmengine - INFO - torchscript: 1.10.0
07/12 13:55:39 - mmengine - INFO - torchscript custom ops:  NotAvailable
07/12 13:55:39 - mmengine - INFO - rknn-toolkit:    None
07/12 13:55:39 - mmengine - INFO - rknn-toolkit2:   None
07/12 13:55:39 - mmengine - INFO - ascend:  None
07/12 13:55:39 - mmengine - INFO - coreml:  None
07/12 13:55:41 - mmengine - INFO - tvm: 0.9.dev0
07/12 13:55:41 - mmengine - INFO - vacc:    None
07/12 13:55:41 - mmengine - INFO - 

07/12 13:55:41 - mmengine - INFO - **********Codebase information**********
07/12 13:55:41 - mmengine - INFO - mmdet:   3.1.0
07/12 13:55:41 - mmengine - INFO - mmseg:   1.0.0rc3
07/12 13:55:41 - mmengine - INFO - mmcls:   None
07/12 13:55:41 - mmengine - INFO - mmocr:   None
07/12 13:55:41 - mmengine - INFO - mmedit:  1.0.0rc5
07/12 13:55:41 - mmengine - INFO - mmdet3d: None
07/12 13:55:41 - mmengine - INFO - mmpose:  1.0.0rc1
07/12 13:55:41 - mmengine - INFO - mmrotate:    1.0.0rc1
07/12 13:55:41 - mmengine - INFO - mmaction:    1.0.0rc1

ran script

python3 ./tools/deploy.py \
configs/mmdet/detection/detection_tensorrt_dynamic-320x320-1344x1344.py \
../mmdetection/configs/retinanet/retinanet_r50_fpn_1x_coco.py \
"../mmdeploy_checkpoints/mmdet/retinanet/retinanet_r50_fpn_1x_coco_20200130-c2398f9e.pth" \
"../mmdetection/demo/demo.jpg" \
--work-dir "../mmdeploy_regression_working_dir/mmdet/retinanet/tensorrt/dynamic/fp32/retinanet_r50_fpn_1x_coco_20200130-c2398f9e" \
--device cuda:0 \
--log-level INFO \
--test-img ./tests/data/tiger.jpeg

[azuryl]

@azuryl hi, just tested on my machine by changing the python_requires='>=3.6' in setup.py and installing mmcv, mmengine, mmdetection, mmdeploy from source. And the result was OK. Maybe you could align the env to it.

jetson_release

 - Jetson-AGX
   * Jetpack 4.6 [L4T 32.6.1]
   * NV Power Mode: MAXN - Type: 0
   * jetson_stats.service: inactive
 - Libraries:
   * CUDA: 10.2.300
   * cuDNN: 8.2.1.32
   * TensorRT: 8.0.1.6
   * Visionworks: 1.6.0.501
   * OpenCV: 4.1.1 compiled CUDA: NO
   * VPI: 1.1.12
   * Vulkan: 1.2.70

check env

07/12 13:55:35 - mmengine - INFO - 

07/12 13:55:35 - mmengine - INFO - **********Environmental information**********
jet07/12 13:55:39 - mmengine - INFO - sys.platform: linux
07/12 13:55:39 - mmengine - INFO - Python: 3.6.15 | packaged by conda-forge | (default, Dec  3 2021, 19:12:04) [GCC 9.4.0]
07/12 13:55:39 - mmengine - INFO - CUDA available: True
07/12 13:55:39 - mmengine - INFO - numpy_random_seed: 2147483648
07/12 13:55:39 - mmengine - INFO - GPU 0: Xavier
07/12 13:55:39 - mmengine - INFO - CUDA_HOME: /usr/local/cuda-10.2
07/12 13:55:39 - mmengine - INFO - NVCC: Cuda compilation tools, release 10.2, V10.2.300
07/12 13:55:39 - mmengine - INFO - GCC: gcc (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0
07/12 13:55:39 - mmengine - INFO - PyTorch: 1.10.0
07/12 13:55:39 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - GCC 7.5
  - C++ Version: 201402
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: NO AVX
  - CUDA Runtime 10.2
  - NVCC architecture flags: -gencode;arch=compute_53,code=sm_53;-gencode;arch=compute_62,code=sm_62;-gencode;arch=compute_72,code=sm_72
  - CuDNN 8.2.1
    - Built with CuDNN 8.0
  - Build settings: BLAS_INFO=open, BUILD_TYPE=Release, CUDA_VERSION=10.2, CUDNN_VERSION=8.0.0, CXX_COMPILER=/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -DMISSING_ARM_VST1 -DMISSING_ARM_VLD1 -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=open, TORCH_VERSION=1.10.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EIGEN_FOR_BLAS=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=OFF, USE_MKLDNN=OFF, USE_MPI=ON, USE_NCCL=0, USE_NNPACK=ON, USE_OPENMP=ON, 

07/12 13:55:39 - mmengine - INFO - TorchVision: 0.11.0a0+fa347eb
07/12 13:55:39 - mmengine - INFO - OpenCV: 4.7.0
07/12 13:55:39 - mmengine - INFO - MMEngine: 0.8.2
07/12 13:55:39 - mmengine - INFO - MMCV: 2.0.1
07/12 13:55:39 - mmengine - INFO - MMCV Compiler: GCC 7.5
07/12 13:55:39 - mmengine - INFO - MMCV CUDA Compiler: 10.2
07/12 13:55:39 - mmengine - INFO - MMDeploy: 1.0.0rc3+0a8cbe2
07/12 13:55:39 - mmengine - INFO - 

07/12 13:55:39 - mmengine - INFO - **********Backend information**********
07/12 13:55:39 - mmengine - INFO - tensorrt:  8.0.1.6
07/12 13:55:39 - mmengine - INFO - tensorrt custom ops:   Available
07/12 13:55:39 - mmengine - INFO - ONNXRuntime:   None
07/12 13:55:39 - mmengine - INFO - ONNXRuntime-gpu:   1.11.0
07/12 13:55:39 - mmengine - INFO - ONNXRuntime custom ops:    NotAvailable
07/12 13:55:39 - mmengine - INFO - pplnn: None
07/12 13:55:39 - mmengine - INFO - ncnn:  None
07/12 13:55:39 - mmengine - INFO - snpe:  None
07/12 13:55:39 - mmengine - INFO - openvino:  None
07/12 13:55:39 - mmengine - INFO - torchscript:   1.10.0
07/12 13:55:39 - mmengine - INFO - torchscript custom ops:    NotAvailable
07/12 13:55:39 - mmengine - INFO - rknn-toolkit:  None
07/12 13:55:39 - mmengine - INFO - rknn-toolkit2: None
07/12 13:55:39 - mmengine - INFO - ascend:    None
07/12 13:55:39 - mmengine - INFO - coreml:    None
07/12 13:55:41 - mmengine - INFO - tvm:   0.9.dev0
07/12 13:55:41 - mmengine - INFO - vacc:  None
07/12 13:55:41 - mmengine - INFO - 

07/12 13:55:41 - mmengine - INFO - **********Codebase information**********
07/12 13:55:41 - mmengine - INFO - mmdet: 3.1.0
07/12 13:55:41 - mmengine - INFO - mmseg: 1.0.0rc3
07/12 13:55:41 - mmengine - INFO - mmcls: None
07/12 13:55:41 - mmengine - INFO - mmocr: None
07/12 13:55:41 - mmengine - INFO - mmedit:    1.0.0rc5
07/12 13:55:41 - mmengine - INFO - mmdet3d:   None
07/12 13:55:41 - mmengine - INFO - mmpose:    1.0.0rc1
07/12 13:55:41 - mmengine - INFO - mmrotate:  1.0.0rc1
07/12 13:55:41 - mmengine - INFO - mmaction:  1.0.0rc1

ran script

python3 ./tools/deploy.py \
configs/mmdet/detection/detection_tensorrt_dynamic-320x320-1344x1344.py \
../mmdetection/configs/retinanet/retinanet_r50_fpn_1x_coco.py \
"../mmdeploy_checkpoints/mmdet/retinanet/retinanet_r50_fpn_1x_coco_20200130-c2398f9e.pth" \
"../mmdetection/demo/demo.jpg" \
--work-dir "../mmdeploy_regression_working_dir/mmdet/retinanet/tensorrt/dynamic/fp32/retinanet_r50_fpn_1x_coco_20200130-c2398f9e" \
--device cuda:0 \
--log-level INFO \
--test-img ./tests/data/tiger.jpeg

my jetpack is Jetpack 4.5.1 python 3.6.9 since my device have another project on this version, so I can't update to jetpack 4.6

I had rebuild my enviorment python -m mmdet.utils.collect_env /home/nvidia/archiconda3/envs/mmdeploy/lib/python3.6/runpy.py:125: RuntimeWarning: 'mmdet.utils.collect_env' found in sys.modules after import of package 'mmdet.utils', but prior to execution of 'mmdet.utils.collect_env'; this may result in unpredictable behaviour warn(RuntimeWarning(msg)) fatal: not a git repository (or any of the parent directories): .git sys.platform: linux Python: 3.6.9 | packaged by conda-forge | (default, Mar 6 2020, 18:58:41) [GCC 7.3.0] CUDA available: True GPU 0: Xavier CUDA_HOME: /usr/local/cuda-10.2 NVCC: Cuda compilation tools, release 10.2, V10.2.89 GCC: gcc (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0 PyTorch: 1.10.0 PyTorch compiling details: PyTorch built with:

• GCC 7.5
• C++ Version: 201402
• OpenMP 201511 (a.k.a. OpenMP 4.5)
• LAPACK is enabled (usually provided by MKL)
• NNPACK is enabled
• CPU capability usage: NO AVX
• CUDA Runtime 10.2
• NVCC architecture flags: -gencode;arch=compute_53,code=sm_53;-gencode;arch=compute_62,code=sm_62;-gencode;arch=compute_72,code=sm_72
• CuDNN 8.0
• Build settings: BLAS_INFO=open, BUILD_TYPE=Release, CUDA_VERSION=10.2, CUDNN_VERSION=8.0.0, CXX_COMPILER=/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -DMISSING_ARM_VST1 -DMISSING_ARM_VLD1 -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=open, TORCH_VERSION=1.10.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EIGEN_FOR_BLAS=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=OFF, USE_MKLDNN=OFF, USE_MPI=ON, USE_NCCL=0, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.11.1 OpenCV: 4.8.0 MMCV: 1.3.17 MMCV Compiler: GCC 7.5 MMCV CUDA Compiler: 10.2 MMDetection: 2.27.0+

python ./tools/deploy.py configs/mmdet/detection/detection_tensorrt_dynamic-320x320-1344x1344.py /data/azuryl/mmdetection_2.27.0/configs/retinanet/retinanet_r18_fpn_1x_coco.py /data/azuryl/retinanet_r18_fpn_1x_coco_20220407_171055-614fd399.pth /data/azuryl/mmdetection_2.27.0/demo/demo.jpg --work-dir work_dir --show --device cuda:0 --dump-info

[azuryl]

but when I run python ./tools/deploy.py configs/mmdet/detection/detection_tensorrt_dynamic-320x320-1344x1344.py /data/azuryl/mmdetection_2.27.0/configs/mask_rcnn/mask_rcnn_r50_fpn_2x_coco.py /data/azuryl/mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392__segm_mAP-0.354_20200505_003907-3e542a40.pth /data/azuryl/mmdetection_2.27.0/demo/demo.jpg --work-dir work_dir --show --device cuda:0 --dump-info

there have problem [TensorRT] INFO: builtin_op_importers.cpp:3659: Searching for plugin: GatherTopk, plugin_version: 1, plugin_namespace: [TensorRT] INFO: builtin_op_importers.cpp:3676: Successfully created plugin: GatherTopk [TensorRT] INFO: ModelImporter.cpp:135: No importer registered for op: TRTBatchedNMS. Attempting to import as plugin. [TensorRT] INFO: builtin_op_importers.cpp:3659: Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace: [TensorRT] INFO: builtin_op_importers.cpp:3676: Successfully created plugin: TRTBatchedNMS [TensorRT] INFO: ModelImporter.cpp:135: No importer registered for op: MMCVMultiLevelRoiAlign. Attempting to import as plugin. [TensorRT] INFO: builtin_op_importers.cpp:3659: Searching for plugin: MMCVMultiLevelRoiAlign, plugin_version: 1, plugin_namespace: [TensorRT] INFO: builtin_op_importers.cpp:3676: Successfully created plugin: MMCVMultiLevelRoiAlign [TensorRT] WARNING: Tensor DataType is determined at build time for tensors not marked as input or output. [TensorRT] INFO: ModelImporter.cpp:135: No importer registered for op: TRTBatchedNMS. Attempting to import as plugin. [TensorRT] INFO: builtin_op_importers.cpp:3659: Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace: [TensorRT] INFO: builtin_op_importers.cpp:3676: Successfully created plugin: TRTBatchedNMS [TensorRT] INFO: ModelImporter.cpp:135: No importer registered for op: MMCVMultiLevelRoiAlign. Attempting to import as plugin. [TensorRT] INFO: builtin_op_importers.cpp:3659: Searching for plugin: MMCVMultiLevelRoiAlign, plugin_version: 1, plugin_namespace: [TensorRT] INFO: builtin_op_importers.cpp:3676: Successfully created plugin: MMCVMultiLevelRoiAlign [TensorRT] INFO: ModelImporter.cpp:135: No importer registered for op: grid_sampler. Attempting to import as plugin. [TensorRT] INFO: builtin_op_importers.cpp:3659: Searching for plugin: grid_sampler, plugin_version: 1, plugin_namespace: [TensorRT] INFO: builtin_op_importers.cpp:3676: Successfully created plugin: grid_sampler [TensorRT] WARNING: Output type must be INT32 for shape outputs [TensorRT] WARNING: Output type must be INT32 for shape outputs [TensorRT] WARNING: DLA requests all profiles have same min, max, and opt value. All dla layers are falling back to GPU [TensorRT] INFO: [TensorRT] INFO: --------------- Layers running on DLA: [TensorRT] INFO: [TensorRT] INFO: --------------- Layers running on GPU: [TensorRT] INFO: Range_968, Range_938, Range_891, Range_780, Range_200, Range_204, Range_239, Range_243, Range_274, Range_278, Range_344, Range_348, Range_309, Range_313, Range_984, Range_988, Conv_9 + Relu_10, [HostToDeviceCopy], (Unnamed Layer 211) [Shuffle]_output[Constant], Mul_205, Reshape_218 + (Unnamed Layer 301) [Shuffle], Expand_225, Tile_226, (Unnamed Layer 362) [Shuffle]_output[Constant], Mul_244, Reshape_255 + (Unnamed Layer 452) [Shuffle], Expand_262, Tile_263, (Unnamed Layer 513) [Shuffle]_output[Constant], Mul_279, Reshape_290 + (Unnamed Layer 603) [Shuffle], Expand_297, Tile_298, (Unnamed Layer 815) [Shuffle]_output[Constant], Mul_349, Reshape_360 + (Unnamed Layer 905) [Shuffle], Expand_367, Tile_368, (Unnamed Layer 664) [Shuffle]_output[Constant], Mul_314, Reshape_325 + (Unnamed Layer 754) [Shuffle], Expand_332, Tile_333, MaxPool_11, Conv_12 + Relu_13, Reshape_334 + Unsqueeze_336, Reshape_369 + Unsqueeze_371, Reshape_299 + Unsqueeze_301, Reshape_264 + Unsqueeze_266, Reshape_228 + Unsqueeze_230, (Unnamed Layer 350) [Constant], Mul_202, Unsqueeze_209 + (Unnamed Layer 249) [Shuffle], Expand_214, Tile_215, Squeeze_216 + Unsqueeze_229, 708 copy, 709 copy, 708 copy, 709 copy, Cast_232, Unsqueeze_233, Add_235, (Unnamed Layer 501) [Constant], Mul_241, Unsqueeze_247 + (Unnamed Layer 400) [Shuffle], Expand_252, Tile_253, Squeeze_254 + Unsqueeze_265, 769 copy, 770 copy, 769 copy, 770 copy, Cast_268, Unsqueeze_269, Add_271, (Unnamed Layer 652) [Constant], Mul_276, Unsqueeze_282 + (Unnamed Layer 551) [Shuffle], Expand_287, Tile_288, Squeeze_289 + Unsqueeze_300, 830 copy, 831 copy, 830 copy, 831 copy, Cast_303, Unsqueeze_304, Add_306, (Unnamed Layer 954) [Constant], Mul_346, Unsqueeze_352 + (Unnamed Layer 853) [Shuffle], Expand_357, Tile_358, Squeeze_359 + Unsqueeze_370, 952 copy, 953 copy, 952 copy, 953 copy, Cast_373, Unsqueeze_374, Add_376, (Unnamed Layer 803) [Constant], Mul_311, Unsqueeze_317 + (Unnamed Layer 702) [Shuffle], Expand_322, Tile_323, Squeeze_324 + Unsqueeze_335, 891 copy, 892 copy, 891 copy, 892 copy, Cast_338, Unsqueeze_339, Add_341, Conv_14 + Relu_15, Reshape_342 + Unsqueeze_581, Reshape_377 + Unsqueeze_644, Reshape_307 + Unsqueeze_518, Reshape_272 + Unsqueeze_455, Reshape_237 + Unsqueeze_389, Conv_16, Conv_17 + Add_18 + Relu_19, Conv_20 + Relu_21, Conv_22 + Relu_23, Conv_24 + Add_25 + Relu_26, Conv_27 + Relu_28, Conv_29 + Relu_30, Conv_31 + Add_32 + Relu_33, Conv_34 + Relu_35, Conv_36 + Relu_37, Conv_38, Conv_39 + Add_40 + Relu_41, Conv_42 + Relu_43, Conv_44 + Relu_45, Conv_46 + Add_47 + Relu_48, Conv_49 + Relu_50, Conv_51 + Relu_52, Conv_53 + Add_54 + Relu_55, Conv_56 + Relu_57, Conv_58 + Relu_59, Conv_60 + Add_61 + Relu_62, Conv_63 + Relu_64, Conv_65 + Relu_66, Conv_67, Conv_68 + Add_69 + Relu_70, Conv_71 + Relu_72, Conv_73 + Relu_74, Conv_75 + Add_76 + Relu_77, Conv_78 + Relu_79, Conv_80 + Relu_81, Conv_82 + Add_83 + Relu_84, Conv_85 + Relu_86, Conv_87 + Relu_88, Conv_89 + Add_90 + Relu_91, Conv_92 + Relu_93, Conv_94 + Relu_95, Conv_96 + Add_97 + Relu_98, Conv_99 + Relu_100, Conv_101 + Relu_102, Conv_103 + Add_104 + Relu_105, Conv_106 + Relu_107, Conv_108 + Relu_109, Conv_110, Conv_111 + Add_112 + Relu_113, Conv_114 + Relu_115, Conv_116 + Relu_117, Conv_118 + Add_119 + Relu_120, Conv_121 + Relu_122, Conv_123 + Relu_124, Conv_125 + Add_126 + Relu_127, Conv_131, Resize_139, Conv_160, Conv_174 + Relu_175, MaxPool_161, Conv_178 + Relu_179, Conv_130 + Add_140, Conv_181 || Conv_180, Transpose_642 + Reshape_643, Transpose_638 + Reshape_639, Conv_177 || Conv_176, Transpose_579 + Reshape_580, Transpose_575 + Reshape_576, Resize_147, Conv_159, Conv_170 + Relu_171, Sigmoid_577, Reshape_578, (Unnamed Layer 1313) [Constant] + (Unnamed Layer 1314) [Shuffle], ConstantOfShape_588, PWN((Unnamed Layer 1317) [Constant] + (Unnamed Layer 1318) [Shuffle], Add_589), 1315 copy, 1338 copy, Squeeze_607, Sigmoid_640, Reshape_641, (Unnamed Layer 1421) [Constant] + (Unnamed Layer 1422) [Shuffle], ConstantOfShape_651, PWN((Unnamed Layer 1425) [Constant] + (Unnamed Layer 1426) [Shuffle], Add_652), 1427 copy, 1450 copy, Squeeze_670, Conv_129 + Add_148, Conv_173 || Conv_172, Transpose_516 + Reshape_517, Transpose_512 + Reshape_513, Resize_155, Conv_158, Conv_166 + Relu_167, TopK_671, GatherTopk_684, TopK_608, GatherTopk_621, Sigmoid_514, Reshape_515, (Unnamed Layer 1205) [Constant] + (Unnamed Layer 1206) [Shuffle], ConstantOfShape_525, PWN((Unnamed Layer 1209) [Constant] + (Unnamed Layer 1210) [Shuffle], Add_526), 1203 copy, 1226 copy, Squeeze_544, (Unnamed Layer 1330) [Constant] + (Unnamed Layer 1331) [Shuffle], ConstantOfShape_597, 1323 copy, 1351 copy, (Unnamed Layer 1392) [Constant], (Unnamed Layer 1393) [Shuffle], Mul_632, Add_633, (Unnamed Layer 1344) [Constant] + (Unnamed Layer 1345) [Shuffle], ConstantOfShape_605, 1324 copy, 1364 copy, Flatten_631, (Unnamed Layer 1438) [Constant] + (Unnamed Layer 1439) [Shuffle], ConstantOfShape_660, 1435 copy, 1463 copy, (Unnamed Layer 1500) [Constant], (Unnamed Layer 1501) [Shuffle], Mul_695, Add_696, (Unnamed Layer 1452) [Constant] + (Unnamed Layer 1453) [Shuffle], ConstantOfShape_668, 1436 copy, 1476 copy, Flatten_694, Conv_128 + Add_156, Conv_169 || Conv_168, Transpose_453 + Reshape_454, Transpose_449 + Reshape_450, Conv_157, Conv_162 + Relu_163, Gather_697, GatherTopk_675, Gather_634, GatherTopk_612, TopK_545, GatherTopk_558, Sigmoid_451, Reshape_452, (Unnamed Layer 1097) [Constant] + (Unnamed Layer 1098) [Shuffle], ConstantOfShape_462, PWN((Unnamed Layer 1101) [Constant] + (Unnamed Layer 1102) [Shuffle], Add_463), 1091 copy, 1114 copy, Squeeze_481, (Unnamed Layer 1222) [Constant] + (Unnamed Layer 1223) [Shuffle], ConstantOfShape_534, 1211 copy, 1239 copy, (Unnamed Layer 1284) [Constant], (Unnamed Layer 1285) [Shuffle], Mul_569, Add_570, (Unnamed Layer 1236) [Constant] + (Unnamed Layer 1237) [Shuffle], ConstantOfShape_542, 1212 copy, 1252 copy, Flatten_568, Conv_165 || Conv_164, Transpose_386 + Reshape_388, Transpose_379 + Reshape_382, Gather_571, GatherTopk_549, TopK_482, GatherTopk_495, Sigmoid_383, Reshape_385, (Unnamed Layer 989) [Constant] + (Unnamed Layer 990) [Shuffle], ConstantOfShape_396, PWN((Unnamed Layer 993) [Constant] + (Unnamed Layer 994) [Shuffle], Add_398), 979 copy, 1002 copy, Squeeze_416, (Unnamed Layer 1114) [Constant] + (Unnamed Layer 1115) [Shuffle], ConstantOfShape_471, 1099 copy, 1127 copy, (Unnamed Layer 1176) [Constant], (Unnamed Layer 1177) [Shuffle], Mul_506, Add_507, (Unnamed Layer 1128) [Constant] + (Unnamed Layer 1129) [Shuffle], ConstantOfShape_479, 1100 copy, 1140 copy, Flatten_505, Gather_508, GatherTopk_486, TopK_417, GatherTopk_431, (Unnamed Layer 1006) [Constant] + (Unnamed Layer 1007) [Shuffle], ConstantOfShape_406, 987 copy, 1015 copy, (Unnamed Layer 1068) [Constant], (Unnamed Layer 1069) [Shuffle], Mul_443, Add_444, (Unnamed Layer 1020) [Constant] + (Unnamed Layer 1021) [Shuffle], ConstantOfShape_414, 988 copy, 1028 copy, Flatten_441, Gather_445, GatherTopk_422, (Unnamed Layer 1651) [Constant], 1071 copy, 1183 copy, 1295 copy, 1407 copy, 1519 copy, Unsqueeze_718, Slice_719, Slice_720, (Unnamed Layer 1525) [Constant] + (Unnamed Layer 1526) [Shuffle], (Unnamed Layer 1522) [Constant] + (Unnamed Layer 1523) [Shuffle], Cast_423, Cast_487, Cast_550, Cast_613, Cast_676, 1043 copy, 1155 copy, 1267 copy, 1379 copy, 1491 copy, Reshape_710, PWN(Mul_712, Add_714), Slice_716, Sub_724, PWN(Mul_725, PWN(PWN(Add_721, (Unnamed Layer 1635) [Constant] + (Unnamed Layer 1636) [Shuffle] + Mul_723), Add_730)), Slice_717, PWN(PWN(PWN(Clip_726, Exp_727), Mul_728), (Unnamed Layer 1643) [Constant] + (Unnamed Layer 1644) [Shuffle] + Mul_729), Sub_731, (Unnamed Layer 1649) [Constant], (Unnamed Layer 1655) [Constant], Add_732, (Unnamed Layer 1653) [Constant], Gather_737, Gather_738, Gather_734, Gather_736, Cast_432, Cast_496, Cast_559, Cast_622, Cast_685, 1053 copy, 1165 copy, 1277 copy, 1389 copy, 1501 copy, Cast_751, (Unnamed Layer 1676) [Shuffle], Cast_740, (Unnamed Layer 1660) [Shuffle], PWN(PWN(Div_741, Clip_747), Mul_752), Unsqueeze_757, Cast_753, (Unnamed Layer 1679) [Shuffle], Cast_743, (Unnamed Layer 1665) [Shuffle], PWN(PWN(Div_744, Clip_748), Mul_754), Unsqueeze_758, (Unnamed Layer 1681) [Shuffle], (Unnamed Layer 1667) [Shuffle], PWN(PWN(Div_745, Clip_749), Mul_755), Unsqueeze_759, (Unnamed Layer 1683) [Shuffle], (Unnamed Layer 1669) [Shuffle], PWN(PWN(Div_746, Clip_750), Mul_756), Unsqueeze_760, 1646 copy, 1647 copy, 1648 copy, 1649 copy, TRTBatchedNMS_766, Slice_797, Cast_781, Reshape_783, Expand_795, 1711 copy, 1716 copy, Reshape_802, MMCVMultiLevelRoiAlign_803, (Unnamed Layer 2033) [Constant], Reshape_817, Slice_831, Unsqueeze_844, Slice_845, Slice_846, Sub_849, (Unnamed Layer 1907) [Constant] + (Unnamed Layer 1908) [Shuffle], (Unnamed Layer 1904) [Constant] + (Unnamed Layer 1905) [Shuffle], (Unnamed Layer 1830) [Constant] + (Unnamed Layer 1831) [Shuffle], roi_head.bbox_head.fc_reg.weight, (Unnamed Layer 1819) [Constant] + (Unnamed Layer 1820) [Shuffle], roi_head.bbox_head.shared_fcs.1.weight, (Unnamed Layer 1813) [Constant] + (Unnamed Layer 1814) [Shuffle], roi_head.bbox_head.shared_fcs.0.weight, Flatten_804, Gemm_805, (Unnamed Layer 1815) [ElementWise] + Relu_806, Gemm_807, (Unnamed Layer 1821) [ElementWise] + Relu_808, Gemm_810, (Unnamed Layer 1832) [ElementWise], Reshape_829 + Reshape_838, PWN(Mul_840, Add_841), Slice_842, PWN(Mul_850, PWN(PWN(Add_847, (Unnamed Layer 2017) [Constant] + (Unnamed Layer 2018) [Shuffle] + Mul_848), Add_855)), Slice_843, PWN(PWN(PWN(Clip_851, Exp_852), Mul_853), (Unnamed Layer 2025) [Constant] + (Unnamed Layer 2026) [Shuffle] + Mul_854), Sub_856, (Unnamed Layer 2031) [Constant], (Unnamed Layer 2037) [Constant], Add_857, (Unnamed Layer 2035) [Constant], (Unnamed Layer 1825) [Constant] + (Unnamed Layer 1826) [Shuffle], roi_head.bbox_head.fc_cls.weight, Gemm_809, (Unnamed Layer 1827) [ElementWise], Softmax_830, (Unnamed Layer 1866) [Shuffle], Slice_882, Reshape_884, ArgMax_885, Gather_860, Gather_861, Gather_858, Gather_859, (Unnamed Layer 2051) [Shuffle], (Unnamed Layer 2039) [Shuffle], PWN(PWN(Div_862, Clip_866), Mul_870), Unsqueeze_874, (Unnamed Layer 2053) [Shuffle], (Unnamed Layer 2041) [Shuffle], PWN(PWN(Div_863, Clip_867), Mul_871), Unsqueeze_875, (Unnamed Layer 2055) [Shuffle], (Unnamed Layer 2043) [Shuffle], PWN(PWN(Div_864, Clip_868), Mul_872), Unsqueeze_876, (Unnamed Layer 2057) [Shuffle], (Unnamed Layer 2045) [Shuffle], PWN(PWN(Div_865, Clip_869), Mul_873), Unsqueeze_877, 1890 copy, 1891 copy, 1892 copy, 1893 copy, Reshape_887 + Flatten_894, 2065[Constant], (Unnamed Layer 2142) [Shuffle], Unsqueeze_892, Mul_895, Add_896, Gather_897, Reshape_900 + Unsqueeze_901, TRTBatchedNMS_902, (Unnamed Layer 2186) [Constant] + (Unnamed Layer 2187) [Shuffle], ConstantOfShape_924, 1953 copy, 1987 copy, Slice_934, (Unnamed Layer 2246) [Constant] + (Unnamed Layer 2247) [Shuffle], ConstantOfShape_943, Unsqueeze_944, Cast_939, Reshape_940 + (Unnamed Layer 2259) [Shuffle], Add_945, 2027 copy, 2005 copy, Reshape_947, MMCVMultiLevelRoiAlign_948, Conv_949 + Relu_950, Conv_951 + Relu_952, Conv_953 + Relu_954, Conv_955 + Relu_956, ConvTranspose_957, Relu_958, Conv_959, Sigmoid_963, Flatten_972, (Unnamed Layer 2196) [Constant] + (Unnamed Layer 2197) [Shuffle], ConstantOfShape_929, 1954 copy, 1995 copy, Reshape_962, Mul_973, Add_974, Gather_975, Reshape_978 + Unsqueeze_979, Cast_1024, Cast_989, PWN((Unnamed Layer 2384) [Constant] + (Unnamed Layer 2385) [Shuffle], Add_990), (Unnamed Layer 2403) [Shuffle], Reshape_961, Slice_964, Split_980, Split_980_2, PWN(PWN(PWN(Sub_999, PWN(Sub_998, Div_1000)), PWN((Unnamed Layer 2407) [Constant] + (Unnamed Layer 2408) [Shuffle], Mul_1001)), PWN((Unnamed Layer 2410) [Constant] + (Unnamed Layer 2411) [Shuffle], Sub_1002)), Unsqueeze_1003, Expand_1018, Unsqueeze_1021, Cast_985, PWN((Unnamed Layer 2372) [Constant] + (Unnamed Layer 2373) [Shuffle], Add_986), (Unnamed Layer 2390) [Shuffle], Split_980_1, Split_980_3, PWN(PWN(PWN(Sub_992, PWN(Sub_991, Div_993)), PWN((Unnamed Layer 2394) [Constant] + (Unnamed Layer 2395) [Shuffle], Mul_995)), PWN((Unnamed Layer 2397) [Constant] + (Unnamed Layer 2398) [Shuffle], Sub_997)), Unsqueeze_1019, Expand_1020, Unsqueeze_1022, 2164 copy, 2165 copy, grid_sampler_1025, (Unnamed Layer 2481) [Constant], Gather_1026, Reshape_1030, [TensorRT] INFO: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.

[TensorRT] ERROR: ../builder/cudnnBuilderUtils.cpp (427) - Cuda Error in findFastestTactic: 700 (an illegal memory access was encountered) [TensorRT] ERROR: ../rtSafe/safeRuntime.cpp (32) - Cuda Error in free: 700 (an illegal memory access was encountered) terminate called after throwing an instance of 'nvinfer1::CudaError' what(): std::exception 2023-07-12 03:09:53,547 - mmdeploy - ERROR - mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt with Call id: 1 failed. exit.

[RunningLeon]

Hi, the workspace mem is not large enough. Could increase it or use deploy config with static shape

[azuryl]

Hi, the workspace mem is not large enough. Could increase it or use deploy config with static shape

when I set configs/mmdet/detection/detection_tensorrt_dynamic-300x300-512x512.py

python ./tools/deploy.py configs/mmdet/detection/detection_tensorrt_dynamic-300x300-512x512.py /data/azuryl/mmdetection_2.27.0/configs/mask_rcnn/mask_rcnn_r50_fpn_2x_coco.py /data/azuryl/mask_rcnn_r50_fpn_2x_coco_bbox_mAP-0.392__segm_mAP-0.354_20200505_003907-3e542a40.pth /data/azuryl/mmdetection_2.27.0/demo/demo.jpg --work-dir work_dir --show --device cuda:0 --dump-info

[TensorRT] INFO: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output. [TensorRT] INFO: Detected 1 inputs and 8 output network tensors. 2023-07-12 10:22:15,398 - mmdeploy - INFO - Finish pipeline mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt 2023-07-12 10:22:18,297 - mmdeploy - INFO - visualize tensorrt model start. 2023-07-12 10:23:02,291 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /data/azuryl/mmdeploy_0.11.0/mmdeploy/lib/libmmdeploy_tensorrt_ops.so 2023-07-12:10:23:02 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /data/azuryl/mmdeploy_0.11.0/mmdeploy/lib/libmmdeploy_tensorrt_ops.so 2023-07-12 10:23:02,293 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /data/azuryl/mmdeploy_0.11.0/mmdeploy/lib/libmmdeploy_tensorrt_ops.so 2023-07-12:10:23:02 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /data/azuryl/mmdeploy_0.11.0/mmdeploy/lib/libmmdeploy_tensorrt_ops.so /data/azuryl/mmdetection_2.27.0/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file. 'data pipeline in your config file.', UserWarning) 2023-07-12:10:23:15 - root - ERROR - Input shape should be between (1, 3, 300, 300) and (1, 3, 512, 512) but get (1, 3, 800, 1216). Traceback (most recent call last): File "/data/azuryl/mmdeploy_0.11.0/mmdeploy/utils/utils.py", line 41, in target_wrapper result = target(args, kwargs) File "/data/azuryl/mmdeploy_0.11.0/mmdeploy/apis/visualize.py", line 74, in visualize_model result = task_processor.run_inference(model, model_inputs)[0] File "/data/azuryl/mmdeploy_0.11.0/mmdeploy/codebase/mmdet/deploy/object_detection.py", line 204, in run_inference return model(model_inputs, return_loss=False, rescale=True) File "/home/nvidia/archiconda3/envs/mmdeploy/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(input, kwargs) File "/data/azuryl/mmdeploy_0.11.0/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 199, in forward outputs = self.forward_test(input_img, img_metas, *args, *kwargs) File "/data/azuryl/mmdeploy_0.11.0/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 275, in forward_test outputs = self.wrapper({self.input_name: imgs}) File "/home/nvidia/archiconda3/envs/mmdeploy/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(input, kwargs) File "/data/azuryl/mmdeploy_0.11.0/mmdeploy/backend/tensorrt/wrapper.py", line 142, in forward

• f' but get {tuple(input_tensor.shape)}.' AssertionError: Input shape should be between (1, 3, 300, 300) and (1, 3, 512, 512) but get (1, 3, 800, 1216). 2023-07-12 10:23:16,890 - mmdeploy - ERROR - visualize tensorrt model failed.

[RunningLeon]

As AssertionError: Input shape should be between (1, 3, 300, 300) and (1, 3, 512, 512) but get (1, 3, 800, 1216). shows, the shape of input image is not valid. You can add onnx_config = dict(input_shape=[512, 512]) in the deploy config or use an image with valid shape.

[azuryl]

As AssertionError: Input shape should be between (1, 3, 300, 300) and (1, 3, 512, 512) but get (1, 3, 800, 1216). shows, the shape of input image is not valid. You can add onnx_config = dict(input_shape=[512, 512]) in the deploy config or use an image with valid shape.

base = ['./base_dynamic.py', '../../base/backends/tensorrt.py']

onnx_config = dict(input_shape=(512, 512)) #add onnx_config = dict(input_shape=[512, 512])

in detection_tensorrt_dynamic-300x300-512x512.py

backend_config = dict( common_config=dict(max_workspace_size=1 << 30), model_inputs=[ dict( input_shapes=dict( input=dict( min_shape=[1, 3, 300, 300], opt_shape=[1, 3, 300, 300], max_shape=[1, 3, 512, 512]))) ])

[TensorRT] INFO: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output. [TensorRT] INFO: Detected 1 inputs and 8 output network tensors. 2023-07-13 01:11:41,510 - mmdeploy - INFO - Finish pipeline mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt 2023-07-13 01:11:44,182 - mmdeploy - INFO - visualize tensorrt model start. [2023-07-13 01:11:54.913] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel' 2023-07-13 01:12:22,760 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /data/azuryl/mmdeploy_0.7.0/mmdeploy/lib/libmmdeploy_tensorrt_ops.so 2023-07-13:01:12:22 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /data/azuryl/mmdeploy_0.7.0/mmdeploy/lib/libmmdeploy_tensorrt_ops.so 2023-07-13 01:12:22,762 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /data/azuryl/mmdeploy_0.7.0/mmdeploy/lib/libmmdeploy_tensorrt_ops.so 2023-07-13:01:12:22 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /data/azuryl/mmdeploy_0.7.0/mmdeploy/lib/libmmdeploy_tensorrt_ops.so /data/azuryl/mmdetection_2.27.0/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file. 'data pipeline in your config file.', UserWarning) [TensorRT] ERROR: ../rtSafe/cuda/genericReformat.cu (1294) - Cuda Error in executeMemcpy: 1 (invalid argument) [TensorRT] ERROR: FAILED_EXECUTION: std::exception 2023-07-13:01:12:40 - root - ERROR - CUDA error: invalid argument CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Traceback (most recent call last): File "/data/azuryl/mmdeploy_0.7.0/mmdeploy/utils/utils.py", line 41, in target_wrapper result = target(args, kwargs) File "/data/azuryl/mmdeploy_0.7.0/mmdeploy/apis/visualize.py", line 72, in visualize_model result = task_processor.run_inference(model, model_inputs)[0] File "/data/azuryl/mmdeploy_0.7.0/mmdeploy/codebase/mmdet/deploy/object_detection.py", line 203, in run_inference return model(model_inputs, return_loss=False, rescale=True) File "/home/nvidia/archiconda3/envs/mmdeploy_test/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(input, **kwargs) File "/data/azuryl/mmdeploy_0.7.0/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 199, in forward outputs = End2EndModel.clear_outputs(outputs) File "/data/azuryl/mmdeploy_0.7.0/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 108, in clear_outputs inds = test_outputs[0][i, :, 4] > 0.0 RuntimeError: CUDA error: invalid argument CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. 2023-07-13 01:12:41,949 - mmdeploy - ERROR - visualize tensorrt model failed.

[azuryl]

@RunningLeon do you have solutions

[RunningLeon]

Hi, this is a bit strange. Could you try a docker image of jetson so we can reproduce it?

[azuryl]

Hi, this is a bit strange. Could you try a docker image of jetson so we can reproduce it?

you can reproduce my enviorment I have install guide in https://github.com/azuryl/mmdeploy_Jetpack4.5.1 python tools/check_env.py 2023-07-17 00:58:45,792 - mmdeploy - INFO -

2023-07-17 00:58:45,793 - mmdeploy - INFO - **Environmental information** 2023-07-17 00:58:54,170 - mmdeploy - INFO - sys.platform: linux 2023-07-17 00:58:54,171 - mmdeploy - INFO - Python: 3.6.9 | packaged by conda-forge | (default, Mar 6 2020, 18:58:41) [GCC 7.3.0] 2023-07-17 00:58:54,172 - mmdeploy - INFO - CUDA available: True 2023-07-17 00:58:54,172 - mmdeploy - INFO - GPU 0: Xavier 2023-07-17 00:58:54,172 - mmdeploy - INFO - CUDA_HOME: /usr/local/cuda-10.2 2023-07-17 00:58:54,173 - mmdeploy - INFO - NVCC: Cuda compilation tools, release 10.2, V10.2.89 2023-07-17 00:58:54,173 - mmdeploy - INFO - GCC: gcc (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0 2023-07-17 00:58:54,173 - mmdeploy - INFO - PyTorch: 1.10.0 2023-07-17 00:58:54,174 - mmdeploy - INFO - PyTorch compiling details: PyTorch built with:

• GCC 7.5
• C++ Version: 201402
• OpenMP 201511 (a.k.a. OpenMP 4.5)
• LAPACK is enabled (usually provided by MKL)
• NNPACK is enabled
• CPU capability usage: NO AVX
• CUDA Runtime 10.2
• NVCC architecture flags: -gencode;arch=compute_53,code=sm_53;-gencode;arch=compute_62,code=sm_62;-gencode;arch=compute_72,code=sm_72
• CuDNN 8.0
• Build settings: BLAS_INFO=open, BUILD_TYPE=Release, CUDA_VERSION=10.2, CUDNN_VERSION=8.0.0, CXX_COMPILER=/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -DMISSING_ARM_VST1 -DMISSING_ARM_VLD1 -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=open, TORCH_VERSION=1.10.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EIGEN_FOR_BLAS=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=OFF, USE_MKLDNN=OFF, USE_MPI=ON, USE_NCCL=0, USE_NNPACK=ON, USE_OPENMP=ON,

2023-07-17 00:58:54,174 - mmdeploy - INFO - TorchVision: 0.11.1 2023-07-17 00:58:54,174 - mmdeploy - INFO - OpenCV: 4.8.0 2023-07-17 00:58:54,174 - mmdeploy - INFO - MMCV: 1.3.17 2023-07-17 00:58:54,175 - mmdeploy - INFO - MMCV Compiler: GCC 7.5 2023-07-17 00:58:54,175 - mmdeploy - INFO - MMCV CUDA Compiler: 10.2 2023-07-17 00:58:54,175 - mmdeploy - INFO - MMDeploy: 0.7.0+83b11bc 2023-07-17 00:58:54,175 - mmdeploy - INFO -

2023-07-17 00:58:54,176 - mmdeploy - INFO - **Backend information** 2023-07-17 00:58:57,795 - mmdeploy - INFO - onnxruntime: None ops_is_avaliable : False 2023-07-17 00:58:57,982 - mmdeploy - INFO - tensorrt: 7.1.3.0 ops_is_avaliable : True 2023-07-17 00:58:58,091 - mmdeploy - INFO - ncnn: None ops_is_avaliable : False 2023-07-17 00:58:58,098 - mmdeploy - INFO - pplnn_is_avaliable: False 2023-07-17 00:58:58,105 - mmdeploy - INFO - openvino_is_avaliable: False 2023-07-17 00:58:58,200 - mmdeploy - INFO - snpe_is_available: False 2023-07-17 00:58:58,200 - mmdeploy - INFO -

2023-07-17 00:58:58,201 - mmdeploy - INFO - **Codebase information** 2023-07-17 00:58:58,210 - mmdeploy - INFO - mmdet: 2.27.0 2023-07-17 00:58:58,211 - mmdeploy - INFO - mmseg: None 2023-07-17 00:58:58,211 - mmdeploy - INFO - mmcls: None 2023-07-17 00:58:58,211 - mmdeploy - INFO - mmocr: None 2023-07-17 00:58:58,211 - mmdeploy - INFO - mmedit: None 2023-07-17 00:58:58,212 - mmdeploy - INFO - mmdet3d: None 2023-07-17 00:58:58,212 - mmdeploy - INFO - mmpose: None 2023-07-17 00:58:58,212 - mmdeploy - INFO - mmrotate: None

[azuryl]

You can add onnx_config = dict(input_shape=[512, 512]) in the deploy config or use an image with valid shape.

@RunningLeon excuse me ,can you give me a example how to add onnx_config = dict(input_shape=[512, 512]) in the deploy config or use an image with valid shape.

[RunningLeon]

hi, your modification is good and it should work.

base = ['./base_dynamic.py', '../../base/backends/tensorrt.py']

onnx_config = dict(input_shape=(512, 512)) #add onnx_config = dict(input_shape=[512, 512])

in detection_tensorrt_dynamic-300x300-512x512.py

backend_config = dict( common_config=dict(max_workspace_size=1 << 30), model_inputs=[ dict( input_shapes=dict( input=dict( min_shape=[1, 3, 300, 300], opt_shape=[1, 3, 300, 300], max_shape=[1, 3, 512, 512]))) ])

[azuryl]

hi, your modification is good and it should work.

base = ['./base_dynamic.py', '../../base/backends/tensorrt.py']

onnx_config = dict(input_shape=(512, 512)) #add onnx_config = dict(input_shape=[512, 512]) #in detection_tensorrt_dynamic-300x300-512x512.py backend_config = dict( common_config=dict(max_workspace_size=1 << 30), model_inputs=[ dict( input_shapes=dict( input=dict( min_shape=[1, 3, 300, 300], opt_shape=[1, 3, 300, 300], max_shape=[1, 3, 512, 512]))) ])

35 + Relu_936, ConvTranspose_937, Relu_938, Conv_939, Sigmoid_942, Flatten_951, (Unnamed Layer 2198) [Constant] + (Unnamed Layer 2199) [Shuffle], ConstantOfShape_903, 1976 copy, 1996 copy, Reshape_941, Mul_952, Add_953, Gather_954, Reshape_957 + Unsqueeze_958, Cast_1003, Cast_968, PWN((Unnamed Layer 2393) [Constant] + (Unnamed Layer 2394) [Shuffle], Add_969), (Unnamed Layer 2412) [Shuffle], Reshape_940, Slice_943, Split_959, Split_959_2, PWN(PWN(PWN(Sub_978, PWN(Sub_977, Div_979)), PWN((Unnamed Layer 2416) [Constant] + (Unnamed Layer 2417) [Shuffle], Mul_980)), PWN((Unnamed Layer 2419) [Constant] + (Unnamed Layer 2420) [Shuffle], Sub_981)), Unsqueeze_982, Expand_997, Unsqueeze_1000, Cast_964, PWN((Unnamed Layer 2381) [Constant] + (Unnamed Layer 2382) [Shuffle], Add_965), (Unnamed Layer 2399) [Shuffle], Split_959_1, Split_959_3, PWN(PWN(PWN(Sub_971, PWN(Sub_970, Div_972)), PWN((Unnamed Layer 2403) [Constant] + (Unnamed Layer 2404) [Shuffle], Mul_974)), PWN((Unnamed Layer 2406) [Constant] + (Unnamed Layer 2407) [Shuffle], Sub_976)), Unsqueeze_998, Expand_999, Unsqueeze_1001, 2173 copy, 2174 copy, grid_sampler_1004, (Unnamed Layer 2490) [Constant], Gather_1005, Reshape_1008, [TensorRT] INFO: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output. [TensorRT] INFO: Detected 1 inputs and 8 output network tensors. 2023-07-18 00:21:13,304 - mmdeploy - INFO - Finish pipeline mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt 2023-07-18 00:21:16,433 - mmdeploy - INFO - visualize tensorrt model start. [2023-07-18 00:21:26.497] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel' 2023-07-18 00:21:56,393 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /data/azuryl/mmdeploy_0.7.0/mmdeploy/lib/libmmdeploy_tensorrt_ops.so 2023-07-18:00:21:56 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /data/azuryl/mmdeploy_0.7.0/mmdeploy/lib/libmmdeploy_tensorrt_ops.so 2023-07-18 00:21:56,395 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /data/azuryl/mmdeploy_0.7.0/mmdeploy/lib/libmmdeploy_tensorrt_ops.so 2023-07-18:00:21:56 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /data/azuryl/mmdeploy_0.7.0/mmdeploy/lib/libmmdeploy_tensorrt_ops.so /data/azuryl/mmdetection_2.27.0/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file. 'data pipeline in your config file.', UserWarning) [TensorRT] ERROR: ../rtSafe/cuda/genericReformat.cu (1294) - Cuda Error in executeMemcpy: 1 (invalid argument) [TensorRT] ERROR: FAILED_EXECUTION: std::exception 2023-07-18:00:22:13 - root - ERROR - CUDA error: invalid argument CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Traceback (most recent call last): File "/data/azuryl/mmdeploy_0.7.0/mmdeploy/utils/utils.py", line 41, in target_wrapper result = target(args, kwargs) File "/data/azuryl/mmdeploy_0.7.0/mmdeploy/apis/visualize.py", line 72, in visualize_model result = task_processor.run_inference(model, model_inputs)[0] File "/data/azuryl/mmdeploy_0.7.0/mmdeploy/codebase/mmdet/deploy/object_detection.py", line 203, in run_inference return model(model_inputs, return_loss=False, rescale=True) File "/data/azuryl/archiconda3/envs/mmdeploy_test/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "/data/azuryl/mmdeploy_0.7.0/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 199, in forward outputs = End2EndModel.clear_outputs(outputs) File "/data/azuryl/mmdeploy_0.7.0/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 108, in clear_outputs inds = test_outputs[0][i, :, 4] > 0.0 RuntimeError: CUDA error: invalid argument CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. 2023-07-18 00:22:14,646 - mmdeploy - ERROR - visualize tensorrt model failed. @RunningLeon

It seems mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt is successful. and visual start but CUDA kernel errors https://github.com/open-mmlab/mmdetection/blob/v2.27.0/mmdet/datasets/utils.py#L70 /data/azuryl/mmdetection_2.27.0/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file. 'data pipeline in your config file.', UserWarning)

it seems the size is not correct

it is similar to https://github.com/open-mmlab/mmdetection/issues/4790(it is not solved ,why it is closed?)

in https://github.com/open-mmlab/mmdetection/issues/4447 /mmdetection/configs/base/datasets/coco_instance.py I changed test_pipeline = [ dict(type='LoadImageFromFile'), dict( type='MultiScaleFlipAug', img_scale=(512, 512), #replace img_scale=(1333, 800), flip=False, transforms=[ dict(type='Resize', keep_ratio=True), dict(type='RandomFlip'), dict(type='Normalize', **img_norm_cfg), dict(type='Pad', size_divisor=32), dict(type='ImageToTensor', keys=['img']), dict(type='Collect', keys=['img']), ]) ]
but still error /data/azuryl/mmdetection_2.27.0/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file. 'data pipeline in your config file.', UserWarning) [TensorRT] ERROR: ../rtSafe/cuda/genericReformat.cu (1294) - Cuda Error in executeMemcpy: 1 (invalid argument)

[RunningLeon]

hi, could you try mmdeploy with 0.14.0 or use master branch. 0.7.0 is a bit old.

[azuryl]

hi, could you try mmdeploy with 0.14.0 or use master branch. 0.7.0 is a bit old.

in fact is not the size and not the mmdeploy version's or docker's issue, 0.14 version need python 3.7, 0.7 is the highest version for python 3.6,because Jetpack 4.5.1 only support python 3.6, I almost tried from 0.4-1.4, from 0.8 begin use python3.7

from "/data/azuryl/mmdetection_2.27.0/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file. 'data pipeline in your config file.', UserWarning)" the problem is the expected file should be configs/mmdet/instance-seg/instance-seg_tensorrt_dynamic-320x320-1344x1344.py for Mask-RCNN While configs/mmdet/detection/detection_tensorrt_dynamic-320x320-1344x1344.py is NOT for instance segmentation task. https://github.com/open-mmlab/mmdeploy/issues/389#issuecomment-1119329070

I have wrote a guide for install in Jetpack 4.5.1 https://github.com/azuryl/mmdeploy_Jetpack4.5.1/blob/main/README.md

[RunningLeon]

hi, glad to know it is solved. The error log is not clear for mismatch deploy config. We may add a check of the deploy config later.