Closed psyhtest closed 2 years ago
On the one hand, the Python 3.8 error is understood because TensorRT is only natively installed for Python 3.6. On the other hand, TensorRT should not be required to preprocess any data.
I've got similar errors with the v1.1 release:
With Python 3.6:
anton@xavier:/datasets/inference_results_v1.1/closed/NVIDIA$ python3.6 code/resnet50/tensorrt/preprocess_data.py
Traceback (most recent call last):
File "code/resnet50/tensorrt/preprocess_data.py", line 27, in <module>
from code.common import logging
File "/datasets/inference_results_v1.1/closed/NVIDIA/code/__init__.py", line 22, in <module>
from code.common import logging
File "/datasets/inference_results_v1.1/closed/NVIDIA/code/common/__init__.py", line 33, in <module>
from code.common.system_list import KnownSystems, MIGConfiguration
File "/datasets/inference_results_v1.1/closed/NVIDIA/code/common/system_list.py", line 22, in <module>
from code.common.constants import CPUArch
File "/datasets/inference_results_v1.1/closed/NVIDIA/code/common/constants.py", line 15
from __future__ import annotations
^
SyntaxError: future feature annotations is not defined
With Python 3.8:
anton@xavier:/datasets/inference_results_v1.1/closed/NVIDIA$ python3.8 code/resnet50/tensorrt/preprocess_data.py
Traceback (most recent call last):
File "code/resnet50/tensorrt/preprocess_data.py", line 27, in <module>
from code.common import logging
File "/datasets/inference_results_v1.1/closed/NVIDIA/code/__init__.py", line 22, in <module>
from code.common import logging
File "/datasets/inference_results_v1.1/closed/NVIDIA/code/common/__init__.py", line 33, in <module>
from code.common.system_list import KnownSystems, MIGConfiguration
File "/datasets/inference_results_v1.1/closed/NVIDIA/code/common/system_list.py", line 22, in <module>
from code.common.constants import CPUArch
File "/datasets/inference_results_v1.1/closed/NVIDIA/code/common/constants.py", line 27, in <module>
import tensorrt as trt
ModuleNotFoundError: No module named 'tensorrt'
The same problem seems to prevent automatic platform detection:
katya@xavier:/datasets/inference_results_v1.1/closed/NVIDIA$ python3.6 scripts/get_system_id.py
Traceback (most recent call last):
File "scripts/get_system_id.py", line 21, in <module>
from code.common import get_system
File "/datasets/inference_results_v1.1/closed/NVIDIA/code/__init__.py", line 22, in <module>
from code.common import logging
File "/datasets/inference_results_v1.1/closed/NVIDIA/code/common/__init__.py", line 33, in <module>
from code.common.system_list import KnownSystems, MIGConfiguration
File "/datasets/inference_results_v1.1/closed/NVIDIA/code/common/system_list.py", line 22, in <module>
from code.common.constants import CPUArch
File "/datasets/inference_results_v1.1/closed/NVIDIA/code/common/constants.py", line 15
from __future__ import annotations
^
SyntaxError: future feature annotations is not defined
katya@xavier:/datasets/inference_results_v1.1/closed/NVIDIA$ python3.8 scripts/get_system_id.py
Traceback (most recent call last):
File "scripts/get_system_id.py", line 21, in <module>
from code.common import get_system
File "/datasets/inference_results_v1.1/closed/NVIDIA/code/__init__.py", line 22, in <module>
from code.common import logging
File "/datasets/inference_results_v1.1/closed/NVIDIA/code/common/__init__.py", line 33, in <module>
from code.common.system_list import KnownSystems, MIGConfiguration
File "/datasets/inference_results_v1.1/closed/NVIDIA/code/common/system_list.py", line 22, in <module>
from code.common.constants import CPUArch
File "/datasets/inference_results_v1.1/closed/NVIDIA/code/common/constants.py", line 27, in <module>
import tensorrt as trt
ModuleNotFoundError: No module named 'tensorrt'
so it has to be manually forced:
katya@xavier:/datasets/inference_results_v1.1/closed/NVIDIA$ git diff Makefile
diff --git a/closed/NVIDIA/Makefile b/closed/NVIDIA/Makefile
index 4e371a92..676bf377 100644
--- a/closed/NVIDIA/Makefile
+++ b/closed/NVIDIA/Makefile
@@ -36,7 +36,7 @@ SUBMITTER ?= NVIDIA
SYSTEM_NAME ?= $(shell $(PYTHON3_CMD) scripts/get_system_id.py 2> /dev/null)
TARGET_X86_64 := 0
TARGET_AARCH64 := 0
-IS_XAVIER := 0
+IS_XAVIER := 1
ifeq ($(ARCH), x86_64)
TARGET_X86_64 = 1
endif
I'm beginning to suspect that the installation of dependencies has not been completely successful. Will try to rerun it.
The README says:
JetPack 4.6 (21.08 Jetson CUDA-X AI Developer Preview)
- Includes TensorRT 8.0.1.6
- Includes cuDNN 8.2.3.8 (If you are using the production Jetpack 4.6, this is the only package you need to install before running the dependency script below)
- Dependencies can be installed by running the script located at closed/NVIDIA/scripts/install_xavier_dependencies.sh
Is this up-to-date? JetPack 5.0 is supposed to be used in the v2.0 round, and the script is not available.
Luckily, this script is available in e.g. Azure's submission... alongside with similar scripts for Orin:
anton@xavier:/datasets/inference_results_v2.0/closed/Azure$ ls -la scripts/install_*
-rw-rw-r-- 1 anton krai 2412 May 16 14:40 scripts/install_orin_auto_dependencies_internal.sh
-rw-rw-r-- 1 anton krai 7050 May 16 14:40 scripts/install_orin_auto_dependencies.sh
-rw-rw-r-- 1 anton krai 1619 May 16 14:40 scripts/install_orin_jetson_dependencies_internal.sh
-rw-rw-r-- 1 anton krai 6842 May 16 14:40 scripts/install_orin_jetson_dependencies.sh
-rw-rw-r-- 1 anton krai 7029 May 16 14:40 scripts/install_xavier_dependencies.sh
The install_xavier_dependencies.sh
script is practically the same as for v1.1:
anton@xavier:~$ diff \
/datasets/inference_results_v1.1/closed/NVIDIA/scripts/install_xavier_dependencies.sh \
/datasets/inference_results_v2.0/closed/Azure/scripts/install_xavier_dependencies.sh
2c2
< # Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
---
> # Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
I suspect it might need to be updated e.g. TensorRT should probably be v8.0, not v7.2.
It appears the v1.1/v2.0 version of install_xavier_dependencies.sh
is incomplete: it attempts to switch to Python 3.8 on JetPack 4.6, but does not build Python 3.8 bindings for TensorRT. I've attempted to do it myself following instructions here, but hit a brick wall. (For starters, there's no aarch64 deb package for Python 3.8.)
I've gone a bit further with the following steps:
export EXT_PATH=~/external
mkdir -p $EXT_PATH
cd $EXT_PATH
git clone https://github.com/pybind/pybind11.git
wget https://www.python.org/ftp/python/3.8.13/Python-3.8.13.tgz
tar xvzf Python-3.8.13.tgz
mkdir python3.8
cp -r Python-3.8.13/Include/ python3.8/include
cp /usr/include/aarch64-linux-gnu/python3.8/pyconfig.h python3.8/include/
git clone https://github.com/NVIDIA/TensorRT
cd TensorRT
git checkout release/8.0
cd python
EXT_PATH=~/external CPATH=~/external/pybind11/include TRT_OSSPATH=~/external/TensorRT \
PYTHON_MAJOR_VERSION=3 PYTHON_MINOR_VERSION=8 TARGET_ARCHITECTURE=aarch64 \
./build.sh
The build log, however, is not clean:
~/external/TensorRT/python/build ~/external/TensorRT/python
ForwardDeclarations.h utils.h
-- Configuring done
-- Generating done
-- Build files have been written to: /home/anton/external/TensorRT/python/build
[ 8%] Building CXX object CMakeFiles/tensorrt.dir/src/infer/pyCore.cpp.o
[ 33%] Building CXX object CMakeFiles/tensorrt.dir/src/infer/pyInt8.cpp.o
[ 33%] Building CXX object CMakeFiles/tensorrt.dir/src/infer/pyFoundationalTypes.cpp.o
[ 33%] Building CXX object CMakeFiles/tensorrt.dir/src/infer/pyAlgorithmSelector.cpp.o
[ 41%] Building CXX object CMakeFiles/tensorrt.dir/src/parsers/pyOnnx.cpp.o
[ 50%] Building CXX object CMakeFiles/tensorrt.dir/src/infer/pyPlugin.cpp.o
[ 58%] Building CXX object CMakeFiles/tensorrt.dir/src/infer/pyGraph.cpp.o
[ 66%] Building CXX object CMakeFiles/tensorrt.dir/src/parsers/pyCaffe.cpp.o
[ 75%] Building CXX object CMakeFiles/tensorrt.dir/src/pyTensorRT.cpp.o
[ 83%] Building CXX object CMakeFiles/tensorrt.dir/src/utils.cpp.o
[ 91%] Building CXX object CMakeFiles/tensorrt.dir/src/parsers/pyUff.cpp.o
In file included from /home/anton/external/TensorRT/python/include/ForwardDeclarations.h:18:0,
from /home/anton/external/TensorRT/python/src/infer/pyInt8.cpp:18:
/home/anton/external/TensorRT/python/src/infer/pyInt8.cpp: In member function ‘virtual const void* tensorrt::pyIInt8LegacyCalibrator::readHistogramCache(std::size_t&)’:
/home/anton/external/pybind11/include/pybind11/pybind11.h:2742:70: error: void value not ignored as it ought to be
return pybind11::detail::cast_safe<ret_type>(std::move(o)); \
^
/home/anton/external/pybind11/include/pybind11/pybind11.h:2776:9: note: in expansion of macro ‘PYBIND11_OVERRIDE_IMPL’
PYBIND11_OVERRIDE_IMPL(PYBIND11_TYPE(ret_type), PYBIND11_TYPE(cname), name, __VA_ARGS__); \
^~~~~~~~~~~~~~~~~~~~~~
/home/anton/external/pybind11/include/pybind11/pybind11.h:2835:5: note: in expansion of macro ‘PYBIND11_OVERRIDE_PURE_NAME’
PYBIND11_OVERRIDE_PURE_NAME( \
^~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/anton/external/TensorRT/python/src/infer/pyInt8.cpp:125:9: note: in expansion of macro ‘PYBIND11_OVERLOAD_PURE_NAME’
PYBIND11_OVERLOAD_PURE_NAME(
^~~~~~~~~~~~~~~~~~~~~~~~~~~
CMakeFiles/tensorrt.dir/build.make:133: recipe for target 'CMakeFiles/tensorrt.dir/src/infer/pyInt8.cpp.o' failed
make[2]: *** [CMakeFiles/tensorrt.dir/src/infer/pyInt8.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
CMakeFiles/Makefile2:94: recipe for target 'CMakeFiles/tensorrt.dir/all' failed
make[1]: *** [CMakeFiles/tensorrt.dir/all] Error 2
Makefile:102: recipe for target 'all' failed
make: *** [all] Error 2
Generating python 3.8 bindings for TensorRT 8.0.1.6
~/external/TensorRT/python/packaging ~/external/TensorRT/python/build ~/external/TensorRT/python
~/external/TensorRT/python/build ~/external/TensorRT/python
/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
~/external/TensorRT/python
and even after successful installation:
$ python3 -m pip install build/dist/tensorrt-*.whl
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Processing ./build/dist/tensorrt-8.0.1.6-cp38-none-linux_aarch64.whl
Installing collected packages: tensorrt
Successfully installed tensorrt-8.0.1.6
$ python3 -m pip show tensorrt
Name: tensorrt
Version: 8.0.1.6
Summary: A high performance deep learning inference library
Home-page:
Author: NVIDIA
Author-email:
License: Proprietary
Location: /home/anton/.local/lib/python3.8/site-packages
Requires:
Required-by:
importing tensorrt
fails:
$ python3 -c "import tensorrt"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/anton/.local/lib/python3.8/site-packages/tensorrt/__init__.py", line 36, in <module>
from .tensorrt import *
ModuleNotFoundError: No module named 'tensorrt.tensorrt'
Pinned down that line to a pybind11
commit after v2.6.2, which was recommended for Xavier. Checking out v2.6.2 makes it work.
So the complete sequence of instructions is:
export EXT_PATH=/tmp/tensorrt-bindings
rm -rf $EXT_PATH
mkdir $EXT_PATH
cd $EXT_PATH
git clone https://github.com/pybind/pybind11.git
cd pybind11
git checkout v2.6.2
cd $EXT_PATH
wget https://www.python.org/ftp/python/3.8.13/Python-3.8.13.tgz
tar xvzf Python-3.8.13.tgz
mkdir python3.8
cp -r Python-3.8.13/Include/ python3.8/include
cp /usr/include/aarch64-linux-gnu/python3.8/pyconfig.h python3.8/include/
git clone https://github.com/NVIDIA/TensorRT
cd TensorRT
git checkout release/8.0
cd python
CPATH=$EXT_PATH/pybind11/include TRT_OSSPATH=$EXT_PATH/TensorRT \
PYTHON_MAJOR_VERSION=3 PYTHON_MINOR_VERSION=8 TARGET=aarch64 \
./build.sh
python3 -m pip uninstall tensorrt -y
python3 -m pip install build/dist/tensorrt-*.whl
cd $EXT_PATH
python3 -c "import tensorrt"
@psyhtest Are your problems on Xavier resolved now?
Yes, thanks @nv-ananjappa.
Meet the same issue on Xavier NX Devkit with JetPack 4.6, can't solve following your instructions,
@psyhtest Could you give me more help, thank you!
importing tensorrt fails:
zack@zack-desktop:~/inference_results_v2.0/closed/NVIDIA$ python3 -c "import tensorrt" Traceback (most recent call last): File "<string>", line 1, in <module> File "/home/zack/.local/lib/python3.8/site-packages/tensorrt/__init__.py", line 36, in <module> from .tensorrt import * ModuleNotFoundError: No module named 'tensorrt.tensorrt'
Pinned down that line to a
pybind11
commit after v2.6.2, which was recommended for Xavier. Checking out v2.6.2 makes it work.So the complete sequence of instructions is:
export EXT_PATH=/tmp/tensorrt-bindings rm -rf $EXT_PATH mkdir $EXT_PATH cd $EXT_PATH git clone https://github.com/pybind/pybind11.git cd pybind11 git checkout v2.6.2 cd $EXT_PATH wget https://www.python.org/ftp/python/3.8.13/Python-3.8.13.tgz tar xvzf Python-3.8.13.tgz mkdir python3.8 cp -r Python-3.8.13/Include/ python3.8/include cp /usr/include/aarch64-linux-gnu/python3.8/pyconfig.h python3.8/include/ git clone https://github.com/NVIDIA/TensorRT cd TensorRT git checkout release/8.0 cd python CPATH=$EXT_PATH/pybind11/include TRT_OSSPATH=$EXT_PATH/TensorRT \ PYTHON_MAJOR_VERSION=3 PYTHON_MINOR_VERSION=8 TARGET=aarch64 \ ./build.sh python3 -m pip uninstall tensorrt -y python3 -m pip install build/dist/tensorrt-*.whl cd $EXT_PATH python3 -c "import tensorrt"
Pinned down that line to a
pybind11
commit after v2.6.2, which was recommended for Xavier. Checking out v2.6.2 makes it work.So the complete sequence of instructions is:
export EXT_PATH=/tmp/tensorrt-bindings rm -rf $EXT_PATH mkdir $EXT_PATH cd $EXT_PATH git clone https://github.com/pybind/pybind11.git cd pybind11 git checkout v2.6.2 cd $EXT_PATH wget https://www.python.org/ftp/python/3.8.13/Python-3.8.13.tgz tar xvzf Python-3.8.13.tgz mkdir python3.8 cp -r Python-3.8.13/Include/ python3.8/include cp /usr/include/aarch64-linux-gnu/python3.8/pyconfig.h python3.8/include/ git clone https://github.com/NVIDIA/TensorRT cd TensorRT git checkout release/8.0 cd python CPATH=$EXT_PATH/pybind11/include TRT_OSSPATH=$EXT_PATH/TensorRT \ PYTHON_MAJOR_VERSION=3 PYTHON_MINOR_VERSION=8 TARGET=aarch64 \ ./build.sh python3 -m pip uninstall tensorrt -y python3 -m pip install build/dist/tensorrt-*.whl cd $EXT_PATH python3 -c "import tensorrt"
Thanks a lot. The overall procedure works on my Jetson Nano 2Gb. The only thing I modified is inside the build.sh, where I changed the number of jobs (-j12) to 2 (-j2) since this Nano only has 4 cores, otherwise the building freezes.
I've been trying to preprocess the ImageNet validation dataset on a Jetson AGX Xavier with JetPack 4.6. To my surprise, what used to work with the v1.0 release:
(and still does!), does not work with the v2.0 release.
With Python 3.6:
With Python 3.8: