TensorRT Execution Build Fails on Jetson Jetpack 4.6.1

[Jacobsolawetz]

Describe the issue

ORT fails to build on Jetson Jetpack 4.6.1

I am using the 4.6.1 docker image nvcr.io/nvidia/l4t-ml:r32.7.1-py3

I am following the guide published by onnxruntime https://onnxruntime.ai/docs/build/eps.html#nvidia-jetson-tx1tx2nanoxavier

I suspect that the repository has outgrown this documentation. Is there new documentation?

notably this Microsoft prebuilt image for Jetson mcr.microsoft.com/azureml/onnxruntime:v.1.4.0-jetpack4.4-l4t-base-r32.4.3 does not include TRT execution bindings. Is there a new supported image?

Thank you for your assistance.

Urgency

Yes, this issue is blocking our production roadmap

Target platform

Jetson Xaiver NX, Jetpack 4.6.1

Build script

FROM nvcr.io/nvidia/l4t-ml:r32.7.1-py3

ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update RUN apt-get install lshw -y RUN apt-get install git -y RUN git clone --recursive https://github.com/microsoft/onnxruntime RUN export CUDACXX="/usr/local/cuda/bin/nvcc"

RUN apt-get update -y RUN apt-get install python3-pip -y RUN apt-get install python3-matplotlib -y RUN apt-get install gfortran -y RUN apt-get install -y build-essential libatlas-base-dev RUN apt-get install ffmpeg libsm6 libxext6 -y

RUN apt install -y --no-install-recommends build-essential software-properties-common libopenblas-dev RUN wget https://github.com/Kitware/CMake/releases/download/v3.23.0-rc1/cmake-3.23.0-rc1-linux-aarch64.tar.gz RUN tar xf cmake-3.23.0-rc1-linux-aarch64.tar.gz RUN export PATH="/cmake-3.20.0-rc1-linux-aarch64/bin:$PATH" RUN apt remove cmake -y RUN pwd RUN ln -s /cmake-3.23.0-rc1-linux-aarch64/bin/* /usr/local/bin RUN which cmake RUN cmake --version

ENV PATH="/usr/local/cuda/bin:${PATH}" ENV LD_LIBRARY_PATH="/usr/local/cuda/lib64:${LD_LIBRARY_PATH}"

RUN pip3 install --upgrade pip RUN pip3 uninstall torch -y RUN pip3 install torch==1.10.0

RUN echo "$PATH" && echo "$LD_LIBRARY_PATH"

WORKDIR /onnxruntime

RUN ./build.sh --config Release --update --build --parallel --build_wheel --use_tensorrt --cuda_home /usr/local/cuda --cudnn_home /usr/lib/aarch64-linux-gnu --tensorrt_home /usr/lib/aarch64-linux-gnu

Error / output

[ 13%] Building CXX object _deps/protobuf-build/CMakeFiles/libprotobuf.dir/__/protobuf-src/src/google/protobuf/wrappers.pb.cc.o [ 13%] Linking CXX static library libprotobuf.a [ 13%] Built target libprotobuf Makefile:165: recipe for target 'all' failed make: ** [all] Error 2 Traceback (most recent call last): File "/onnxruntime/tools/ci_build/build.py", line 2741, in sys.exit(main()) File "/onnxruntime/tools/ci_build/build.py", line 2638, in main build_targets(args, cmake_path, build_dir, configs, num_parallel_jobs, args.target) File "/onnxruntime/tools/ci_build/build.py", line 1399, in build_targets run_subprocess(cmd_args, env=env) File "/onnxruntime/tools/ci_build/build.py", line 766, in run_subprocess return run(args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env) File "/onnxruntime/tools/python/util/run.py", line 57, in run shell=shell, File "/usr/lib/python3.6/subprocess.py", line 438, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['/usr/local/bin/cmake', '--build', '/onnxruntime/build/Linux/Release', '--config', 'Release', '--', '-j6']' returned non-zero exit status 2. The command '/bin/sh -c ./onnxruntime/build.sh --config Release --update --build --parallel --build_wheel --use_tensorrt --cuda_home /usr/local/cuda --cudnn_home /usr/lib/aarch64-linux-gnu --tensorrt_home /usr/lib/aarch64-linux-gnu' returned a non-zero code: 1

Visual Studio Version

No response

GCC / Compiler Version

No response

[jywu-msft]

Hi, for jetpack 4.6.1 you can pip install the package from https://elinux.org/Jetson_Zoo search for onnxruntime 1.11.0

[Jacobsolawetz]

@jywu-msft do those wheels include the TensorRT execution provider?

I believe they are the same wheels that are installed in mcr.microsoft.com/azureml/onnxruntime:v.1.4.0-jetpack4.4-l4t-base-r32.4.3 , which does not include the TensorRT execution provider on Jetson

[Jacobsolawetz]

@jywu-msft any update or new thinking from your side on this thread?

[jywu-msft]

@jywu-msft any update or new thinking from your side on this thread?

do not use that image. we have not supported any mcr images in 3+ years. the one you pointed to is from 4 years ago (i.e. OnnxRuntime version 1.4.0). we started making wheels available on jetson zoo, so users can easily pip install in their own dockerfiles. please try the jetson zoo wheels at https://elinux.org/Jetson_Zoo#ONNX_Runtime as I suggested.

[Jacobsolawetz]

@jywu-msft ok, copy that! I will report back here how it goes

[lofyol]

I want to compile the. so file with the backend of Tensorrt on NX2. How to solve this problem（bug: subprocess.CalledProcessError: Command '['/usr/local/bin/cmake', '--build', '/onnxruntime/build/Linux/Release', '--config', 'Release', '--', '-j6']' returned non-zero exit status 2.）

[Cats-eat-peppercorns]

If I want to use onnxruntime1.6.0 in cmake for c++, can I install it with pip? If so, how should I use it after installing it?