search

microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.23k stars 2.87k forks source link

[Build] Cannot build onnxruntime #16583

Open bruno-darochac opened 1 year ago

bruno-darochac commented 1 year ago

Describe the issue

Hello, I have created on my Orin a c++ programm that run inference on GPU with onnxruntime. For no reason it stop to work. I wanted to rebuild and reinstall Onnxruntime libs but not succeed on it...

Have anyone face the same comportement ?

Urgency

No response

Target platform

Linux 4 Tegra ORIN AGX

Build script

$ ./build.sh --build_dir ./build --config RelWithDebInfo --arm64 --build_shared_lib --use_cuda --cudnn_home /usr/lib/aarch64-linux-gnu --cuda_home /usr/local/cuda

Error / output

2023-07-04 08:41:20,159 tools_python_utils [INFO] - flatbuffers module is not installed. parse_config will not be available 2023-07-04 08:41:20,163 build [DEBUG] - Command line arguments: --build_dir /orin_ssd/onnxruntime/build/Linux --build_dir ./build --config RelWithDebInfo --arm64 --build_shared_lib --use_cuda --cudnn_home /usr/lib/aarch64-linux-gnu --cuda_home /usr/local/cuda 2023-07-04 08:41:20,173 build [DEBUG] - Defaulting to running update, build [and test for native builds]. 2023-07-04 08:41:20,174 build [INFO] - Build started 2023-07-04 08:41:20,174 util.run [INFO] - Running subprocess in '/orin_ssd/onnxruntime' git submodule sync --recursive Synchronizing submodule url for 'cmake/external/eigen' Synchronizing submodule url for 'cmake/external/emsdk' Synchronizing submodule url for 'cmake/external/libprotobuf-mutator' Synchronizing submodule url for 'cmake/external/onnx' Synchronizing submodule url for 'cmake/external/onnx/third_party/benchmark' Synchronizing submodule url for 'cmake/external/onnx/third_party/pybind11' Synchronizing submodule url for 'cmake/external/onnxruntime-extensions' 2023-07-04 08:41:20,248 util.run [DEBUG] - Subprocess completed. Return code: 0 2023-07-04 08:41:20,249 util.run [INFO] - Running subprocess in '/orin_ssd/onnxruntime' git submodule update --init --recursive 2023-07-04 08:41:20,457 util.run [DEBUG] - Subprocess completed. Return code: 0 2023-07-04 08:41:20,457 build [INFO] - Generating CMake build tree 2023-07-04 08:41:20,458 util.run [INFO] - Running subprocess in './build/RelWithDebInfo' /usr/bin/cmake /orin_ssd/onnxruntime/cmake -Donnxruntime_RUN_ONNX_TESTS=OFF -Donnxruntime_GENERATE_TEST_REPORTS=ON -DPython_EXECUTABLE=/usr/bin/python3 -DPYTHON_EXECUTABLE=/usr/bin/python3 -Donnxruntime_USE_MIMALLOC=OFF -Donnxruntime_ENABLE_PYTHON=OFF -Donnxruntime_BUILD_CSHARP=OFF -Donnxruntime_BUILD_JAVA=OFF -Donnxruntime_BUILD_NODEJS=OFF -Donnxruntime_BUILD_OBJC=OFF -Donnxruntime_BUILD_SHARED_LIB=ON -Donnxruntime_BUILD_APPLE_FRAMEWORK=OFF -Donnxruntime_USE_DNNL=OFF -Donnxruntime_USE_NNAPI_BUILTIN=OFF -Donnxruntime_USE_RKNPU=OFF -Donnxruntime_USE_LLVM=OFF -Donnxruntime_ENABLE_MICROSOFT_INTERNAL=OFF -Donnxruntime_USE_VITISAI=OFF -Donnxruntime_USE_TENSORRT=OFF -Donnxruntime_USE_TENSORRT_BUILTIN_PARSER=ON -Donnxruntime_USE_TVM=OFF -Donnxruntime_TVM_CUDA_RUNTIME=OFF -Donnxruntime_TVM_USE_HASH=OFF -Donnxruntime_USE_MIGRAPHX=OFF -Donnxruntime_DISABLE_CONTRIB_OPS=OFF -Donnxruntime_DISABLE_ML_OPS=OFF -Donnxruntime_DISABLE_RTTI=OFF -Donnxruntime_DISABLE_EXCEPTIONS=OFF -Donnxruntime_MINIMAL_BUILD=OFF -Donnxruntime_EXTENDED_MINIMAL_BUILD=OFF -Donnxruntime_MINIMAL_BUILD_CUSTOM_OPS=OFF -Donnxruntime_REDUCED_OPS_BUILD=OFF -Donnxruntime_ENABLE_LANGUAGE_INTEROP_OPS=OFF -Donnxruntime_USE_DML=OFF -Donnxruntime_USE_WINML=OFF -Donnxruntime_BUILD_MS_EXPERIMENTAL_OPS=OFF -Donnxruntime_USE_TELEMETRY=OFF -Donnxruntime_ENABLE_LTO=OFF -Donnxruntime_USE_ACL=OFF -Donnxruntime_USE_ACL_1902=OFF -Donnxruntime_USE_ACL_1905=OFF -Donnxruntime_USE_ACL_1908=OFF -Donnxruntime_USE_ACL_2002=OFF -Donnxruntime_USE_ARMNN=OFF -Donnxruntime_ARMNN_RELU_USE_CPU=ON -Donnxruntime_ARMNN_BN_USE_CPU=ON -Donnxruntime_USE_JSEP=OFF -Donnxruntime_ENABLE_NVTX_PROFILE=OFF -Donnxruntime_ENABLE_TRAINING=OFF -Donnxruntime_ENABLE_TRAINING_OPS=OFF -Donnxruntime_ENABLE_TRAINING_APIS=OFF -Donnxruntime_ENABLE_CPU_FP16_OPS=OFF -Donnxruntime_USE_NCCL=OFF -Donnxruntime_BUILD_BENCHMARKS=OFF -Donnxruntime_USE_ROCM=OFF -DOnnxruntime_GCOV_COVERAGE=OFF -Donnxruntime_USE_MPI=OFF -Donnxruntime_ENABLE_MEMORY_PROFILE=OFF -Donnxruntime_ENABLE_CUDA_LINE_NUMBER_INFO=OFF -Donnxruntime_BUILD_WEBASSEMBLY=OFF -Donnxruntime_BUILD_WEBASSEMBLY_STATIC_LIB=OFF -Donnxruntime_ENABLE_WEBASSEMBLY_EXCEPTION_CATCHING=ON -Donnxruntime_ENABLE_WEBASSEMBLY_API_EXCEPTION_CATCHING=OFF -Donnxruntime_ENABLE_WEBASSEMBLY_EXCEPTION_THROWING=ON -Donnxruntime_WEBASSEMBLY_RUN_TESTS_IN_BROWSER=OFF -Donnxruntime_ENABLE_WEBASSEMBLY_THREADS=OFF -Donnxruntime_ENABLE_WEBASSEMBLY_DEBUG_INFO=OFF -Donnxruntime_ENABLE_WEBASSEMBLY_PROFILING=OFF -Donnxruntime_ENABLE_LAZY_TENSOR=OFF -Donnxruntime_ENABLE_EXTERNAL_CUSTOM_OP_SCHEMAS=OFF -Donnxruntime_ENABLE_CUDA_PROFILING=OFF -Donnxruntime_ENABLE_ROCM_PROFILING=OFF -Donnxruntime_USE_XNNPACK=OFF -Donnxruntime_USE_CANN=OFF -Donnxruntime_NVCC_THREADS=1 -DCUDA_CUDA_LIBRARY=/usr/local/cuda/lib64/stubs -Donnxruntime_CROSS_COMPILING=ON -DCMAKE_TLS_VERIFY=ON -DFETCHCONTENT_QUIET=OFF -Donnxruntime_USE_CUDA=ON -Donnxruntime_CUDA_HOME=/usr/local/cuda -Donnxruntime_CUDNN_HOME=/usr/lib/aarch64-linux-gnu -Donnxruntime_PYBIND_EXPORT_OPSCHEMA=OFF -Donnxruntime_ENABLE_MEMLEAK_CHECKER=OFF -DCMAKE_BUILD_TYPE=RelWithDebInfo '' CMake Error: The source directory "/orin_ssd/onnxruntime/build/RelWithDebInfo" does not appear to contain CMakeLists.txt. Specify --help for usage, or press the help button on the CMake GUI. Traceback (most recent call last): File "/orin_ssd/onnxruntime/tools/ci_build/build.py", line 2669, in sys.exit(main()) File "/orin_ssd/onnxruntime/tools/ci_build/build.py", line 2531, in main generate_build_tree( File "/orin_ssd/onnxruntime/tools/ci_build/build.py", line 1367, in generate_build_tree run_subprocess( File "/orin_ssd/onnxruntime/tools/ci_build/build.py", line 779, in run_subprocess return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env) File "/orin_ssd/onnxruntime/tools/python/util/run.py", line 49, in run completed_process = subprocess.run( File "/usr/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/cmake', '/orin_ssd/onnxruntime/cmake', '-Donnxruntime_RUN_ONNX_TESTS=OFF', '-Donnxruntime_GENERATE_TEST_REPORTS=ON', '-DPython_EXECUTABLE=/usr/bin/python3', '-DPYTHON_EXECUTABLE=/usr/bin/python3', '-Donnxruntime_USE_MIMALLOC=OFF', '-Donnxruntime_ENABLE_PYTHON=OFF', '-Donnxruntime_BUILD_CSHARP=OFF', '-Donnxruntime_BUILD_JAVA=OFF', '-Donnxruntime_BUILD_NODEJS=OFF', '-Donnxruntime_BUILD_OBJC=OFF', '-Donnxruntime_BUILD_SHARED_LIB=ON', '-Donnxruntime_BUILD_APPLE_FRAMEWORK=OFF', '-Donnxruntime_USE_DNNL=OFF', '-Donnxruntime_USE_NNAPI_BUILTIN=OFF', '-Donnxruntime_USE_RKNPU=OFF', '-Donnxruntime_USE_LLVM=OFF', '-Donnxruntime_ENABLE_MICROSOFT_INTERNAL=OFF', '-Donnxruntime_USE_VITISAI=OFF', '-Donnxruntime_USE_TENSORRT=OFF', '-Donnxruntime_USE_TENSORRT_BUILTIN_PARSER=ON', '-Donnxruntime_USE_TVM=OFF', '-Donnxruntime_TVM_CUDA_RUNTIME=OFF', '-Donnxruntime_TVM_USE_HASH=OFF', '-Donnxruntime_USE_MIGRAPHX=OFF', '-Donnxruntime_DISABLE_CONTRIB_OPS=OFF', '-Donnxruntime_DISABLE_ML_OPS=OFF', '-Donnxruntime_DISABLE_RTTI=OFF', '-Donnxruntime_DISABLE_EXCEPTIONS=OFF', '-Donnxruntime_MINIMAL_BUILD=OFF', '-Donnxruntime_EXTENDED_MINIMAL_BUILD=OFF', '-Donnxruntime_MINIMAL_BUILD_CUSTOM_OPS=OFF', '-Donnxruntime_REDUCED_OPS_BUILD=OFF', '-Donnxruntime_ENABLE_LANGUAGE_INTEROP_OPS=OFF', '-Donnxruntime_USE_DML=OFF', '-Donnxruntime_USE_WINML=OFF', '-Donnxruntime_BUILD_MS_EXPERIMENTAL_OPS=OFF', '-Donnxruntime_USE_TELEMETRY=OFF', '-Donnxruntime_ENABLE_LTO=OFF', '-Donnxruntime_USE_ACL=OFF', '-Donnxruntime_USE_ACL_1902=OFF', '-Donnxruntime_USE_ACL_1905=OFF', '-Donnxruntime_USE_ACL_1908=OFF', '-Donnxruntime_USE_ACL_2002=OFF', '-Donnxruntime_USE_ARMNN=OFF', '-Donnxruntime_ARMNN_RELU_USE_CPU=ON', '-Donnxruntime_ARMNN_BN_USE_CPU=ON', '-Donnxruntime_USE_JSEP=OFF', '-Donnxruntime_ENABLE_NVTX_PROFILE=OFF', '-Donnxruntime_ENABLE_TRAINING=OFF', '-Donnxruntime_ENABLE_TRAINING_OPS=OFF', '-Donnxruntime_ENABLE_TRAINING_APIS=OFF', '-Donnxruntime_ENABLE_CPU_FP16_OPS=OFF', '-Donnxruntime_USE_NCCL=OFF', '-Donnxruntime_BUILD_BENCHMARKS=OFF', '-Donnxruntime_USE_ROCM=OFF', '-DOnnxruntime_GCOV_COVERAGE=OFF', '-Donnxruntime_USE_MPI=OFF', '-Donnxruntime_ENABLE_MEMORY_PROFILE=OFF', '-Donnxruntime_ENABLE_CUDA_LINE_NUMBER_INFO=OFF', '-Donnxruntime_BUILD_WEBASSEMBLY=OFF', '-Donnxruntime_BUILD_WEBASSEMBLY_STATIC_LIB=OFF', '-Donnxruntime_ENABLE_WEBASSEMBLY_EXCEPTION_CATCHING=ON', '-Donnxruntime_ENABLE_WEBASSEMBLY_API_EXCEPTION_CATCHING=OFF', '-Donnxruntime_ENABLE_WEBASSEMBLY_EXCEPTION_THROWING=ON', '-Donnxruntime_WEBASSEMBLY_RUN_TESTS_IN_BROWSER=OFF', '-Donnxruntime_ENABLE_WEBASSEMBLY_THREADS=OFF', '-Donnxruntime_ENABLE_WEBASSEMBLY_DEBUG_INFO=OFF', '-Donnxruntime_ENABLE_WEBASSEMBLY_PROFILING=OFF', '-Donnxruntime_ENABLE_LAZY_TENSOR=OFF', '-Donnxruntime_ENABLE_EXTERNAL_CUSTOM_OP_SCHEMAS=OFF', '-Donnxruntime_ENABLE_CUDA_PROFILING=OFF', '-Donnxruntime_ENABLE_ROCM_PROFILING=OFF', '-Donnxruntime_USE_XNNPACK=OFF', '-Donnxruntime_USE_CANN=OFF', '-Donnxruntime_NVCC_THREADS=1', '-DCUDA_CUDA_LIBRARY=/usr/local/cuda/lib64/stubs', '-Donnxruntime_CROSS_COMPILING=ON', '-DCMAKE_TLS_VERIFY=ON', '-DFETCHCONTENT_QUIET=OFF', '-Donnxruntime_USE_CUDA=ON', '-Donnxruntime_CUDA_HOME=/usr/local/cuda', '-Donnxruntime_CUDNN_HOME=/usr/lib/aarch64-linux-gnu', '-Donnxruntime_PYBIND_EXPORT_OPSCHEMA=OFF', '-Donnxruntime_ENABLE_MEMLEAK_CHECKER=OFF', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '']' returned non-zero exit status 1.

Visual Studio Version

No response

GCC / Compiler Version

No response

tianleiwu commented 1 year ago

2023-07-04 08:41:20,159 tools_python_utils [INFO] - flatbuffers module is not installed. parse_config will not be available

Try the following before building:

pip install -r requirements-dev.txt
pip install cmake
bruno-darochac commented 1 year ago

2023-07-04 08:41:20,159 tools_python_utils [INFO] - flatbuffers module is not installed. parse_config will not be available

Try the following before building:

pip install -r requirements-dev.txt
pip install cmake

Many thanks for your answer ! Now I'm able to build the lib. But still a problem, after the building I've only 63% of tested passed : 

63% tests passed, 3 tests failed out of 8

Total Test time (real) =  29.59 sec

The following tests FAILED:
      1 - onnxruntime_test_all (Subprocess aborted)
      4 - onnxruntime_shared_lib_test (Subprocess aborted)
      5 - onnxruntime_global_thread_pools_test (Failed)

Have you any idea of why ?

baijumeswani commented 1 year ago

Do you need to run all the tests? I would recommend using the build flag --skip_tests to avoid running all tests unless you have all test dependencies in place. Then, run only those tests that you're interested in with the gtest filter.

If you need to run all tests, please share the error messages here so we can guide you.

snnn commented 1 year ago

  1. The “--arm64” option is for Windows only. On Linux we do not support cross-compiling in that way. Setting up a cross-compiling environment on Linux is much complicated. If you want to give a try, find the document at: https://onnxruntime.ai/docs/build/inferencing.html

  2. Jeton is a bit different. Search the word "Jetson" in our Github issues. There are a lot of discussions. Some of them might be helpful for you.