[Build] impossible to build onnxruntime with vs2022

[megachomba]

Describe the issue

impossible to build onnx with vs2022 build tool + cuda 11.6 + cudnn 8.5.0. cuda and cudnn correctly installed, i checkout on 1.13.1 as specified in here run this bat ->

.\build.bat --config Release --build_nuget --parallel --cmake_generator "Visual Studio 17 2022" --use_cuda --cuda_version 11.6 --cudnn_home "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6" --cuda_home "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6"

Urgency

High, running inference is the only step missing in the project and deadline is end of week. it should've been a trivial installation but seems is not.

Target platform

windows x64

Build script

.\build.bat --config Release --build_nuget --parallel --skip_tests --cmake_generator "Visual Studio 17 2022" --use_cuda --cuda_version 11.6 --cudnn_home "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6" --cuda_home "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6"

Error / output

C:\cpplibs\onnx\onnxruntime\build\Windows\Release_deps\abseil_cpp-src\absl\strings\internal\cord_rep_btree_navigator.cc(155,27): warning C4267: '=': conversion from 'size_t' to 'uint8_t', possible loss of data [C:\cpplibs\onnx\onnxruntime\build\Windows\Re lease\external\abseil-cpp\absl\strings\absl_cord_internal.vcxproj] C:\cpplibs\onnx\onnxruntime\build\Windows\Release_deps\abseil_cpp-src\absl\strings\internal\cord_rep_btree_navigator.cc(179,20): warning C4267: '=': conversion from 'size_t' to 'uint8_t', possible loss of data [C:\cpplibs\onnx\onnxruntime\build\Windows\Re lease\external\abseil-cpp\absl\strings\absl_cord_internal.vcxproj] C:\cpplibs\onnx\onnxruntime\build\Windows\Release_deps\abseil_cpp-src\absl\time\time.cc(319,17): warning C4244: '=': conversion from 'time_t' to 'long', possible loss of data [C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abseil-cpp\absl\time \absl_time.vcxproj] C:\cpplibs\onnx\onnxruntime\build\Windows\Release_deps\abseil_cpp-src\absl\time\clock.cc(561,18): warning C4244: 'argument': conversion from 'int64_t' to 'DWORD', possible loss of data [C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abseil-cpp \absl\time\absl_time.vcxproj] absl_hash.vcxproj -> C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abseil-cpp\absl\hash\Release\absl_hash.lib CUDACOMPILE : nvcc warning : The 'compute_35', 'compute_37', 'sm_35', and 'sm_37' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning). [C:\cpplibs\onnx\onnxruntime\build\Windows\Release \custom_op_library.vcxproj] cuda_ops.cu C:\cpplibs\onnx\onnxruntime\build\Windows\Release_deps\abseil_cpp-src\absl\time\duration.cc(212,36): warning C4244: 'argument': conversion from 'double' to 'int64_t', possible loss of data [C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abseil -cpp\absl\time\absl_time.vcxproj] C:\cpplibs\onnx\onnxruntime\build\Windows\Release_deps\abseil_cpp-src\absl\time\duration.cc(648,17): warning C4244: '=': conversion from 'time_t' to 'long', possible loss of data [C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abseil-cpp\absl\ time\absl_time.vcxproj] C:\cpplibs\onnx\onnxruntime\build\Windows\Release_deps\abseil_cpp-src\absl\time\duration.cc(744,21): warning C4244: 'initializing': conversion from 'double' to 'int64_t', possible loss of data [C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\ab seil-cpp\absl\time\absl_time.vcxproj] C:\cpplibs\onnx\onnxruntime\build\Windows\Release_deps\abseil_cpp-src\absl\time\duration.cc(745,20): warning C4244: 'initializing': conversion from 'double' to 'int64_t', possible loss of data [C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\ab seil-cpp\absl\time\absl_time.vcxproj] C:\cpplibs\onnx\onnxruntime\build\Windows\Release_deps\abseil_cpp-src\absl\time\duration.cc(249,46): warning C4244: 'argument': conversion from 'int64_t' to 'const _Ty', possible loss of data [C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abs eil-cpp\absl\time\absl_time.vcxproj] with [ _Ty=double ] C:\cpplibs\onnx\onnxruntime\build\Windows\Release_deps\abseil_cpp-src\absl\time\duration.cc(460,55): message : see reference to function template instantiation 'absl::lts_20211102::Duration absl::lts_20211102::`anonymous-namespace'::ScaleDouble<std::multi plies>(absl::lts_20211102::Duration,double)' being compiled [C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abseil-cpp\absl\time\absl_time.vcxproj] C:\cpplibs\onnx\onnxruntime\build\Windows\Release_deps\abseil_cpp-src\absl\time\duration.cc(263,16): warning C4244: 'initializing': conversion from 'double' to 'int64_t', possible loss of data [C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\ab seil-cpp\absl\time\absl_time.vcxproj] C:\cpplibs\onnx\onnxruntime\build\Windows\Release_deps\abseil_cpp-src\absl\time\duration.cc(268,32): warning C4244: 'argument': conversion from 'int64_t' to 'double', possible loss of data [C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abseil -cpp\absl\time\absl_time.vcxproj] C:\cpplibs\onnx\onnxruntime\build\Windows\Release_deps\abseil_cpp-src\absl\time\duration.cc(268,21): warning C4244: 'argument': conversion from 'int64_t' to 'double', possible loss of data [C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abseil -cpp\absl\time\absl_time.vcxproj] absl_symbolize.vcxproj -> C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abseil-cpp\absl\debugging\Release\absl_symbolize.lib custom_op_library.cc absl_cord_internal.vcxproj -> C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abseil-cpp\absl\strings\Release\absl_cord_internal.lib absl_time.vcxproj -> C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abseil-cpp\absl\time\Release\absl_time.lib Building Custom Rule C:/cpplibs/onnx/onnxruntime/build/Windows/Release/_deps/abseil_cpp-src/absl/synchronization/CMakeLists.txt barrier.cc blocking_counter.cc create_thread_identity.cc per_thread_sem.cc waiter.cc notification.cc mutex.cc flatbuffers.vcxproj -> C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\flatbuffers\Release\flatbuffers.lib Creating library C:/cpplibs/onnx/onnxruntime/build/Windows/Release/Release/custom_op_library.lib and object C:/cpplibs/onnx/onnxruntime/build/Windows/Release/Release/custom_op_library.exp custom_op_library.vcxproj -> C:\cpplibs\onnx\onnxruntime\build\Windows\Release\Release\custom_op_library.dll C:\cpplibs\onnx\onnxruntime\build\Windows\Release_deps\abseil_cpp-src\absl\synchronization\mutex.cc(1286,18): warning C4267: '+=': conversion from 'size_t' to 'int', possible loss of data [C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abseil- cpp\absl\synchronization\absl_synchronization.vcxproj] absl_synchronization.vcxproj -> C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abseil-cpp\absl\synchronization\Release\absl_synchronization.lib Building Custom Rule C:/cpplibs/onnx/onnxruntime/build/Windows/Release/_deps/abseil_cpp-src/absl/container/CMakeLists.txt Building Custom Rule C:/cpplibs/onnx/onnxruntime/build/Windows/Release/_deps/abseil_cpp-src/absl/strings/CMakeLists.txt hashtablez_sampler.cc hashtablez_sampler_force_weak_definition.cc cordz_handle.cc absl_hashtablez_sampler.vcxproj -> C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abseil-cpp\absl\container\Release\absl_hashtablez_sampler.lib Building Custom Rule C:/cpplibs/onnx/onnxruntime/build/Windows/Release/_deps/abseil_cpp-src/absl/container/CMakeLists.txt absl_cordz_handle.vcxproj -> C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abseil-cpp\absl\strings\Release\absl_cordz_handle.lib raw_hash_set.cc Building Custom Rule C:/cpplibs/onnx/onnxruntime/build/Windows/Release/_deps/abseil_cpp-src/absl/strings/CMakeLists.txt cordz_info.cc absl_raw_hash_set.vcxproj -> C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abseil-cpp\absl\container\Release\absl_raw_hash_set.lib absl_cordz_info.vcxproj -> C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abseil-cpp\absl\strings\Release\absl_cordz_info.lib Building Custom Rule C:/cpplibs/onnx/onnxruntime/build/Windows/Release/_deps/abseil_cpp-src/absl/strings/CMakeLists.txt cord.cc absl_cord.vcxproj -> C:\cpplibs\onnx\onnxruntime\build\Windows\Release\external\abseil-cpp\absl\strings\Release\absl_cord.lib Traceback (most recent call last): File "C:\cpplibs\onnx\onnxruntime\tools\ci_build\build.py", line 2812, in sys.exit(main()) File "C:\cpplibs\onnx\onnxruntime\tools\ci_build\build.py", line 2727, in main build_targets(args, cmake_path, build_dir, configs, num_parallel_jobs, args.target) File "C:\cpplibs\onnx\onnxruntime\tools\ci_build\build.py", line 1349, in build_targets run_subprocess(cmd_args, env=env) File "C:\cpplibs\onnx\onnxruntime\tools\ci_build\build.py", line 740, in run_subprocess return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env) File "C:\cpplibs\onnx\onnxruntime\tools\python\util\run.py", line 57, in run shell=shell, File "C:\Users\noname\AppData\Local\Programs\Python\Python37\lib\subprocess.py", line 468, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['C:\Program Files\CMake\bin\cmake.EXE', '--build', 'C:\cpplibs\onnx\onnxruntime\\build\Windows\Release', '--config', 'Release', '--', '/maxcpucount:20', '/nodeReuse:False']' returned non-zero exit status 1.

Visual Studio Version

2022

GCC / Compiler Version

No response

[yuslepukhin]

We build it everyday with VS 2022. Try RelWithDebInfo configuration. Any reason why pre-built package would not work for you?

[wschin]

I think another problem is that the produced build error message is really not helpful (for a long while). I don't know what's going on and I can build with VS2022 on my side. This is my build command;

.\build.bat --config RelWithDebInfo --build_shared_lib --parallel --enable_training --cmake_generator "Visual Studio 17 2022" --use_cuda --cudnn_home "C:\*\cudnn-windows-x86_64-8.4.1.50_cuda11.6-archive" --cuda_home "C:\*\NVIDIA GPU Computing Toolkit\CUDA\v11.7" --cuda_version=11.7 --build_java

[zhijxu-MS]

@megachomba is it the build error still happen in your env? did you try remove the build dir and try again?

is the shared log is all the output log, i can only see warning not error from the log. you could try build again after fail and the output log should be less because of incremental compilation

[SimonRelu]

Hey, I know this is an old issue but I also recently ran into this. The issue is that you are using a 32 bit python install instead of 64 bit