Build error coming from onnxruntime_extensions dependency

[alando46]

I'm having a difficult time sorting out an issue related to the audio extensions when compiling onnxruntime-genai from source. my build command is:

2024-11-20 10:43:16 - INFO - Running the following build commands: `['.\\build.bat', '--config', 'Release', '--update', '--build', '--ort_home', 'C:\\Users\\music\\code\\onnxruntime-genai\\ort_cuda_v12.6_build_dir', '--skip_wheel', '--skip_tests', '--parallel', '--use_cuda', '--cuda_home', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.6']`
2024-11-20 10:43:16,822 util.run [INFO] - Running subprocess in 'C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.6_build_dir'
  'C:\Program Files\CMake\bin\cmake.EXE' -G 'Visual Studio 17 2022' -T 'host=x64,cuda=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6' -DCMAKE_BUILD_TYPE=Release -S 'C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.6_build_dir' -B 'C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.6_build_dir\build\Windows\Release' -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DUSE_CUDA=ON -DUSE_ROCM=OFF -DUSE_WEBGPU=OFF -DUSE_DML=OFF -DENABLE_JAVA=OFF -DBUILD_WHEEL=OFF '-DORT_HOME=C:\Users\music\code\onnxruntime-genai\ort_cuda_v12.6_build_dir' '-DCMAKE_CUDA_COMPILER=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6\bin\nvcc'
-- Selecting Windows SDK version 10.0.26100.0 to target Windows 10.0.22631.
-- The C compiler identification is MSVC 19.38.33143.0
-- The CXX compiler identification is MSVC 19.38.33143.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using ONNX Runtime from C:\Users\music\code\onnxruntime-genai\ort_cuda_v12.6_build_dir
-- ORT_HEADER_DIR: C:\Users\music\code\onnxruntime-genai\ort_cuda_v12.6_build_dir/include
-- ORT_LIB_DIR: C:\Users\music\code\onnxruntime-genai\ort_cuda_v12.6_build_dir/lib
...

Eventually this exits after hitting:

------------------Enabling model benchmark------------------
-- Configuring done (29.0s)
-- Generating done (0.2s)
-- Build files have been written to: C:/Users/music/code/onnxruntime-genai/ortgenai_cuda_v12.6_build_dir/build/Windows/Release
PS C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.6_build_dir> & "C:\Program Files\CMake\bin\cmake.EXE" --build "C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.6_build_dir\build\Windows\Release" --config Release --parallel                                                                    
MSBuild version 17.12.6+db5f6012c for .NET Framework                                                                                                                                                                                                                                                                

  Checking File Globs
  1>Checking Build System
  Building Custom Rule C:/Users/music/code/onnxruntime-genai/ortgenai_cuda_v12.6_build_dir/build/Windows/Release/_deps/onnxruntime_extensions-src/CMakeLists.txt
  base64.cc
  ocos.cc
  string_tensor.cc
  string_utils.cc
  audio.cc
  audio_decoder.cc
  Generating Code...
C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.6_build_dir\build\Windows\Release\_deps\onnxruntime_extensions-src\operators\audio\audio_decoder.cc(177,38): error C2653: 'ort_extensions': is not a class or namespace name [C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.6_build_dir\build\Win
dows\Release\_deps\onnxruntime_extensions-build\noexcep_operators.vcxproj]
C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.6_build_dir\build\Windows\Release\_deps\onnxruntime_extensions-src\operators\audio\audio_decoder.cc(177,54): error C2065: 'narrow': undeclared identifier [C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.6_build_dir\build\Windows\Release\_deps\ 
onnxruntime_extensions-build\noexcep_operators.vcxproj]
C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.6_build_dir\build\Windows\Release\_deps\onnxruntime_extensions-src\operators\audio\audio_decoder.cc(177,61): error C2275: 'int64_t': expected an expression instead of a type [C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.6_build_dir\build\Wi 
ndows\Release\_deps\onnxruntime_extensions-build\noexcep_operators.vcxproj]
Traceback (most recent call last):
  File "C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.6_build_dir\\build.py", line 653, in <module>
    build(arguments, environment)
  File "C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.6_build_dir\\build.py", line 585, in build
    util.run(make_command, env=env)
  File "C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.6_build_dir\tools\python\util\run.py", line 56, in run
    completed_process = subprocess.run(
                        ^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['C:\\Program Files\\CMake\\bin\\cmake.EXE', '--build', 'C:\\Users\\music\\code\\onnxruntime-genai\\ortgenai_cuda_v12.6_build_dir\\build\\Windows\\Release', '--config', 'Release', '--parallel']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "C:\Users\music\code\onnxruntime-genai\build_v2.py", line 379, in <module>
    main()
  File "C:\Users\music\code\onnxruntime-genai\build_v2.py", line 340, in main
    build_ortgenai_cuda(
  File "C:\Users\music\code\onnxruntime-genai\build_v2.py", line 287, in build_ortgenai_cuda
    subprocess.run(ort_cuda_build_commands, check=True)
  File "C:\Python312\Lib\subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['.\\build.bat', '--config', 'Release', '--update', '--build', '--ort_home', 'C:\\Users\\music\\code\\onnxruntime-genai\\ort_cuda_v12.6_build_dir', '--skip_wheel', '--skip_tests', '--parallel', '--use_cuda', '--cuda_home', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.6']' returned non-zero exit status 1.

It looks like the error is coming from this line, although I'm not seeing any related issues in onnxruntime-extensions so I assume this is stemming from how i built onnxruntime or perhaps the arguments i passed when building onnxruntime-genai.

Is there anything obvious that I'm missing? Thanks in advance for any suggestions.

[skyline75489]

Hi! We're building for Win-x64 in our CI all the time, for example this.

Could you try cleaning all then perform a CPU build first? Just python build.py --parallel should do

[alando46]

@skyline75489 thanks for your help. I can confirm cpu build works fine for Release and ReleaseWithDebInfo, however & ".\build.bat" --config Release --update --build --ort_home "C:\Users\music\code\onnxruntime-genai\ort_cuda_v12.6_build_dir" --skip_wheel --skip_tests --parallel --use_cuda --cuda_home "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6" fails with the same error as above.

maybe these are only compatible with cuda 12.2? unfortunately MSVC 14.38.33130 is a hard requirement for me so hopefully that's not the issue.

[skyline75489]

It could be the CUDA version. I'm not sure. But good to hear that CPU build works. Unfortunately, we don't have support for 12.6 at this moment. Adding CUDA 12.6 support would require upstream project, that is ORT itself, to supports 12.6. Otherwise, it doesn't make much sense, right?

[alando46]

I will try tomorrow with 12.2. For what it's worth, main of ORT compiles with 12.6 and cuDNN 9.5. Thanks again for your help @skyline75489, i will update here when i try 12.2.

[skyline75489]

Thanks for your feedback! If you find anything wrong with the code, feel free to contact us.

[baijumeswani]

I wonder if it has anything to do with the cxx standard we use to build cuda: https://github.com/microsoft/onnxruntime-genai/blob/d4d3a88dcce8336a1c4635fbbe2225951955e5ea/cmake/cxx_standard.cmake#L5-L7

Could you change that to set(CMAKE_CXX_STANDARD 20) and remove the add_compile_definitions(USE_CXX17=1) for cuda builds? Let me know if that fixes it for you.

[alando46]

@skyline75489 @baijumeswani thanks again for your help. I tried:

1. compiling with 12.2 cuda and 9.3 cudnn. for this i recompiled ort with these values (success) and then passed the 12.2 cuda path and the newly compiled ort (main branch) to the genai build script (also main branch here). i still got the same error.
2. additionally, i tried @baijumeswani 's suggestion for changing the cxx standard to 20 for ortgenai. After doing this, I tried to compile using the same previously built cuda 12.2 ort and passed the same arguments to ortgenai. still getting the same errors.

for reference, the updated cmake/cxx_standard.cmake is:

if ("${CMAKE_C_COMPILER_ID}" STREQUAL "GNU" AND CMAKE_C_COMPILER_VERSION VERSION_LESS 10)
    add_compile_definitions(USE_CXX17=1)
    message("Test is using C++17 because GCC Version is less than 10")
    set(CMAKE_CXX_STANDARD 17)
elseif (USE_CUDA AND CMAKE_CUDA_COMPILER)
    set(CMAKE_CXX_STANDARD 20)
elseif (ANDROID)
    add_compile_definitions(USE_CXX17=1)
    set(CMAKE_CXX_STANDARD 17)
else ()
    message("Test is using C++20")
    set(CMAKE_CXX_STANDARD 20)
endif ()

and the errors are still:

  Generating Code...
C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.2_build_dir\build\Windows\Release\_deps\onnxruntime_extensions-src\operators\audio\audio_decoder.cc(177,38): error C2653:
 'ort_extensions': is not a class or namespace name [C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.2_build_dir\build\Windows\Release\_deps\onnxruntime_extensions-build 
\noexcep_operators.vcxproj]
C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.2_build_dir\build\Windows\Release\_deps\onnxruntime_extensions-src\operators\audio\audio_decoder.cc(177,54): error C2065: 
 'narrow': undeclared identifier [C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.2_build_dir\build\Windows\Release\_deps\onnxruntime_extensions-build\noexcep_operators. 
vcxproj]
C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.2_build_dir\build\Windows\Release\_deps\onnxruntime_extensions-src\operators\audio\audio_decoder.cc(177,61): error C2275: 
 'int64_t': expected an expression instead of a type [C:\Users\music\code\onnxruntime-genai\ortgenai_cuda_v12.2_build_dir\build\Windows\Release\_deps\onnxruntime_extensions-buil 
d\noexcep_operators.vcxproj]
Traceback (most recent call last):

could this be related to my Release build target for ort and ortgenai? I noticed for the nugget builds you are doing RelWithDebInfo.

if helpful, i am pulling a fresh copy of the repo on every build so there shouldn't be any cmake caching issues.

is there a way to exclude building these audio components? for my usecase they aren't even needed so maybe i can skip them in the build process. the other day i tried making some changes to one of the CMakeFiles in the extensions to disable audio but i don't think i did it properly because it was still hitting this error. I also don't need opencv, maybe down the road an option to build for chat only would be helpful.

anyways, thanks for your help in debugging this issue. much appreciated.