search

k2-fsa / k2

FSA/FST algorithms, differentiable, with PyTorch compatibility.
https://k2-fsa.github.io/k2
Apache License 2.0
1.08k stars 211 forks source link

Fix building cuda version of k2 on Windows #1257

Closed csukuangfj closed 8 months ago

csukuangfj commented 8 months ago

See https://github.com/k2-fsa/k2/issues/1256#issuecomment-1783762489 and https://gist.github.com/mc-marcocheng/badc1cbaee2ceaa05a6c304439899e29

      C:/Users/Mc/AppData/Local/Temp/pip-req-build-y_nc_tt1\k2/csrc/log.h(90): error : expected an identifier [C:\Users\Mc\AppData\Local\Temp\pip-req-build-y_nc_tt1\build\temp.win-amd64-cpython-310\Release\k2\csrc\context.vcxproj]
          constexpr LogLevel 0 = LogLevel::kError;
                             ^

      C:/Users/Mc/AppData/Local/Temp/pip-req-build-y_nc_tt1\k2/csrc/log.h(141): error : this constant expression has type "int" instead of the required "k2::internal::LogLevel" type [C:\Users\Mc\AppData\Local\Temp\pip-req-build-y_nc_tt1\build\temp.win-amd64-cpython-310\Release\k2\csrc\context.vcxproj]
                case 0:
                     ^

      C:/Users/Mc/AppData/Local/Temp/pip-req-build-y_nc_tt1\k2/csrc/log.h(142): error : no operator "<=" matches these operands [C:\Users\Mc\AppData\Local\Temp\pip-req-build-y_nc_tt1\build\temp.win-amd64-cpython-310\Release\k2\csrc\context.vcxproj]
                    operand types are: k2::internal::LogLevel <= int
                  if (cur_level_ <= 0) printf("[E] ");
mc-marcocheng commented 8 months ago

Yes I am able to build and install it. Thanks

jerrypcl commented 8 months ago

Great! Thank you very much for the efforts. The release version is all set. Debug version is still having problem...

csukuangfj commented 8 months ago

Could you post detailed error logs for the Debug version?

jerrypcl commented 8 months ago

1 first of all, I use set -DCMAKE_BUILD_TYPE="Debug"
2 it seems it does not really using debug mode (from the following command issued by the scripts) cmake --build build\temp.win-amd64-cpython-311\Release --target install --config Release -- -m

3 then quickly the following error D:\experiment\k2_31_10_2003\k2\build\temp.win-amd64-cpython-311\Release\k2\torch\csrc>"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11. 7\bin\nvcc.exe" --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\HostX64\x 64" -x cu -ID:\experiment\k2_31_10_2003\k2 -IC:\Users\pcl\AppData\Local\Programs\Python\Python311\Include -I"D:\experiment\k2_31_10_2003\k2\ build\temp.win-amd64-cpython-311\Release_deps\moderngpu-src\src" -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\include\nvtx3" - I"C:\Users\pcl\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\include" -I"C:\Users\pcl\AppData\Local\Programs\Python\Python31 1\Lib\site-packages\torch\include\torch\csrc\api\include" -I"C:\Program Files\NVIDIA Corporation\NvToolsExt\include" -I"C:\Program Files\NVID IA GPU Computing Toolkit\CUDA\v11.7\include" -I"C:\Program Files\NVIDIA Corporation\NvToolsExt\include" -I"C:\Program Files\NVIDIA GPU Computi ng Toolkit\CUDA\v11.7\include" --keep-dir x64\Release -use_fast_math -maxrregcount=0 --machine 64 --compile -cudart static --expt-relaxed -constexpr -Wno-deprecated-gpu-targets -lineinfo --expt-extended-lambda -Xptxas=-w --expt-extended-lambda -gencode arch=compute_86,code=sm_86 -gencode arch=compute_86,code=sm_86 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=integer_sign_change,--diag_suppress=useless_us ing_declaration,--diag_suppress=set_but_not_used,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_inte rface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=implicit_re turn_from_non_void_function,--diag_suppress=unsigned_compare_with_zero,--diag_suppress=declared_but_not_referenced,--diag_suppress=badfriend decl --Werror cross-execution-space-call --no-host-device-move-forward --expt-relaxed-constexpr --expt-extended-lambda -std=c++14 -Xcompiler=" /EHsc /wd4005 /wd4018 /wd4067 /wd4068 /wd4099 /wd4101 /wd4190 /wd4224 /wd4244 /wd4251 /wd4267 /wd4275 /wd4305 /wd4522 /wd4551 /wd4624 /wd4700 /wd4722 /wd4805 /wd4819 /wd4838 /wd4996 -Ob2" -D_WINDOWS -DONNX_NAMESPACE=onnx_c2 -DNDEBUG -DK2_USE_PYTORCH -DK2_WITHCUDA -DNOMINMAX -DUSE DISTRIBUTED -DUSE_C10D_GLOO -DK2_TORCH_VERSION_MAJOR=2 -DK2_TORCH_VERSION_MINOR=0 -DCUB_WRAPPED_NAMESPACE=k2 -DTHRUST_NS_QUALIFIER=thrust -DK2 _ENABLE_NVTX=1 -D"CMAKE_INTDIR=\"Release\"" -D_MBCS -DWIN32 -D_WINDOWS -DNDEBUG -DK2_USE_PYTORCH -DK2_WITH_CUDA -DNOMINMAX -DUSE_DISTRIBUTED - DUSE_C10D_GLOO -DK2_TORCH_VERSION_MAJOR=2 -DK2_TORCH_VERSION_MINOR=0 -DCUB_WRAPPED_NAMESPACE=k2 -DTHRUST_NS_QUALIFIER=thrust -DK2_ENABLE_NVTX= 1 -D"CMAKE_INTDIR=\"Release\"" -Xcompiler "/EHsc /W3 /nologo /O2 /FdD:\experiment\k2_31_10_2003\k2\build\temp.win-amd64-cpython-311\Release\li b\Release\k2_torch_api.pdb /FS /MD /GR" -o k2_torch_api.dir\Release\torch_api.obj "D:\experiment\k2_31_10_2003\k2\k2\torch\csrc\torch_api.cu " torch_api.cu k2_torch_api.vcxproj -> D:\experiment\k2_31_10_2003\k2\build\temp.win-amd64-cpython-311\Release\lib\Release\k2_torch_api.lib Creating library D:/experiment/k2_31_10_2003/k2/build/temp.win-amd64-cpython-311/Release/lib/Release/pruned_stateless_transducer.lib and ob ject D:/experiment/k2_31_10_2003/k2/build/temp.win-amd64-cpython-311/Release/lib/Release/pruned_stateless_transducer.exp LINK : warning LNK4098: defaultlib 'LIBCMT' conflicts with use of other libs; use /NODEFAULTLIB:library [D:\experiment\k2_31_10_2003\k2\build\te mp.win-amd64-cpython-311\Release\k2\torch\bin\pruned_stateless_transducer.vcxproj] Creating library D:/experiment/k2_31_10_2003/k2/build/temp.win-amd64-cpython-311/Release/lib/Release/online_decode.lib and object D:/experi ment/k2_31_10_2003/k2/build/temp.win-amd64-cpython-311/Release/lib/Release/online_decode.exp Creating library D:/experiment/k2_31_10_2003/k2/build/temp.win-amd64-cpython-311/Release/lib/Release/attention_rescore.lib and object D:/ex periment/k2_31_10_2003/k2/build/temp.win-amd64-cpython-311/Release/lib/Release/attention_rescore.exp Creating library D:/experiment/k2_31_10_2003/k2/build/temp.win-amd64-cpython-311/Release/lib/Release/ngram_lm_rescore.lib and object D:/exp eriment/k2_31_10_2003/k2/build/temp.win-amd64-cpython-311/Release/lib/Release/ngram_lm_rescore.exp Creating library D:/experiment/k2_31_10_2003/k2/build/temp.win-amd64-cpython-311/Release/lib/Release/ctc_decode.lib and object D:/experimen t/k2_31_10_2003/k2/build/temp.win-amd64-cpython-311/Release/lib/Release/ctc_decode.exp Creating library D:/experiment/k2_31_10_2003/k2/build/temp.win-amd64-cpython-311/Release/lib/Release/rnnt_demo.lib and object D:/experiment /k2_31_10_2003/k2/build/temp.win-amd64-cpython-311/Release/lib/Release/rnnt_demo.exp Creating library D:/experiment/k2_31_10_2003/k2/build/temp.win-amd64-cpython-311/Release/lib/Release/hlg_decode.lib and object D:/experimen t/k2_31_10_2003/k2/build/temp.win-amd64-cpython-311/Release/lib/Release/hlg_decode.exp LINK : warning LNK4098: defaultlib 'LIBCMT' conflicts with use of other libs; use /NODEFAULTLIB:library [D:\experiment\k2_31_10_2003\k2\build\te mp.win-amd64-cpython-311\Release\k2\torch\bin\online_decode.vcxproj] LINK : warning LNK4098: defaultlib 'LIBCMT' conflicts with use of other libs; use /NODEFAULTLIB:library [D:\experiment\k2_31_10_2003\k2\build\te mp.win-amd64-cpython-311\Release\k2\torch\bin\attention_rescore.vcxproj] LINK : warning LNK4098: defaultlib 'LIBCMT' conflicts with use of other libs; use /NODEFAULTLIB:library [D:\experiment\k2_31_10_2003\k2\build\te mp.win-amd64-cpython-311\Release\k2\torch\bin\ngram_lm_rescore.vcxproj] LINK : warning LNK4098: defaultlib 'LIBCMT' conflicts with use of other libs; use /NODEFAULTLIB:library [D:\experiment\k2_31_10_2003\k2\build\te mp.win-amd64-cpython-311\Release\k2\torch\bin\ctc_decode.vcxproj] LINK : warning LNK4098: defaultlib 'LIBCMT' conflicts with use of other libs; use /NODEFAULTLIB:library [D:\experiment\k2_31_10_2003\k2\build\te mp.win-amd64-cpython-311\Release\k2\torch\bin\rnnt_demo.vcxproj] LINK : warning LNK4098: defaultlib 'LIBCMT' conflicts with use of other libs; use /NODEFAULTLIB:library [D:\experiment\k2_31_10_2003\k2\build\te mp.win-amd64-cpython-311\Release\k2\torch\bin\hlg_decode.vcxproj] k2context.lib(pytorch_context.obj) : error LNK2019: unresolved external symbol cudaRegisterLinkedBinary_ba435483_18_pytorch_context_cu_9ac040b 0 referenced in function "void cdecl sti__cudaRegisterAll(void)" (?sti__cudaRegisterAll@@YAXXZ) [D:\experiment\k2_31_10_2003\k2\build\ temp.win-amd64-cpython-311\Release\k2\torch\bin\pruned_stateless_transducer.vcxproj] k2context.lib(pytorch_context.obj) : error LNK2019: unresolved external symbol cudaRegisterLinkedBinary_ba435483_18_pytorch_context_cu_9ac040b 0 referenced in function "void cdecl sti__cudaRegisterAll(void)" (?sti__cudaRegisterAll@@YAXXZ) [D:\experiment\k2_31_10_2003\k2\build\ temp.win-amd64-cpython-311\Release\k2\torch\bin\ngram_lm_rescore.vcxproj]

csukuangfj commented 8 months ago

@jerrypcl Please try https://github.com/k2-fsa/k2/pull/1259