search

abetlen / llama-cpp-python

Python bindings for llama.cpp
https://llama-cpp-python.readthedocs.io
MIT License
7.64k stars 918 forks source link

unsupported Microsoft Visual Studio version #1586

Open sardetushar opened 1 month ago

sardetushar commented 1 month ago

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\include\crt/host_config.h(153): fatal error C1189: #error: -- unsupported Microsoft Visual Studio version! Only the versions between 2017 and 2022 (inclusive) are supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk. [C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\cmTC_d9b18.vcxproj]

I have visual studio 2022 also installed build tool but no luck, i am using following command

set CMAKE_ARGS=-DGGML_CUDA=on ^
-DCUDAToolkit_VERSION=12.1 ^
-DCUDA_PATH="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1" ^
-DCUDAToolkit_ROOT="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1" ^
-DCUDAToolkit_INCLUDE_DIR="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\include" ^
-DCUDAToolkit_LIBRARY_DIR="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\x64" ^
-DCMAKE_CUDA_COMPILER="C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1/bin/nvcc.exe" ^
-DCMAKE_CUDA_STANDARD_LIBRARIES="C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1/lib/x64/cudart_static.lib" ^
-DCMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES="C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1/include"
set FORCE_CMAKE=1
pip install llama-cpp-python --force-reinstall --no-cache-dir

Full error log

`pip install llama-cpp-python --force-reinstall --no-cache-dir Collecting llama-cpp-python Downloading llama_cpp_python-0.2.82.tar.gz (50.7 MB) ---------------------------------------- 50.7/50.7 MB 3.8 MB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing metadata (pyproject.toml) ... done Collecting typing-extensions>=4.5.0 (from llama-cpp-python) Downloading typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB) Collecting numpy>=1.20.0 (from llama-cpp-python) Downloading numpy-2.0.0-cp311-cp311-win_amd64.whl.metadata (60 kB) ---------------------------------------- 60.9/60.9 kB ? eta 0:00:00 Collecting diskcache>=5.6.1 (from llama-cpp-python) Downloading diskcache-5.6.3-py3-none-any.whl.metadata (20 kB) Collecting jinja2>=2.11.3 (from llama-cpp-python) Downloading jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB) Collecting MarkupSafe>=2.0 (from jinja2>=2.11.3->llama-cpp-python) Downloading MarkupSafe-2.1.5-cp311-cp311-win_amd64.whl.metadata (3.1 kB) Downloading diskcache-5.6.3-py3-none-any.whl (45 kB) ---------------------------------------- 45.5/45.5 kB ? eta 0:00:00 Downloading jinja2-3.1.4-py3-none-any.whl (133 kB) ---------------------------------------- 133.3/133.3 kB ? eta 0:00:00 Downloading numpy-2.0.0-cp311-cp311-win_amd64.whl (16.5 MB) ---------------------------------------- 16.5/16.5 MB 3.4 MB/s eta 0:00:00 Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB) Downloading MarkupSafe-2.1.5-cp311-cp311-win_amd64.whl (17 kB) Building wheels for collected packages: llama-cpp-python Building wheel for llama-cpp-python (pyproject.toml) ... error error: subprocess-exited-with-error

× Building wheel for llama-cpp-python (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [109 lines of output] scikit-build-core 0.9.8 using CMake 3.30.0 (wheel) Configuring CMake... 2024-07-10 20:33:04,443 - scikit_build_core - WARNING - Can't find a Python library, got libdir=None, ldlibrary=None, multiarch=None, masd=None loading initial cache file C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeInit.txt -- Building for: Visual Studio 17 2022 -- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.19045. -- The C compiler identification is MSVC 19.40.33812.0 -- The CXX compiler identification is MSVC 19.40.33812.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.40.33807/bin/Hostx64/x64/cl.exe - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.40.33807/bin/Hostx64/x64/cl.exe - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.45.2.windows.1") -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - not found -- Found Threads: TRUE -- Found OpenMP_C: -openmp (found version "2.0") -- Found OpenMP_CXX: -openmp (found version "2.0") -- Found OpenMP: TRUE (found version "2.0") -- OpenMP found -- Using ggml SGEMM -- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.8/include (found version "11.8.89") -- CUDA found -- Using CUDA architectures: 52;61;70;75 -- The CUDA compiler identification is NVIDIA 11.8.89 -- Detecting CUDA compiler ABI info -- Detecting CUDA compiler ABI info - failed -- Check for working CUDA compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.8/bin/nvcc.exe -- Check for working CUDA compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.8/bin/nvcc.exe - broken CMake Error at C:/Users/pc1/AppData/Local/Temp/pip-build-env-vh06lqz3/normal/Lib/site-packages/cmake/data/share/cmake-3.30/Modules/CMakeTestCUDACompiler.cmake:59 (message): The CUDA compiler

      "C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.8/bin/nvcc.exe"

    is not able to compile a simple test program.

    It fails with the following output:

      Change Dir: 'C:/Users/pc1/AppData/Local/Temp/tmp6zdbj_1l/build/CMakeFiles/CMakeScratch/TryCompile-8swq4p'

      Run Build Command(s): "C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/MSBuild/Current/Bin/amd64/MSBuild.exe" cmTC_d9b18.vcxproj /p:Configuration=Debug /p:Platform=x64 /p:VisualStudioVersion=17.0 /v:n
      MSBuild version 17.10.4+10fbfbf2e for .NET Framework
      Build started 7/10/2024 8:33:26 PM.

      Project "C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\cmTC_d9b18.vcxproj" on node 1 (default targets).
      PrepareForBuild:
        Creating directory "cmTC_d9b18.dir\Debug\".
      C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\MSBuild\Microsoft\VC\v170\Microsoft.CppBuild.targets(541,5): warning MSB8029: The Intermediate directory or Output directory cannot reside under the Temporary directory as it could lead to issues with incremental build. [C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\cmTC_d9b18.vcxproj]
        Structured output is enabled. The formatting of compiler diagnostics will reflect the error hierarchy. See https://aka.ms/cpp/structured-output for more details.
        Creating directory "C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\Debug\".
        Creating directory "cmTC_d9b18.dir\Debug\cmTC_d9b18.tlog\".
      InitializeBuildStatus:
        Creating "cmTC_d9b18.dir\Debug\cmTC_d9b18.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
        Touching "cmTC_d9b18.dir\Debug\cmTC_d9b18.tlog\unsuccessfulbuild".
      AddCudaCompileDeps:
        C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64\cl.exe /E /nologo /showIncludes /TP /D__CUDACC__ /D__CUDACC_VER_MAJOR__=11 /D__CUDACC_VER_MINOR__=8 /D_WINDOWS /DCMAKE_INTDIR="Debug" /D_MBCS /DCMAKE_INTDIR="Debug" /I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin" /I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\include" /I. /FIcuda_runtime.h /c C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\main.cu
      Project "C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\cmTC_d9b18.vcxproj" (1) is building "C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\cmTC_d9b18.vcxproj" (1:2) on node 1 (CudaBuildCore target(s)).
      CudaBuildCore:
        Compiling CUDA source file main.cu...
        cmd.exe /C "C:\Users\pc1\AppData\Local\Temp\tmpb218d8d5b2b54075895d08584d18d864.cmd"
        "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin\nvcc.exe"  --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64" -x cu    -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\include"     --keep-dir cmTC_d9b18\x64\Debug  -maxrregcount=0  --machine 64 --compile -cudart static --generate-code=arch=compute_52,code=[compute_52,sm_52] --generate-code=arch=compute_61,code=[compute_61,sm_61] --generate-code=arch=compute_70,code=[compute_70,sm_70] --generate-code=arch=compute_75,code=[compute_75,sm_75] -Xcompiler="/EHsc -Zi -Ob0" -g  -D_WINDOWS -D"CMAKE_INTDIR=\"Debug\"" -D_MBCS -D"CMAKE_INTDIR=\"Debug\"" -Xcompiler "/EHsc /W1 /nologo /Od /FdcmTC_d9b18.dir\Debug\vc143.pdb /FS /Zi /RTC1 /MDd " -o cmTC_d9b18.dir\Debug\main.obj "C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\main.cu"

        (rag_tut) C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p>"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin\nvcc.exe"  --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64" -x cu    -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\include"     --keep-dir cmTC_d9b18\x64\Debug  -maxrregcount=0  --machine 64 --compile -cudart static --generate-code=arch=compute_52,code=[compute_52,sm_52] --generate-code=arch=compute_61,code=[compute_61,sm_61] --generate-code=arch=compute_70,code=[compute_70,sm_70] --generate-code=arch=compute_75,code=[compute_75,sm_75] -Xcompiler="/EHsc -Zi -Ob0" -g  -D_WINDOWS -D"CMAKE_INTDIR=\"Debug\"" -D_MBCS -D"CMAKE_INTDIR=\"Debug\"" -Xcompiler "/EHsc /W1 /nologo /Od /FdcmTC_d9b18.dir\Debug\vc143.pdb /FS /Zi /RTC1 /MDd " -o cmTC_d9b18.dir\Debug\main.obj "C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\main.cu"
      C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\include\crt/host_config.h(153): fatal error C1189: #error:  -- unsupported Microsoft Visual Studio version! Only the versions between 2017 and 2022 (inclusive) are supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk. [C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\cmTC_d9b18.vcxproj]
        main.cu
      C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\MSBuild\Microsoft\VC\v170\BuildCustomizations\CUDA 11.8.targets(785,9): error MSB3721: The command ""C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin\nvcc.exe"  --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64" -x cu    -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\include"     --keep-dir cmTC_d9b18\x64\Debug  -maxrregcount=0  --machine 64 --compile -cudart static --generate-code=arch=compute_52,code=[compute_52,sm_52] --generate-code=arch=compute_61,code=[compute_61,sm_61] --generate-code=arch=compute_70,code=[compute_70,sm_70] --generate-code=arch=compute_75,code=[compute_75,sm_75] -Xcompiler="/EHsc -Zi -Ob0" -g  -D_WINDOWS -D"CMAKE_INTDIR=\"Debug\"" -D_MBCS -D"CMAKE_INTDIR=\"Debug\"" -Xcompiler "/EHsc /W1 /nologo /Od /FdcmTC_d9b18.dir\Debug\vc143.pdb /FS /Zi /RTC1 /MDd " -o cmTC_d9b18.dir\Debug\main.obj "C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\main.cu"" exited with code 2. [C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\cmTC_d9b18.vcxproj]
      Done Building Project "C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\cmTC_d9b18.vcxproj" (CudaBuildCore target(s)) -- FAILED.
      Done Building Project "C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\cmTC_d9b18.vcxproj" (default targets) -- FAILED.

      Build FAILED.

      "C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\cmTC_d9b18.vcxproj" (default target) (1) ->
      (PrepareForBuild target) ->
        C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\MSBuild\Microsoft\VC\v170\Microsoft.CppBuild.targets(541,5): warning MSB8029: The Intermediate directory or Output directory cannot reside under the Temporary directory as it could lead to issues with incremental build. [C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\cmTC_d9b18.vcxproj]

      "C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\cmTC_d9b18.vcxproj" (default target) (1) ->
      "C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\cmTC_d9b18.vcxproj" (CudaBuildCore target) (1:2) ->
      (CudaBuildCore target) ->
        C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\include\crt/host_config.h(153): fatal error C1189: #error:  -- unsupported Microsoft Visual Studio version! Only the versions between 2017 and 2022 (inclusive) are supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk. [C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\cmTC_d9b18.vcxproj]
        C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\MSBuild\Microsoft\VC\v170\BuildCustomizations\CUDA 11.8.targets(785,9): error MSB3721: The command ""C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin\nvcc.exe"  --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64" -x cu    -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\include"     --keep-dir cmTC_d9b18\x64\Debug  -maxrregcount=0  --machine 64 --compile -cudart static --generate-code=arch=compute_52,code=[compute_52,sm_52] --generate-code=arch=compute_61,code=[compute_61,sm_61] --generate-code=arch=compute_70,code=[compute_70,sm_70] --generate-code=arch=compute_75,code=[compute_75,sm_75] -Xcompiler="/EHsc -Zi -Ob0" -g  -D_WINDOWS -D"CMAKE_INTDIR=\"Debug\"" -D_MBCS -D"CMAKE_INTDIR=\"Debug\"" -Xcompiler "/EHsc /W1 /nologo /Od /FdcmTC_d9b18.dir\Debug\vc143.pdb /FS /Zi /RTC1 /MDd " -o cmTC_d9b18.dir\Debug\main.obj "C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\main.cu"" exited with code 2. [C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\cmTC_d9b18.vcxproj]

          1 Warning(s)
          2 Error(s)

      Time Elapsed 00:00:01.05

    CMake will not be able to correctly generate this project.
  Call Stack (most recent call first):
    vendor/llama.cpp/ggml/src/CMakeLists.txt:271 (enable_language)

  -- Configuring incomplete, errors occurred!

  *** CMake configuration failed
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for llama-cpp-python Failed to build llama-cpp-python ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects`

gformcreation commented 1 month ago

Hi I dont know the details but for now you can utilize prebuilt package. Ref :- https://github.com/abetlen/llama-cpp-python#:~:text=llama%2Dcpp%2Dpython-,Pre%2Dbuilt%20Wheel%20(New),Python%20Version%20is%203.10%2C%203.11%20or%203.12,-pip%20install%20llama

Assuming you are using cuda 12.1 based on the logs

Use the following command

pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121