Open sardetushar opened 1 month ago
Hi I dont know the details but for now you can utilize prebuilt package. Ref :- https://github.com/abetlen/llama-cpp-python#:~:text=llama%2Dcpp%2Dpython-,Pre%2Dbuilt%20Wheel%20(New),Python%20Version%20is%203.10%2C%203.11%20or%203.12,-pip%20install%20llama
Assuming you are using cuda 12.1 based on the logs
Use the following command
pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\include\crt/host_config.h(153): fatal error C1189: #error: -- unsupported Microsoft Visual Studio version! Only the versions between 2017 and 2022 (inclusive) are supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk. [C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeFiles\CMakeScratch\TryCompile-8swq4p\cmTC_d9b18.vcxproj]
I have visual studio 2022 also installed build tool but no luck, i am using following command
Full error log
`pip install llama-cpp-python --force-reinstall --no-cache-dir Collecting llama-cpp-python Downloading llama_cpp_python-0.2.82.tar.gz (50.7 MB) ---------------------------------------- 50.7/50.7 MB 3.8 MB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing metadata (pyproject.toml) ... done Collecting typing-extensions>=4.5.0 (from llama-cpp-python) Downloading typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB) Collecting numpy>=1.20.0 (from llama-cpp-python) Downloading numpy-2.0.0-cp311-cp311-win_amd64.whl.metadata (60 kB) ---------------------------------------- 60.9/60.9 kB ? eta 0:00:00 Collecting diskcache>=5.6.1 (from llama-cpp-python) Downloading diskcache-5.6.3-py3-none-any.whl.metadata (20 kB) Collecting jinja2>=2.11.3 (from llama-cpp-python) Downloading jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB) Collecting MarkupSafe>=2.0 (from jinja2>=2.11.3->llama-cpp-python) Downloading MarkupSafe-2.1.5-cp311-cp311-win_amd64.whl.metadata (3.1 kB) Downloading diskcache-5.6.3-py3-none-any.whl (45 kB) ---------------------------------------- 45.5/45.5 kB ? eta 0:00:00 Downloading jinja2-3.1.4-py3-none-any.whl (133 kB) ---------------------------------------- 133.3/133.3 kB ? eta 0:00:00 Downloading numpy-2.0.0-cp311-cp311-win_amd64.whl (16.5 MB) ---------------------------------------- 16.5/16.5 MB 3.4 MB/s eta 0:00:00 Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB) Downloading MarkupSafe-2.1.5-cp311-cp311-win_amd64.whl (17 kB) Building wheels for collected packages: llama-cpp-python Building wheel for llama-cpp-python (pyproject.toml) ... error error: subprocess-exited-with-error
× Building wheel for llama-cpp-python (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [109 lines of output] scikit-build-core 0.9.8 using CMake 3.30.0 (wheel) Configuring CMake... 2024-07-10 20:33:04,443 - scikit_build_core - WARNING - Can't find a Python library, got libdir=None, ldlibrary=None, multiarch=None, masd=None loading initial cache file C:\Users\pc1\AppData\Local\Temp\tmp6zdbj_1l\build\CMakeInit.txt -- Building for: Visual Studio 17 2022 -- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.19045. -- The C compiler identification is MSVC 19.40.33812.0 -- The CXX compiler identification is MSVC 19.40.33812.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.40.33807/bin/Hostx64/x64/cl.exe - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.40.33807/bin/Hostx64/x64/cl.exe - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.45.2.windows.1") -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - not found -- Found Threads: TRUE -- Found OpenMP_C: -openmp (found version "2.0") -- Found OpenMP_CXX: -openmp (found version "2.0") -- Found OpenMP: TRUE (found version "2.0") -- OpenMP found -- Using ggml SGEMM -- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.8/include (found version "11.8.89") -- CUDA found -- Using CUDA architectures: 52;61;70;75 -- The CUDA compiler identification is NVIDIA 11.8.89 -- Detecting CUDA compiler ABI info -- Detecting CUDA compiler ABI info - failed -- Check for working CUDA compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.8/bin/nvcc.exe -- Check for working CUDA compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.8/bin/nvcc.exe - broken CMake Error at C:/Users/pc1/AppData/Local/Temp/pip-build-env-vh06lqz3/normal/Lib/site-packages/cmake/data/share/cmake-3.30/Modules/CMakeTestCUDACompiler.cmake:59 (message): The CUDA compiler
note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for llama-cpp-python Failed to build llama-cpp-python ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects`