ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

[bossrhino]

OS: Ubuntu 23.10 GPU: Nvidia 3070

Nvidia NVCC version 12.0 Build: cuda_12.0.r12.0/compiler.32267302_0

nvidia-smi is working.

gcc version: 13.2.0 g++ version: 13.2.0

References:

1. https://www.youtube.com/watch?v=rwqPA4qi0H8
2. https://docs.privategpt.dev/installation

Command to replicate problem:

• CMAKE_ARGS='-DLLAMA_CUBLAS=on' poetry run pip install --force-reinstall --no-cache-dir llama-cpp-python

Note: In the installation doc I'm on the 'Linux NVIDIA GPU support and Windows-WSL' section.

---

I'm working on my linux desktop to install private gpt. Followed most of the tutorials on the docs page + novaspirits youtube video. I am running this in a python virtual environment, so I'm not sure if that has something to do with this error. But this is the error I'm stuck on:

Building wheels for collected packages: llama-cpp-python
  Building wheel for llama-cpp-python (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [90 lines of output]
      *** scikit-build-core 0.7.1 using CMake 3.27.4 (wheel)
      *** Configuring CMake...
      loading initial cache file /tmp/tmpolouxedf/build/CMakeInit.txt
      -- The C compiler identification is GNU 13.2.0
      -- The CXX compiler identification is GNU 13.2.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Found Git: /usr/bin/git (found version "2.40.1")
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
      -- Found Threads: TRUE
      -- Found CUDAToolkit: /usr/include (found version "12.0.140")
      -- cuBLAS found
      CMake Error at /usr/share/cmake-3.27/Modules/CMakeDetermineCompilerId.cmake:753 (message):
        Compiling the CUDA compiler identification source file
        "CMakeCUDACompilerId.cu" failed.

        Compiler: /usr/bin/nvcc

        Build flags:

        Id flags: --keep;--keep-dir;tmp -v

        The output was:

        1

        #$ _NVVM_BRANCH_=nvvm

        #$ _SPACE_=

        #$ _CUDART_=cudart

        #$ _HERE_=/usr/lib/nvidia-cuda-toolkit/bin

        #$ _THERE_=/usr/lib/nvidia-cuda-toolkit/bin

        #$ _TARGET_SIZE_=

        #$ _TARGET_DIR_=

        #$ _TARGET_SIZE_=64

        #$ NVVMIR_LIBRARY_DIR=/usr/lib/nvidia-cuda-toolkit/libdevice

        #$
        PATH=/usr/lib/nvidia-cuda-toolkit/bin:/tmp/pip-build-env-8taqq3j1/overlay/bin:/tmp/pip-build-env-8taqq3j1/normal/bin:/home/jefffredericks/ai/privateGPT/privategpt_ai_venv/bin:/home/jefffredericks/ai/privateGPT/privategpt_ai_venv/bin:/home/jefffredericks//.nix-profile/bin:/nix/var/nix/profiles/default/bin:/home/jefffredericks//.local/bin:/home/jefffredericks//.nix-profile/bin:/nix/var/nix/profiles/default/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin

        #$ LIBRARIES= -L/usr/lib/x86_64-linux-gnu/stubs -L/usr/lib/x86_64-linux-gnu

        #$ rm tmp/a_dlink.reg.c

        #$ gcc -D__CUDA_ARCH_LIST__=520 -E -x c++ -D__CUDACC__ -D__NVCC__
        -D__CUDACC_VER_MAJOR__=12 -D__CUDACC_VER_MINOR__=0
        -D__CUDACC_VER_BUILD__=140 -D__CUDA_API_VER_MAJOR__=12
        -D__CUDA_API_VER_MINOR__=0 -D__NVCC_DIAG_PRAGMA_SUPPORT__=1 -include
        "cuda_runtime.h" -m64 "CMakeCUDACompilerId.cu" -o
        "tmp/CMakeCUDACompilerId.cpp4.ii"

        ERROR: No supported gcc/g++ host compiler found, but clang-14 is available.

               Use 'nvcc -ccbin clang-14' to use that instead.

        # --error 0x1 --

      Call Stack (most recent call first):
        /usr/share/cmake-3.27/Modules/CMakeDetermineCompilerId.cmake:8 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
        /usr/share/cmake-3.27/Modules/CMakeDetermineCompilerId.cmake:53 (__determine_compiler_id_test)
        /usr/share/cmake-3.27/Modules/CMakeDetermineCUDACompiler.cmake:307 (CMAKE_DETERMINE_COMPILER_ID)
        vendor/llama.cpp/CMakeLists.txt:296 (enable_language)

      -- Configuring incomplete, errors occurred!

      *** CMake configuration failed
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

Appreciate any advice, i've not seen this issue in the issues thread yet.

[sumitsodhi88]

you need visual studios

[DiamondGlassDrill]

I have the same error and do have VS 2022 installed but still receive the error. I know in https://github.com/oobabooga/text-generation-webui it is solved programmatically with different system and decision trees. Maybe the coders of privateGPT can have a look into it? Thx in advance for supporting.

[bossrhino]

@sumitsodhi88 I have Visual Studios installed. Specifically Visual Studio Code. Unless I'm missing something, this is the only one available for Ubuntu.

[github-actions[bot]]

Stale issue

[bossrhino]

Stale because no one has responded

[rutgers-master]

For me, the solution was to include the "-ccbin clang-14' to nvcc like the error said, so doing the following on a Debian 12 system allowed me to compile llama-cpp-python with GPU support.

Just add NVCC_PREPEND_FLAGS='-ccbin clang-14' like so:

CMAKE_ARGS='-DLLAMA_CUBLAS=ON' NVCC_PREPEND_FLAGS='-ccbin clang-14' poetry run pip install --force-reinstall --no-cache-dir llama-cpp-python