GPU support dont works

[AIrabbit0]

I get this :

  error: subprocess-exited-with-error

  × Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [40 lines of output]
      *** scikit-build-core 0.7.1 using CMake 3.28.1 (wheel)
      *** Configuring CMake...
      2024-01-10 21:42:14,312 - scikit_build_core - WARNING - Can't find a Python library, got libdir=None, ldlibrary=None, multiarch=None, masd=None
      loading initial cache file C:\Users\PCG\AppData\Local\Temp\tmpcsx434fh\build\CMakeInit.txt
      -- Building for: Visual Studio 17 2022
      -- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.22631.
      -- The C compiler identification is MSVC 19.38.33134.0
      -- The CXX compiler identification is MSVC 19.38.33134.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.43.0.windows.1")
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
      -- Looking for pthread_create in pthreads
      -- Looking for pthread_create in pthreads - not found
      -- Looking for pthread_create in pthread
      -- Looking for pthread_create in pthread - not found
      -- Found Threads: TRUE
      -- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.3/include (found version "12.3.107")
      -- cuBLAS found
      CMake Error at C:/Users/PCG/AppData/Local/Temp/pip-build-env-m7ezkx80/normal/Lib/site-packages/cmake/data/share/cmake-3.28/Modules/CMakeDetermineCompilerId.cmake:529 (message):
        No CUDA toolset found.
      Call Stack (most recent call first):
        C:/Users/PCG/AppData/Local/Temp/pip-build-env-m7ezkx80/normal/Lib/site-packages/cmake/data/share/cmake-3.28/Modules/CMakeDetermineCompilerId.cmake:8 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
        C:/Users/PCG/AppData/Local/Temp/pip-build-env-m7ezkx80/normal/Lib/site-packages/cmake/data/share/cmake-3.28/Modules/CMakeDetermineCompilerId.cmake:53 (__determine_compiler_id_test)
        C:/Users/PCG/AppData/Local/Temp/pip-build-env-m7ezkx80/normal/Lib/site-packages/cmake/data/share/cmake-3.28/Modules/CMakeDetermineCUDACompiler.cmake:135 (CMAKE_DETERMINE_COMPILER_ID)
        vendor/llama.cpp/CMakeLists.txt:302 (enable_language)

when i did $env:CMAKE_ARGS='-DLLAMA_CUBLAS=on'; poetry run pip install --force-reinstall --no-cache-dir llama-cpp-python Also i run nvcc --version and nvidia-smi What can i do?

[NicolasKritter]

I had the same problem, turns out it's linked to the visual studio plugin. I found this link with the solution: https://github.com/NVlabs/tiny-cuda-nn/issues/164 Basically you have to move some file from your cuda install folder to the visual studio install folder

If you don't find the files, make sure you installed the visual studio component alongside cuda

[dadupriv]

I had the same problem, turns out it's linked to the visual studio plugin. I found this link with the solution: NVlabs/tiny-cuda-nn#164 Basically you have to move some file from your cuda install folder to the visual studio install folder

If you don't find the files, make sure you installed the visual studio component alongside cuda

https://github.com/NVlabs/tiny-cuda-nn/issues/164#issuecomment-1529171816