search

withcatai / node-llama-cpp

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level
https://node-llama-cpp.withcat.ai
MIT License
877 stars 84 forks source link

Receiving error when compiling for cuda #262

Closed AliAzlanAziz closed 3 months ago

AliAzlanAziz commented 3 months ago

PS > npx --no node-llama-cpp download --cuda Repo: ggerganov/llama.cpp Release: b3197 CUDA: enabled

✔ Removed existing llama.cpp directory Cloning llama.cpp Clone ggerganov/llama.cpp (local bundle) 100% ████████████████████████████████████████ 0s ◷ Compiling llama.cpp Not searching for unused variables given on the command line. -- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.22631. -- The C compiler identification is MSVC 19.39.33321.0 -- The CXX compiler identification is MSVC 19.39.33321.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Preview/VC/Tools/MSVC/14.39.33321/bin/Hostx64/x64/cl.exe - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Preview/VC/Tools/MSVC/14.39.33321/bin/Hostx64/x64/cl.exe - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.43.0.windows.1") -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - not found -- Found Threads: TRUE -- Found OpenMP_C: -openmp (found version "2.0") -- Found OpenMP_CXX: -openmp (found version "2.0") -- Found OpenMP: TRUE (found version "2.0") -- OpenMP found CMake Warning at llama.cpp/CMakeLists.txt:409 (message): LLAMA_CUBLAS is deprecated and will be removed in the future.

Use LLAMA_CUDA instead

-- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1/include (found version "12.1.66") -- CUDA found -- Using CUDA architectures: 52;61;70 CMake Error at C:/Program Files/CMake/share/cmake-3.30/Modules/CMakeDetermineCompilerId.cmake:587 (message): No CUDA toolset found. Call Stack (most recent call first): C:/Program Files/CMake/share/cmake-3.30/Modules/CMakeDetermineCompilerId.cmake:8 (CMAKE_DETERMINE_COMPILER_ID_BUILD) C:/Program Files/CMake/share/cmake-3.30/Modules/CMakeDetermineCompilerId.cmake:53 (__determine_compiler_id_test) C:/Program Files/CMake/share/cmake-3.30/Modules/CMakeDetermineCUDACompiler.cmake:131 (CMAKE_DETERMINE_COMPILER_ID) llama.cpp/CMakeLists.txt:434 (enable_language)

-- Configuring incomplete, errors occurred! Not searching for unused variables given on the command line. -- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.22631. -- The C compiler identification is MSVC 19.39.33321.0 -- The CXX compiler identification is MSVC 19.39.33321.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Preview/VC/Tools/MSVC/14.39.33321/bin/Hostx64/x64/cl.exe - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Preview/VC/Tools/MSVC/14.39.33321/bin/Hostx64/x64/cl.exe - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.43.0.windows.1") -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - not found -- Found Threads: TRUE -- Found OpenMP_C: -openmp (found version "2.0") -- Found OpenMP_CXX: -openmp (found version "2.0") -- Found OpenMP: TRUE (found version "2.0") -- OpenMP found CMake Warning at llama.cpp/CMakeLists.txt:409 (message): LLAMA_CUBLAS is deprecated and will be removed in the future.

Use LLAMA_CUDA instead

-- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1/include (found version "12.1.66") -- CUDA found -- Using CUDA architectures: 52;61;70 CMake Error at C:/Program Files/CMake/share/cmake-3.30/Modules/CMakeDetermineCompilerId.cmake:587 (message): No CUDA toolset found. Call Stack (most recent call first): C:/Program Files/CMake/share/cmake-3.30/Modules/CMakeDetermineCompilerId.cmake:8 (CMAKE_DETERMINE_COMPILER_ID_BUILD) C:/Program Files/CMake/share/cmake-3.30/Modules/CMakeDetermineCompilerId.cmake:53 (__determine_compiler_id_test) C:/Program Files/CMake/share/cmake-3.30/Modules/CMakeDetermineCUDACompiler.cmake:131 (CMAKE_DETERMINE_COMPILER_ID) llama.cpp/CMakeLists.txt:434 (enable_language)

-- Configuring incomplete, errors occurred! ERR! OMG Process terminated: 1

[node-llama-cpp] To resolve errors related to CUDA compilation, see the CUDA guide: https://withcatai.github.io/node-llama-cpp/guide/CUDA ✖ Failed to compile llama.cpp node-llama-cpp download

Download a release of llama.cpp and compile it

Options: -h, --help Show help [boolean] --repo The GitHub repository to download a release of llama.cpp from. Can also be set via the NODE_LLAMA_CPP_REPO environment variable [string] [default: "ggerganov/llama.cpp"] --release The tag of the llama.cpp release to download. Set to "latest" to download t he latest release. Can also be set via the NODE_LLAMA_CPP_REPO_RELEASE envi ronment variable [string] [default: "b3197"] -a, --arch The architecture to compile llama.cpp for [string] -t, --nodeTarget The Node.js version to compile llama.cpp for. Example: v18.0.0 [string] --cuda Compile llama.cpp with CUDA support. Can also be set via the NODE_LLAMA_CPP _CUDA environment variable [boolean] [default: false] --skipBuild, --sb Skip building llama.cpp after downloading it [boolean] [default: false] --noBundle, --nb Download a llama.cpp release only from GitHub, even if a local git bundle e xists for the release [boolean] [default: false] -v, --version Show version number [boolean]

Error: Command npm run -s cmake-js-llama -- compile --log-level warn --arch=x64 --runtime-version=20.11.0 --CDLLAMA_METAL=OFF --CDLLAMA_CUBLAS=1 exited with code 1
    at ChildProcess.<anonymous> (file:///C:/Users/AliAzlanAziz/Desktop/study/learning/js-projects/llama-play-record/node_modules/node-llama-cpp/dist/utils/spawnCommand.js:27:24)
    at ChildProcess.emit (node:events:518:28)
    at cp.emit (C:\Users\AliAzlanAziz\Desktop\study\learning\js-projects\llama-play-record\node_modules\cross-spawn\lib\enoent.js:34:29)
    at ChildProcess._handle.onexit (node:internal/child_process:294:12)

System Specs: Windows 11, NVIDIA GeForce RTX 4060 8GB, Cuda version 12.1

AliAzlanAziz commented 3 months ago

I compiled it by downloading the current beta version of node-llama-cpp i.e. "node-llama-cpp": "^3.0.0-beta.36" and then downloaded older release by running this command: npx --no node-llama-cpp download --release "b1350" to build with cuda.

giladgd commented 3 months ago

I can't really help you without the necessary information requested when you open a ticket using one of the available templates. If the issue still persists, please open a new ticket using one of the available templates.