search

withcatai / node-llama-cpp

Run AI models locally on your machine with node.js bindings for llama.cpp. Force a JSON schema on the model output on the generation level
https://node-llama-cpp.withcat.ai
MIT License
829 stars 80 forks source link

No CUDA toolset found Error even its Found CUDAToolkit #122

Closed sooraj007 closed 8 months ago

sooraj007 commented 8 months ago

Issue description

when try to build for cuda build is failing its showing cublas and cudatoolkit found but still it failed showing No Cuda toolset found, i have added all the necessory envirnment variables but stil failing

Expected Behavior

I am using cuda toolkit version 12.2 and visual studio 2022 community edition , when i try to build it supposed to be build but it shows error

Expecting to build

Actual Behavior

Its now showing error like this using this command npx --no node-llama-cpp download --cuda env variables : NODE_LLAMA_CPP_CMAKE_OPTION_CMAKE_GENERATOR_TOOLSET : v143 this is one of the env variable that is i thin kproblamatic , above one is using for visual studio , its only working when i give like above but dont know if the same using when cmake build

-- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - not found -- Found Threads: TRUE -- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.2/include (found version "12.2.91") -- cuBLAS found CMake Error at xpack/store/@xpack-dev-tools/cmake/3.26.5-1.1/.content/share/cmake-3.26/Modules/CMakeDetermineCompilerId.cmake:501 (message): No CUDA toolset found. Call Stack (most recent call first): xpack/store/@xpack-dev-tools/cmake/3.26.5-1.1/.content/share/cmake-3.26/Modules/CMakeDetermineCompilerId.cmake:8 (CMAKE_DETERMINE_COMPILER_ID_BUILD)

Steps to reproduce

just run npx --no node-llama-cpp download --cuda then its hapening

My Environment

Dependency Version
Operating System windows
CPU Intel i9 -13900k
Node.js version x.y.zzz
Typescript version x.y.zzz
node-llama-cpp version 3.0.0-beta.1

Additional Context

No response

Relevant Features Used

Are you willing to resolve this issue by submitting a Pull Request?

Yes, I have the time, and I know how to start.

awijshoff commented 8 months ago

Hi, I had a similar problem.

Following the steps of copying the four files mentioned in this GitHub issue, worked for me.

copy all the four files from C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\extras\visual_studio_integration\MSBuildExtensions, and paste them to C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Microsoft\VC\v160\BuildCustomizations.

sooraj007 commented 8 months ago

@awijshoff

Thanks bro, that worked out great, it would be great if author should mention these steps also in the documentation, i was searching it for 2 days. Again thanks

giladgd commented 8 months ago

@sooraj007 For the issues that cannot be solved by node-llama-cpp automatically, I've added a section of issues and solutions in CUDA documentation; it would be great if you could add there the details of the issue you faced and its solution.

You can create a PR by clicking on the "Edit this page" button at the bottom of the page.