search

mdrokz / rust-llama.cpp

LLama.cpp rust bindings
https://crates.io/crates/llama_cpp_rs/
MIT License
290 stars 42 forks source link

Fix CUDA on Windows #41

Closed glcraft closed 2 months ago

glcraft commented 2 months ago

24 - Fix CUDA native build on Windows, so you don't have to rely on Docker

Minor fixes some common or linux specific code applied in the meantime.

glcraft commented 2 months ago

Hold on, llama.cpp doesn't seem to enable the GPU when I load and use the model when I import it in a project.

glcraft commented 2 months ago

Ok it now works fine on my custom project 👌