search

mdrokz / rust-llama.cpp

LLama.cpp rust bindings
https://crates.io/crates/llama_cpp_rs/
MIT License
290 stars 42 forks source link

Include ggml-metal.metal file in source code #32

Closed pixelspark closed 5 months ago

pixelspark commented 5 months ago

To use Metal, the ggml-metal.metal file needs to be placed in the current directory. In the llm crate we have a little hack that patches ggml-metal.m to include the contents of that file directly in the source code, which is more convenient. See https://github.com/rustformers/llm/blob/9376078c12ea1990bd42e63432656819a056d379/crates/ggml/sys/build.rs#L198

The same hack can be applied here too. I can make a PR if this is deemed a good idea...

mdrokz commented 5 months ago

To use Metal, the ggml-metal.metal file needs to be placed in the current directory. In the llm crate we have a little hack that patches ggml-metal.m to include the contents of that file directly in the source code, which is more convenient. See https://github.com/rustformers/llm/blob/9376078c12ea1990bd42e63432656819a056d379/crates/ggml/sys/build.rs#L198

The same hack can be applied here too. I can make a PR if this is deemed a good idea...

This sounds good to me please go ahead and make an PR thanks.