search

sobelio / llm-chain

`llm-chain` is a powerful rust crate for building chains in large language models allowing you to summarise text and complete complex tasks
https://llm-chain.xyz
MIT License
1.27k stars 127 forks source link

Pre-compiled llama.cpp #263

Open CMorrison82z opened 5 months ago

CMorrison82z commented 5 months ago

I'm using an AMD graphics card and had to compile llama.cpp with additional flags and parameters.

From what I can see, llm-chain-llama attempts to compile the llama.cpp module itself, which I imagine is unlikely to produce a suitable result for me.

Is there any way I can use this library with a pre-compiled llama.cpp binary ?

williamhogman commented 5 months ago

If I recall correctly llama.cpp don't provide ready made dynamic libraries to link with so in this case it is hard for us to offer anything along those lines

Any ideas?