search

ahyatt / llm

A package abstracting llm capabilities for emacs.
GNU General Public License v3.0
178 stars 24 forks source link

[feature] Support for the llamacpp server? #8

Closed draxil closed 11 months ago

draxil commented 11 months ago

Unless this has been discarded already for some reason, support for the llama.cpp server (https://github.com/ggerganov/llama.cpp/tree/master/examples/server) ?

There is already an emacs client:

https://github.com/kurnevsky/llama-cpp.el

I know this is an "example" so that might be a reason to reject this idea, but I was interested to at least ask!

ahyatt commented 11 months ago

Certainly possible, I'll look into it!

ahyatt commented 11 months ago

I've added a provider for llama.cpp in the latest commit - let me know if it works for you. I'll make a release with it soon (I'd prefer to wait until you test, but I'll probably release it either way over the weekend).

draxil commented 10 months ago

Oh fantastic I'll give it a shot!