search

DeSinc / SallyBot

AI Chatbot coded in Discord.net C#
MIT License
296 stars 50 forks source link

Koboldcpp LLM interface supported? #29

Open Vladonai opened 1 year ago

Vladonai commented 1 year ago

This is a very popular method of interacting with models. Frequent updates and good performance. Can it be used with your program?

DeSinc commented 1 year ago

Not currently but I may look into it some time.

Sorry late reply because gmail sends github emails to spam

DeSinc commented 1 year ago

Actually, it occurs to me that the extension API which this was built for is deliberately compatible with the Kobold API, and it even runs on the same port number. Have you tried running sallybot with a kobold LLM running to see what happens? It might just already work right off the bat. Specifically, I think the format was made to be identical and cross-compatible with TavernAI or something like that, just from what I can remember of it, which uses KoboldAI to do its thing.

DeSinc commented 1 year ago

I tried using koboldcpp by sending the request on port 5001/5005 or whatever the default is, and it actually sends the request and generates an output, but there's a bit of an error when it comes time for koboldcpp to send back the output back to the SallyBot code. It might be easy to solve but I just haven't been bothered to give it a look.