LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.

https://github.com/lostruins/koboldcpp

GNU Affero General Public License v3.0

4.66k stars 334 forks source link

Feature request: Connect to horde as worker #39

Closed rondlite closed 1 year ago

rondlite commented 1 year ago

Expected Behavior

using API-key and be able to turn on share with horde

Current Behavior

option not there

would love to be able to use as worker so koboldcpp becomes multi-user

LostRuins commented 1 year ago

There are no horde features directing in KoboldCpp.

If you want to use models on the horde, you can use https://lite.koboldai.net

If you want to share it with the horde, you'll have to use the Horde Bridge here: https://github.com/db0/KoboldAI-Horde-Bridge

rondlite commented 1 year ago

I will have a look if it is feasable to integrate the horde bridge into hordecpp myself. currently running a private horde with united branch so got place to test easy

rondlite commented 1 year ago

Turns out that the horde bridge repo works with koboldcpp out of the box