henk717
commented
1 year ago This is entirely on Humi (Developer of TavernAI). There is nothing tavern does that would not work with the horde API and he is in our discord so he can ask multiple contributors for help if he needs anything.
On our side there is nothing to be done to make Tavern work, we also won't be implementing the ability for our own local API to be bridged to a horde model because we do not want a loop where people allows to host horde models using horde models.
So I will close this request on our side knowing your request on the TavernAI side is all that is needed.
If you want an alternative in the meantime give https://koboldai.org/pygmalion a try which is our own attempt. Right now I haven't announced it far and wide since we only had 3 workers doing pygmalion so making a big announcement would flood the horde. But it is already done and has some really cool features like being able to share your bot as a link, running in the browser, also being able to connect to your own KoboldAI when horde is busy and even image generation based on the chat or story.
As for prioritizing your own requests this is automatically done when you use the same API key on the client as you used for your worker.
V1nm4n
Popping in a request to ask for: 1) Ability to connect to KoboldAI via TavernAI even while running a local node for use on the Kobold Horde.
The point of the ask is to convey that the information available from by the Kobold API should allow for Tavern users to prioritize their own traffic/requests over the requests of others when connecting to their own Kobold node (if not already done).
2) KoboldAI should help address any reasonable roadblocks that exist (if any) that prevent the TavernAI dev(s) for implementing the aforementioned.
This would incentivize all Kobold users that can to become nodes, as: A) if it doesn't impact them, why wouldn't they do it, and B) If a user is not currently using Kobold Horde for processing their requests (but rather collab/kaggle/etc), it would increase the number of folks who want to use the node, incentivizing participation on both sides.