search

Rubiksman78 / MonikA.I

Submod for MAS with AI based features
MIT License
112 stars 17 forks source link

Allow KoboldAI backend support #86

Open wereretot opened 9 months ago

wereretot commented 9 months ago

Describe the solution you'd like I want an option to be able to use the KoboldAI API to allow a way of local use.

Describe alternatives you've considered The problems with cloud hosted solutions like c.ai are that they can change and also be taken away at the whim of the company, but with local hosting this would no longer be a worry. It also allows a more reliable and faster method of using MonikA.I if the user's hardware is able to handle the model they have loaded on the KoboldAI backend.

Additional context You can find the KoboldAI GitHub repository here: https://github.com/KoboldAI/KoboldAI-Client If you need assistant with the API or have any questions, you may contact the developers of KAI on the official KoboldAI Discord server here: https://discord.gg/rrsAaPPRYa

Rubiksman78 commented 9 months ago

An update is ongoing using Oobabooga text-generation-webui as a backend that will allow the same kind of features with an easier installation. I believe it will be sufficient for current needs and offers as much stability.