search

Rubiksman78 / MonikA.I

Submod for MAS with AI based features
MIT License
112 stars 17 forks source link

Online support #89

Open Mike77154 opened 10 months ago

Mike77154 commented 10 months ago

Is there a way to connect DDLC monika ai into a online webui like huggingface page or google colab one? instead of local use?

Rubiksman78 commented 10 months ago

Huggingface isn't a Webui but indeed hosts a lot of AI models, there is the possibility for testing models in spaces but I don't believe they are made to be used in production. As for Colab many chatbots were banned and there is the GPU limit that can't be sustainable in the long term.

Generally speaking, I didn't choose to implement a connection with a WebUI like previously with CAI because of the unreliability of those and the difficulty to self-maintain it.

However, everything is now connected with text-generation-webui which runs locally with options to decrease the memory usage on GPU or CPU so that it can be used on less performant hardware.