search

nextcloud / assistant

✨ Nextcloud Assistant
GNU Affero General Public License v3.0
23 stars 4 forks source link

Use a local LLM proxy to aggregate AI models #77

Open ga-it opened 3 months ago

ga-it commented 3 months ago

Describe the feature you'd like to request

Allow administrator to set up access to multiple LLMs in nextcloud via proxy

Describe the solution you'd like

LITELLM does an incredible job of this.

Combination in Nextcloud would allow more options than the current OpenAI, LocalAI and Replicate options but would also provide a single AI abstraction layer and allow integration with Nextcloud user management, etc

LiteLLM's model aliases provide a useful way to have and call one proxy with unlimited Apis beneath.

The integration could allow integration with Nextcloud group permissions and tasks, allowing other apps to call the various AI models via alias and permissions through the abstraction layer.

If the aliases are exposed as bots then they could be called in Talk, etc

Describe alternatives you've considered

Currently calling LiteLLM via the Nextcloud LocalAI integration

ga-it commented 3 months ago

This probably belongs in the https://github.com/nextcloud/integration_openai project

Opened enhancement request here - https://github.com/nextcloud/integration_openai/issues/93