mattermost / mattermost-plugin-ai

Mattermost Copilot plugin supporting multiple LLMs

https://mattermost.com/copilot

Apache License 2.0

136 stars 30 forks source link

Feature Idea: Support multiple AI models #69

Closed a3957273 closed 5 months ago

a3957273 commented 1 year ago

Summary

We run multiple models that are fine tuned for different tasks, as well as supporting multiple different model sizes. At the moment, we have to pick 'one' for our entire Mattermost instance. It would be useful to support multiple, and be able to distinguish between which one we use.

How important this is to me and why

Importance: Medium - This is currently the largest blocker to increasing our usage of the AI plugin.

Use cases:

Support multiple models with different specialties
Support multiple models with differing sizes
Experiment with new models

Additional context/similar features

Preferably, we would like to specify which model in the 'tag' we use. For instance:

@ai-llama2-7b ...
@ai-gpt-4 ...
@ai-xyz ...

At the moment we have recompiled multiple versions of the plugin with different names and configuration, but this is not scaling well as more and more models are being introduced.

esethna commented 1 year ago

Thanks for sharing this @a3957273! This is certainly something we're considering.

At the moment we have recompiled multiple versions of the plugin with different names and configuration, but this is not scaling well as more and more models are being introduced

Do you mind expanding on some of the challenges you're facing with scaling in this way?

a3957273 commented 1 year ago

Hey, apologies for the delay. This account is not regularly checked. When running fewer than half a dozen models, it was okay to spend ~30 minutes changing the plugin identifier, updating the selector, altering all the unique information, updating our internal plugin store and creating the config for it. This is impractical for running dozens or hundreds of models, especially when we want to run multiple versions of the same model.

a3957273 commented 1 year ago

Now with v0.4 and the custom display type, it's become increasingly difficult to support multiple bots. We can no longer run multiple of the same plugins or we run into issues with the frontend plugin.

Is there any route at the moment to running multiple LLMs on one instance?

a3957273 commented 1 year ago

At the moment we've solved this by allocating one "primary" plugin. This plugin is setup to be the only one to add UI functionality. Then, every other plugin is a "secondary" plugin which only provides a bot to chat with.

esethna commented 9 months ago

@a3957273 apologies for the late response. We're working on a design to support multiple bots tied to different LLMs in Enterprise, would love to get your feedback on it if you have a chance: https://www.loom.com/share/9e7cbcc3d36a4c93ac0ce818c1439dd4

Configuration of multiple bots tied to different LLMs:

UI selector to target conversations with different bots:

UI selector to target summaries and other UI interactions with different bots:

Happy to setup a call to chat through it more detail.

Can I also ask what Mattermost server version you are running?

a3957273 commented 9 months ago

This looks fantastic and precisely what we'd be looking for! I've tried out the multiple bot feature that's available in the latest commits and it seems rather a good way of specifying them. I enjoy the added features here such as allowing the selection of custom profile pictures, as well as the simplicity of switching between them.

etiennellipse commented 9 months ago

+1 on this, the proposed UI looks very useful to experiment with various models. In the per bot configuration, it would be ideal to be able to control who can use it.

esethna commented 8 months ago

Thanks for the feedback!

@etiennellipse when you say "In the per bot configuration, it would be ideal to be able to control who can use it." can you expand on the specific controls you're looking for and why they are needed?

For example, is it a list of individuals who have access to each bot, or is it based on teams, channels, custom user groups etc?

etiennellipse commented 8 months ago

@esethna Ideally, we would be able to select a user group. I guess using a private channel would be equivalent? Using a Team like currently can do it, but as I understand it using Teams comes with other constraints that might affect the deployment.

grandtele commented 8 months ago

@esethna this UI available in current repo?

esethna commented 8 months ago

@grandtele not yet. The designs I shared above are concepts we are validating before we build. Please share any feedback you have on them!

crspeller commented 5 months ago

175 Is our implementation of the suggestions in this issue.

It is included in the 0.8.0 release on the repo: https://github.com/mattermost/mattermost-plugin-ai/releases/tag/v0.8.0

@a3957273 and everyone else we would love to hear feedback from everyone here if the solution works for them and how we can improve.