💡 idea: Take into account the user locale for AI actions

mattermost / mattermost-plugin-ai

Mattermost Copilot plugin supporting multiple LLMs

https://mattermost.com/copilot

Apache License 2.0

136 stars 30 forks source link

💡 idea: Take into account the user locale for AI actions #167

Closed muratbayan closed 5 months ago

muratbayan commented 7 months ago

Description

Users having a non-english locale in their Mattermost profile profile would greatly benefit from having the answers in their chosen language.

AI Copilot responds in the language of the user input when using the RHS to chat, however action buttons (Summarize Thread, Open Questions, Actions Items etc.) in the UI will by default return a response in English from my testing.

These actions' answers being only available to the user triggering them, I would love to see them match the user locale, which is available to the AI Copilot already (checked by asking "who am I" to get my locale)

A valuable use case that I use often is conversations with our customers across multiple regions. Multiple languages are spoken in threads as there is often a language barrier. AI Copilot helps me get a summary in English of threads. Follow ups with AI copilot also help me draft responses in the appropriate language.

jprusch commented 7 months ago

That would lead to translatable plugins. +1 from my side.

jespino commented 5 months ago

@muratbayan I think this is relatively easy to implement, but I see some problems. Maybe the user doesn't want the reply in its own language, maybe he prefer to keep the context in the original language, second, we expect that in the future you can post the summary of something back to the channel, so... if it is in your local language you need to ask the AI to translate it back to the original language. I'm not sure if we would like this to be optional or not.

jespino commented 5 months ago

@esethna do you have any strong opinions about the previous comment?

jprusch commented 5 months ago

I see some areas of work here:

Copilot UI language should be made translatable (Weblate?)
What should the initial language be, that the AI uses to answer? +1 to derive it from the user locale. Changeable by asking the AI to answer in another language.
Some models are mainly trained in English (or another specific language). Using another language might not make sense in these models. Maybe a language specific prompt can help here or give the user a hint on what's supported.
On translations, I'm 1/5: This shouldn't be done automatically by the AI unless asked for. Something like "Summarize the chat in Spanish and post it to channel XYZ"

jespino commented 5 months ago

Thanks @jprusch, I have already some initial work on this.

The Copilot UI, as you said, should be translatable, but that wasn't the focus of my changes here. Still, It is an important thing to add.
What I've done is that if you interact directly with the AI by sending it messages, it is going to answer you in the language that you asked, that is consistent.
This is very interesting point that I wasn't considering yet. But you are right, maybe with a bit of "prompt engineering" is enough, something like "please answer me in my language if you know how, if not answer me in English", or something on those lines. But in the worst case scenario we would need to explicitly define that the model is or not multilingual in the configuration to be able to properly behave here.
If you ask the AI for doing something in the specific language that you want, it should behave as expected, I'm not forcing the locale in direct user interactions.

esethna commented 5 months ago

Agree with not forcing the locale in a response. The end user always has the option to contextually interrogate to translate any responses into a specific language. Agree that when interacting in a different language the response should follow the language of the prompt

jespino commented 5 months ago

I added 2 PRs here, one for the locale aware AI, making the AI aware of your locale and asking for the preference of answering in your own language unless you ask something directly to the AI, in that case the AI will answer in the language that you asked. The second PR is adding localization to the interface, making the interface translatable, and in fact I added the Spanish translation for that :)

muratbayan commented 5 months ago

Thanks a lot, forcing the locale is something we should steer away from I agree. In terms of user experience, would there be a use case where the AI might want to stick to the input language of the thread/call transcription when summarising? e.g. I have a Mattermost Call recording with French speakers, or a thread with only German replies?

Another thought: would it make sense to allow some persistent bits of context based on our conversations with Copilot? i.e. can I tell Copilot that I only want it to reply to me in French from now on for all my future direct chats with it?