🐛 bug: failure with anthropic claude engine when multiple user messages in a row

[sblaisot]

Description

in a conversation with @ai assistant using Anthropic claude-3-haiku-20240307 when there is multiple messages in a row from user, the next request to @ai fails with message

{"timestamp":"2024-05-13 08:30:06.853 Z","level":"error","msg":"Streaming result to post failed","caller":"app/plugin_api.go:1003","plugin_id":"mattermost-ai","error":"non 200 response from anthropic: 400 Bad Request\nBody:\n{\"type\":\"error\",\"error\":{\"type\":\"invalid_request_error\",\"message\":\"messages: roles must alternate between \\\"user\\\" and \\\"assistant\\\", but found multiple \\\"user\\\" roles in a row\"}}"}

Steps to reproduce

Start a conversation with @ai bot using anthropic claude engine and add 2 messages in a row

sblaisot [10:44]
@ai say hello

ai BOT [10:44]

Hello there! How can I assist you today?

sblaisot [10:44]
very nice

sblaisot [10:44]
@ai who are you?

ai BOT [10:44]
Sorry! An error occurred while accessing the LLM. See server logs for details.

[sblaisot]

when removing the very nice message and avoiding two user messages in a row @ai work fine again in the conversation

[sblaisot]

Please note I don't have this problem with OpenAI engine so this is specific to Anthropic Claude engine

[ynott]

That same problem is happening to me too.

Mattermost Server Logs

error [2024-07-02 16:07:05.050 +09:00] Streaming result to post failed               caller="app/plugin_api.go:1007" plugin_id=mattermost-ai error="non 200 response from anthropic: 400 Bad Request
Body:
{"type":"error","error":{"type":"invalid_request_error","message":"messages: roles must alternate between \"user\" and \"assistant\", but found multiple \"user\" roles in a row"}}"
error [2024-07-02 16:07:30.845 +09:00] Streaming result to post failed               caller="app/plugin_api.go:1007" plugin_id=mattermost-ai error="non 200 response from anthropic: 400 Bad Request
Body:
{"type":"error","error":{"type":"invalid_request_error","message":"messages: roles must alternate between \"user\" and \"assistant\", but found multiple \"user\" roles in a row"}}"

We are connecting to the Anthropic API from Copilot.

[NicksonYap]

indeed Anthropic API requires alternating the messages

maybe the plugin needs to combine if multiple messages are sent by Users but overall it seems like the plugin is designed more for OpenAI

and that OpenAI is friendly to multiple users and messages before a message from AI