aws-samples / bedrock-access-gateway

OpenAI-Compatible RESTful APIs for Amazon Bedrock
MIT No Attribution
124 stars 24 forks source link

feat: Handle multiple user messages in a single request #26

Open ysekiy opened 1 week ago

ysekiy commented 1 week ago

Issue #25 : Error when including multiple user messages in a single request to /chat/completions API

Description of changes:

This pull request adds a new method _reframe_multi_payloard to handle multiple user messages in a single request to the /chat/completions API. The method converts the OpenAI format messages to the Bedrock conversational API format by aggregating all user messages into a single message with an array of contents.

The method performs the following steps:

  1. Aggregates all user messages and assistant messages separately into two lists.
  2. Creates a new output message list.
  3. Adds the first user message with the aggregated user contents to the output list.
  4. Adds the first assistant message with the aggregated assistant contents to the output list.

This change allows the application to handle multiple user messages in a single request, which was previously causing an error.

daixba commented 5 days ago

The PR tried to merge all the user messages and assistant messages into one. I am afraid that changes the order of the messages (in chat history) which may have impacts to the AI response.

A user message can only be merged with the previous one if the previous one is with the same role.

ysekiy commented 3 days ago

@daixba Thank you for your comment.

I refactored the implementation to not change the order.

Changed to accept OpenAI-compliant input