aws-amplify / amplify-backend

Home to all tools related to Amplify's code-first DX (Gen 2) for building fullstack apps on AWS
Apache License 2.0
184 stars 62 forks source link

Use message history instead of event payload for conversation handler #2047

Closed sobolk closed 1 month ago

sobolk commented 1 month ago

Problem

Conversation handler receives messages in the event. The problem is that event size is constrained to 256 KB for asynchronous invocations. https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html .

When conversation is long enough and/or contains images this limit is hit and lambda can't be called.

Previous logic as defined in upstream resolver can be seen here. Note that it's using outdated gql schema.

Changes

  1. Retrieve message history by:
    1. Querying Appsync by conversationId and currentMessageId to recreate history.
    2. Reconciling the history to pair user-asistant messages and drop unanswered questions.
    3. Inject aiContext.
  2. Keep event.messages for now to be backwards compatible until upstream components absorb this change. This will be removed later.
  3. Tests.
  4. Extract GraphqlRequestExecutor in the ai-constructs since we've hit the rule of 3 and refactor other components that execute GQL queries to use it.

Validation

Added tests, including e2e tests.

Manual tests via snapshot release with upstream components.

Checklist

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

changeset-bot[bot] commented 1 month ago

🦋 Changeset detected

Latest commit: a220ceeea3d49fb33ba4f3ce0d216b2940af7fc4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages | Name | Type | | -------------------------- | ----- | | @aws-amplify/ai-constructs | Minor | | @aws-amplify/backend-ai | Patch |

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR