Add raw incoming invocation payload as header from lambda_http library

[kueckermann]

We currently use the Lambda web adapter which makes use of lambda_http library to parse incoming requests. At the moment, this library only supports event payloads from a limited number of AWS sources. In our case we want to execute our application, which is running via the web adapter extension, directly from SQS. The reason for this is that we have some long running batch operations that need to be executed from SNS triggers. However, SNS has a hard cap on execution times. Seeing that these batches always overflow the execution time, we have had to send the batches to an SQS queue and pass them onto the private endpoint in our API. Creating a separate lambda to manage direct executions from SQS is out of scope for our small development team and a current solution is to create a separate lambda which formats and proxies the request onwards. This works, but is messy.

The request from SQS does come through to our application via the web adapter but it has a blank body, context, etc. From examining the code inside the adapter and the lambda_http library, it seems to be because lambda_http only knows how to parse the payload for a limited number of sources. I understand that parsing all AWS payload formats would be out of scope for this library, but rather what would be incredibly helpful would be if there was a reference stored somewhere to the original invocation payload.

At the moment, the request context and trace information are stored as headers. I'd like to recommend that the original payload be added as a header as well. If this is done, developers can utilise the web adapter to support invocations from any source by parsing the raw payload themselves from the header on the request and letting their own applications decide what to do with the invocation.

I'd recommend something along the lines of: "x-amzn-request-payload": "..." which contains the raw payload.

Note that before the web adapter extension was released, many developers used @vendia/serverless-express to run web applications inside of lambda. serverless-express would allow requests from any source by letting the developer configuring how to manage the requests from those sources, so this kind of functionality is definitely not out of scope for what developers would want out of web adapter.

I'm creating this issue here, since the work to be done seems to be on the lambda_http library not on the web adapter extension. I'm not a rust dev so I could be wrong, but that's what I've understood from the code. If you think this is relevant to the web adapter repo, I'm happy to move this issue there.

Thanks!

[bnusunny]

There is a similar issue in Lambda Web Adapter repo. The proposed solution was to use CloudEvent HTTP transport to pass the event payload to the web application. Does it meet your needs?

[calavera]

I don't think this should be part of the runtime itself, even if other projects support it. As @bnusunny mentions, there are other ways to accomplish what you need, and you could even build your own solution, as you mention. I'd like the runtime itself to remain as general as possible, and something like this breaks the general pattern.

I'm closing this issue in favor of the one in the Web Adapter, where it seems to have more context and traction. It sounds like CloudEvents would provide you the information without modifications in the runtime. Feel free to follow the conversation there.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for the maintainers of this repository to see. If you need more assistance, please open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.