Add support for Lambda Response Streaming

[meiamsome]

AWS just announced Lambda Response Streaming which allows for a Lambda to provide a streamed multipart response instead of a one-shot response, as well as increasing response payload size limits.

This should allow for implementation of @defer and @stream with a lambda which is currently not possible.

[BlenderDude]

I was excited to see this announcement too! But sadly, from the AWS blog post:

Neither API Gateway nor Lambda’s target integration with Application Load Balancer support chunked transfer encoding. It therefore does not support faster TTFB for streamed responses. You can, however, use response streaming with API Gateway to return larger payload responses, up to API Gateway’s 10 MB limit. To implement this, you must configure an HTTP_PROXY integration between your API Gateway and a Lambda function URL, instead of using the LAMBDA_PROXY integration.

Given that both the API Gateway and ALB implementation both buffer the response before returning it to the client; effectively making the benefits of @defer useless. The only implementations that support streaming are the Function URL and a direct invocation.

It is ill-advised to have a function URL directly called from the client as per AWS. There is no DDOS protection, no WAF, no rate limiting, etc., giving any malicious actor a URL to siphon money out of your pocket 😉.

I think this issue should be re-visited and implemented when either the buffering at API gateway is removed, or Function URLs can be locked down better (IAM secured access from CloudFront, rate limiting, etc.). CloudFront proxying to a Function URL is technically possible right now, but still requires the url to be "OPEN" as opposed to "IAM" authenticated. There is a feature request for CloudFront IAM signing, so we can add that to the list of things to wait for!