The state of AWS Lambda Go Streaming support

jmnarloch commented 4 months ago

Is your feature request related to a problem? Please describe. I had been trying make the AWS Lambda response streaming work with aws-lambda-go runtime and I was not able to succeed. Neither the Lambda Function URLs or direct API call using InvokeWithResponseStream work and instead I was retrieving the entire response serialized to JSON.

I made my Lambda function return the events.LambdaFunctionURLStreamingResponse, but the entire response had been serialized and returned as JSON. The passed io.Reader had not been proceesed and suprisingly enough the response contain JSON serialized representation of the io.Reader implementation i.e.

{"StatusCode":200,"Headers":{"Content-Type":"application/octet-stream"},"Body":{"ReadCloser":{}},"Cookies":null}

I had been runing the GO Lambda function on al2023 runtime, compiled the binary with -tags lambda.norpc option and set up the Function URL.

FunctionUrl:
    Type: AWS::Lambda::Url
    Properties:
      TargetFunctionArn: !Ref Function
      AuthType: NONE
      InvokeMode: RESPONSE_STREAM

Neither of this had any impact and the behavior persisted. Even the documented in code example is not working correctly:

// Example:
//
//  lambda.Start(func() (*events.LambdaFunctionURLStreamingResponse, error) {
//      return &events.LambdaFunctionURLStreamingResponse{
//          StatusCode: 200,
//          Headers: map[string]string{
//              "Content-Type": "text/html",
//          },
//          Body: strings.NewReader("<html><body>Hello World!</body></html>"),
//      }, nil
//  })
//
// Note: This response type requires compiling with `-tags lambda.norpc`, or choosing the `provided` or `provided.al2` runtime.

Which results in response from Function URL:

{"StatusCode":200,"Headers":{"Content-Type":"text/html"},"Body":{},"Cookies":null}

Is Response Streaming not supported with Go runtime?

Describe the solution you'd like I would like to use Lambda Response Streaming with my Go runtime.

Describe alternatives you've considered I am not using AWS Lambda for this use case today.

Additional context github.com/aws/aws-lambda-go v1.47.0

bmoffatt commented 4 months ago

Streaming URLs is supported! I'm sorry to see that you're having issues!

The passed io.Reader had not been proceesed and suprisingly enough the response contain JSON serialized representation of the io.Reader implementation i.e. {"StatusCode":200,"Headers":{"Content-Type":"application/octet-stream"},"Body":{"ReadCloser":{}},"Cookies":null}

Can you provide a code sample that produces this response? Also your Go version and version of aws-lambda-go from your go.mod

Neither of this had any impact and the behavior persisted. Even the documented in code example is not working correctly

{"StatusCode":200,"Headers":{"Content-Type":"text/html"},"Body":{},"Cookies":null}

The example code works for me when I tested today.

My main.go:

package main

import (
        "strings"

        "github.com/aws/aws-lambda-go/events"
        "github.com/aws/aws-lambda-go/lambda"
)

func main() {
        lambda.Start(func() (*events.LambdaFunctionURLStreamingResponse, error) {
                return &events.LambdaFunctionURLStreamingResponse{
                        StatusCode: 200,
                        Headers: map[string]string{
                                "Content-Type": "text/html",
                        },
                        Body: strings.NewReader("<html><body>Hello World!</body></html>"),
                }, nil
        })
}

My template.yaml:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
  StreamingFunction:
    Type: AWS::Serverless::Function
    Metadata:
      BuildMethod: go1.x
    Properties:
      CodeUri: .
      Handler: .
      Runtime: provided.al2023
      Architectures: [ arm64 ]
      Timeout: 10
      FunctionUrlConfig:
        AuthType: AWS_IAM
        InvokeMode: RESPONSE_STREAM
Outputs:
  StreamingFunctionURL:
    Description: "Streaming Lambda Function URL"
    Value: !GetAtt StreamingFunctionUrl.FunctionUrl

Deployed and tested sam build && sam deploy --region us-west-2 --profile "$USER".

jmnarloch commented 4 months ago

Thank you for the comprehensive response. I realized that the main reason why I run into this problem is that I had use BuildMethod: makefile once I change it into BuildMethod: go1.x and remove the Makefile from the project the response has been process fully and was not serialized as JSON.

"{\"statusCode\":200,\"headers\":{\"Content-Type\":\"text/html\"}}\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000<html><body>Hello World!</body></html>"

I guess my question is, is there still possible to make this work with a custom Makefile?

The makefile I had been trying to use

build-Function:
    GOOS=linux GOARCH=arm64 go build -o bootstrap -tags lambda.norpc .
    cp ./bootstrap $(ARTIFACTS_DIR)/.

aws / aws-lambda-go

The state of AWS Lambda Go Streaming support #565