search

aws / aws-sam-cli

CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM
https://aws.amazon.com/serverless/sam/
Apache License 2.0
6.5k stars 1.17k forks source link

Bug: Inconsistent Behavior of Streaming Response in AWS SAM CLI #6501

Open dyingsunlight opened 9 months ago

dyingsunlight commented 9 months ago

Description:

AWS Lambda now supports streaming responses, but it appears that the AWS SAM CLI does not. When I try to develop my application locally, it doesn't work at all.

Steps to reproduce:

Create a SAM project with the following template.json:

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Transform": "AWS::Serverless-2016-10-31",
  "Description": "",
  "Resources": {
    "HelloWorld": {
      "Type": "AWS::Serverless::Function",
      "Properties": {
        "CodeUri": "./lambda",
        "Handler": "hello-word.handler",
        "Runtime": "nodejs18.x",
        "Events": {
          "HelloWorldEvent0": {
            "Type": "Api",
            "Properties": {
              "Path": "/hello-world",
              "Method": "get"
            }
          }
        },
        "FunctionUrlConfig": {
          "AuthType": "NONE",
          "MODE": "RESPONSE_STREAM"
        },
        "Timeout": 60,
        "MemorySize": 1024
      }
    }
  }
}

Create index.mjs with the following content:

import * as awslambda from 'aws-lambda';

export const handler = awslambda.streamifyResponse(async (event, responseStream, _context) => {
responseStream = awslambda.HttpResponseStream.from(responseStream, {
statusCode: 200
});
responseStream.write("Streaming with Helper \n");
responseStream.end();
// 'responseStream.finished' does not exist in the local development environment either.
// await responseStream.finished();
});

Run the following commands:

sam local start-api --template template.json --port 8090

Observed result:

After the AWS SAM CLI is ready, visiting http://127.0.0.1:8090/hello-world yields:

Console outputs:

Invalid lambda response received: Lambda response must be valid JSON

The lambda invocation just throws the error and does nothing.

Expected result:

If I upload the code to AWS Cloud, everything works well. I would like the AWS SAM CLI to have consistent behavior between development and production environments.

2024-01-01 18:34:49,059 | Constructed Event 1.0 to invoke Lambda. Event: {'version': '1.0', 'httpMethod': 'GET', 'body': None, 'resource': '/hello-world', 'requestContext':  
{'resourceId': '123456', 'apiId': '1234567890', 'resourcePath': '/hello-world', 'httpMethod': 'GET', 'requestId': '50f39c31-b40e-407e-9251-8d93cfba79d0', 'accountId':        
'123456789012', 'stage': 'Prod', 'identity': {'apiKey': None, 'userArn': None, 'cognitoAuthenticationType': None, 'caller': None, 'userAgent': 'Custom User Agent String',    
'user': None, 'cognitoIdentityPoolId': None, 'cognitoAuthenticationProvider': None, 'sourceIp': '127.0.0.1', 'accountId': None}, 'extendedRequestId': None, 'path':           
'/hello-world', 'protocol': 'HTTP/1.1', 'domainName': '127.0.0.1:8090', 'requestTimeEpoch': 1704105259, 'requestTime': '01/Jan/2024:10:34:19 +0000'}, 'queryStringParameters':
None, 'multiValueQueryStringParameters': None, 'headers': {'Host': '127.0.0.1:8090', 'Connection': 'keep-alive', 'Cache-Control': 'max-age=0', 'Sec-Ch-Ua': '"Not_A           
Brand";v="8", "Chromium";v="120", "Google Chrome";v="120"', 'Sec-Ch-Ua-Mobile': '?0', 'Sec-Ch-Ua-Platform': '"macOS"', 'Upgrade-Insecure-Requests': '1', 'User-Agent':        
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36', 'Accept':                                            
'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7', 'Sec-Fetch-Site': 'none',          
'Sec-Fetch-Mode': 'navigate', 'Sec-Fetch-User': '?1', 'Sec-Fetch-Dest': 'document', 'Accept-Encoding': 'gzip, deflate, br', 'Accept-Language':                                
'zh-CN,zh;q=0.9,zh-TW;q=0.8,en;q=0.7,ja;q=0.6,ru;q=0.5', 'X-Forwarded-Proto': 'http', 'X-Forwarded-Port': '8090'}, 'multiValueHeaders': {'Host': ['127.0.0.1:8090'],          
'Connection': ['keep-alive'], 'Cache-Control': ['max-age=0'], 'Sec-Ch-Ua': ['"Not_A Brand";v="8", "Chromium";v="120", "Google Chrome";v="120"'], 'Sec-Ch-Ua-Mobile': ['?0'],  
'Sec-Ch-Ua-Platform': ['"macOS"'], 'Upgrade-Insecure-Requests': ['1'], 'User-Agent': ['Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/120.0.0.0 Safari/537.36'], 'Accept':                                                                                                                                   
['text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7'], 'Sec-Fetch-Site': ['none'],      
'Sec-Fetch-Mode': ['navigate'], 'Sec-Fetch-User': ['?1'], 'Sec-Fetch-Dest': ['document'], 'Accept-Encoding': ['gzip, deflate, br'], 'Accept-Language':                        
['zh-CN,zh;q=0.9,zh-TW;q=0.8,en;q=0.7,ja;q=0.6,ru;q=0.5'], 'X-Forwarded-Proto': ['http'], 'X-Forwarded-Port': ['8090']}, 'pathParameters': None, 'stageVariables': None,      
'path': '/hello-world', 'isBase64Encoded': False}                                                                                                                             
2024-01-01 18:34:49,069 | Found one Lambda function with name 'HelloWorld'                                                                                                    
2024-01-01 18:34:49,072 | Invoking hello-word.handler (nodejs18.x)                                                                                                            
2024-01-01 18:34:49,073 | No environment variables found for function 'HelloWorld'                                                                                            
2024-01-01 18:34:49,073 | Resolving code path. Cwd=/Users/tenghou/aws-sam-bug-streaming, CodeUri=/Users/tenghou/aws-sam-bug-streaming/lambda                      
2024-01-01 18:34:49,074 | Resolved absolute path to code is /Users/tenghou/aws-sam-bug-streaming/lambda                                                                 
2024-01-01 18:34:49,075 | Code /Users/tenghou/aws-sam-bug-streaming/lambda is not a zip/jar file                                                                        
2024-01-01 18:34:49,094 | Checking free port on 127.0.0.1:5588                                                                                                                
2024-01-01 18:34:49,100 | Using local image: public.ecr.aws/lambda/nodejs:18-rapid-x86_64.

2024-01-01 18:34:49,101 | Mounting /Users/tenghou/aws-sam-bug-streaming/lambda as /var/task:ro,delegated, inside runtime container                                      
2024-01-01 18:34:49,388 | Starting a timer for 60 seconds for function 'HelloWorld'                                                                                           
START RequestId: 1f115a13-138b-43f1-b340-0ff328817e90 Version: $LATEST
END RequestId: 474c6068-753c-4d85-bf5f-5af7f1891de1
REPORT RequestId: 474c6068-753c-4d85-bf5f-5af7f1891de1  Init Duration: 0.38 ms  Duration: 899.27 ms     Billed Duration: 900 ms Memory Size: 1024 MB    Max Memory Used: 1024 MB      
2024-01-01 18:34:50,299 | Failed to deserialize response from RIE, returning the raw response as is

2024-01-01 18:34:50,407 | Cleaning all decompressed code dirs                                                                                                                 
2024-01-01 18:34:50,408 | Invalid lambda response received: Lambda response must be valid json                                                                                
2024-01-01 18:34:50,408 | Lambda execution failed () 
sam --info
{
  "version": "1.105.0",
  "system": {
    "python": "3.8.13",
    "os": "macOS-13.4.1-arm64-arm-64bit"
  },
  "additional_dependencies": {
    "docker_engine": "24.0.2",
    "aws_cdk": "2.115.0 (build 58027ee)",
    "terraform": "1.6.5"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}
mildaniel commented 9 months ago

Thanks for raising this issue. At the moment, sam local commands do not support response streaming and this is a known feature gap.

In the meantime, I recommend trying to test your application with sam sync which will test in the cloud instead of using local emulation.

atonamy commented 7 months ago

The issue still didn't fix? I face exactly the same problem sam sync is not an option for me.

Erol-klart commented 2 months ago

I'm not able to use sam sync either is there any update on this issue ?