aws / aws-sam-cli

CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM
https://aws.amazon.com/serverless/sam/
Apache License 2.0
6.5k stars 1.17k forks source link

sam local start-api does not work for nested stack when supplied with template path #4129

Open EinarasGar opened 2 years ago

EinarasGar commented 2 years ago

Description:

When starting API for nested stack and providing the template file, the API does not work. In this case I am using dotnet AWS template https://github.com/aws/aws-lambda-dotnet/tree/master/Blueprints/BlueprintDefinitions/vs2022/AnnotationsFramework/template/src/BlueprintBaseName.1

Steps to reproduce:

dotnet new serverless.Annotations -n name Enter the source folder of the function sam build -t serverless.template sam local start-api -t serverless.template

Observed result:

The API does not work when performing request to one of the endpoints

2022-08-18 15:43:30,778 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2022-08-18 15:43:30,778 | Using config file: samconfig.toml, config environment: default
2022-08-18 15:43:30,778 | Expand command line arguments to:
2022-08-18 15:43:30,778 | --template_file=/workspaces/Database modeling/AddTransaction/src/AddTransaction/serverless.template --host=127.0.0.1 --port=3000 --static_dir=public --layer_cache_basedir=/home/vscode/.aws-sam/layers-pkg --container_host=localhost --container_host_interface=127.0.0.1 
2022-08-18 15:43:30,860 | local start-api command is called
2022-08-18 15:43:30,867 | No Parameters detected in the template
2022-08-18 15:43:30,890 | There is no customer defined id or cdk path defined for resource AddTransactionFunctionsDefaultGenerated, so we will use the resource logical id as the resource id
2022-08-18 15:43:30,891 | There is no customer defined id or cdk path defined for resource AddTransactionFunctionsAddGenerated, so we will use the resource logical id as the resource id
2022-08-18 15:43:30,891 | There is no customer defined id or cdk path defined for resource AddTransactionFunctionsSubstractGenerated, so we will use the resource logical id as the resource id
2022-08-18 15:43:30,891 | There is no customer defined id or cdk path defined for resource AddTransactionFunctionsMultiplyGenerated, so we will use the resource logical id as the resource id
2022-08-18 15:43:30,891 | There is no customer defined id or cdk path defined for resource AddTransactionFunctionsDivideGenerated, so we will use the resource logical id as the resource id
2022-08-18 15:43:30,891 | There is no customer defined id or cdk path defined for resource ServerlessHttpApi, so we will use the resource logical id as the resource id
2022-08-18 15:43:30,893 | 0 stacks found in the template
2022-08-18 15:43:30,894 | No Parameters detected in the template
2022-08-18 15:43:30,915 | There is no customer defined id or cdk path defined for resource AddTransactionFunctionsDefaultGenerated, so we will use the resource logical id as the resource id
2022-08-18 15:43:30,915 | There is no customer defined id or cdk path defined for resource AddTransactionFunctionsAddGenerated, so we will use the resource logical id as the resource id
2022-08-18 15:43:30,915 | There is no customer defined id or cdk path defined for resource AddTransactionFunctionsSubstractGenerated, so we will use the resource logical id as the resource id
2022-08-18 15:43:30,915 | There is no customer defined id or cdk path defined for resource AddTransactionFunctionsMultiplyGenerated, so we will use the resource logical id as the resource id
2022-08-18 15:43:30,915 | There is no customer defined id or cdk path defined for resource AddTransactionFunctionsDivideGenerated, so we will use the resource logical id as the resource id
2022-08-18 15:43:30,916 | There is no customer defined id or cdk path defined for resource ServerlessHttpApi, so we will use the resource logical id as the resource id
2022-08-18 15:43:30,917 | 6 resources found in the stack 
2022-08-18 15:43:30,917 | Found Serverless function with name='AddTransactionFunctionsDefaultGenerated' and CodeUri='.'
2022-08-18 15:43:30,917 | --base-dir is not presented, adjusting uri . relative to /workspaces/Database modeling/AddTransaction/src/AddTransaction/serverless.template
2022-08-18 15:43:30,918 | Found Serverless function with name='AddTransactionFunctionsAddGenerated' and CodeUri='.'
2022-08-18 15:43:30,919 | --base-dir is not presented, adjusting uri . relative to /workspaces/Database modeling/AddTransaction/src/AddTransaction/serverless.template
2022-08-18 15:43:30,920 | Found Serverless function with name='AddTransactionFunctionsSubstractGenerated' and CodeUri='.'
2022-08-18 15:43:30,920 | --base-dir is not presented, adjusting uri . relative to /workspaces/Database modeling/AddTransaction/src/AddTransaction/serverless.template
2022-08-18 15:43:30,922 | Found Serverless function with name='AddTransactionFunctionsMultiplyGenerated' and CodeUri='.'
2022-08-18 15:43:30,922 | --base-dir is not presented, adjusting uri . relative to /workspaces/Database modeling/AddTransaction/src/AddTransaction/serverless.template
2022-08-18 15:43:30,924 | Found Serverless function with name='AddTransactionFunctionsDivideGenerated' and CodeUri='.'
2022-08-18 15:43:30,924 | --base-dir is not presented, adjusting uri . relative to /workspaces/Database modeling/AddTransaction/src/AddTransaction/serverless.template
2022-08-18 15:43:30,931 | Found '1' API Events in Serverless function with name 'AddTransactionFunctionsDefaultGenerated'
2022-08-18 15:43:30,932 | Found '1' API Events in Serverless function with name 'AddTransactionFunctionsAddGenerated'
2022-08-18 15:43:30,932 | Found '1' API Events in Serverless function with name 'AddTransactionFunctionsSubstractGenerated'
2022-08-18 15:43:30,932 | Found '1' API Events in Serverless function with name 'AddTransactionFunctionsMultiplyGenerated'
2022-08-18 15:43:30,932 | Found '1' API Events in Serverless function with name 'AddTransactionFunctionsDivideGenerated'
2022-08-18 15:43:30,932 | Detected Inline Swagger definition
2022-08-18 15:43:30,932 | Lambda function integration not found in Swagger document at path='/add/{x}/{y}' method='get'
2022-08-18 15:43:30,932 | Lambda function integration not found in Swagger document at path='/divide/{x}/{y}' method='get'
2022-08-18 15:43:30,932 | Lambda function integration not found in Swagger document at path='/substract/{x}/{y}' method='get'
2022-08-18 15:43:30,932 | Lambda function integration not found in Swagger document at path='/' method='get'
2022-08-18 15:43:30,932 | Lambda function integration not found in Swagger document at path='/multiply/{x}/{y}' method='get'
2022-08-18 15:43:30,932 | Found '0' APIs in resource 'ServerlessHttpApi'
2022-08-18 15:43:30,933 | Removed duplicates from '0' Explicit APIs and '5' Implicit APIs to produce '5' APIs
2022-08-18 15:43:30,933 | 5 APIs found in the template
2022-08-18 15:43:30,945 | Mounting AddTransactionFunctionsSubstractGenerated at http://127.0.0.1:3000/substract/{x}/{y} [GET]
2022-08-18 15:43:30,945 | Mounting AddTransactionFunctionsMultiplyGenerated at http://127.0.0.1:3000/multiply/{x}/{y} [GET]
2022-08-18 15:43:30,945 | Mounting AddTransactionFunctionsDefaultGenerated at http://127.0.0.1:3000/ [GET]
2022-08-18 15:43:30,945 | Mounting AddTransactionFunctionsAddGenerated at http://127.0.0.1:3000/add/{x}/{y} [GET]
2022-08-18 15:43:30,945 | Mounting AddTransactionFunctionsDivideGenerated at http://127.0.0.1:3000/divide/{x}/{y} [GET]
2022-08-18 15:43:30,945 | You can now browse to the above endpoints to invoke your functions. You do not need to restart/reload SAM CLI while working on your functions, changes will be reflected instantly/automatically. You only need to restart SAM CLI if you update your AWS SAM template
2022-08-18 15:43:30,945 | Localhost server is starting up. Multi-threading = True
2022-08-18 15:43:30  * Running on http://127.0.0.1:3000/ (Press CTRL+C to quit)
2022-08-18 15:43:36,831 | Constructed String representation of Event Version 2.0 to invoke Lambda. Event: {"version": "2.0", "routeKey": "GET /", "rawPath": "/", "rawQueryString": "", "cookies": [], "headers": {"Host": "127.0.0.1:3000", "Connection": "keep-alive", "Sec-Ch-Ua": "\"Chromium\";v=\"104\", \" Not A;Brand\";v=\"99\", \"Microsoft Edge\";v=\"104\"", "Sec-Ch-Ua-Mobile": "?0", "Sec-Ch-Ua-Platform": "\"Windows\"", "Upgrade-Insecure-Requests": "1", "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.81 Safari/537.36 Edg/104.0.1293.54", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9", "Sec-Fetch-Site": "none", "Sec-Fetch-Mode": "navigate", "Sec-Fetch-User": "?1", "Sec-Fetch-Dest": "document", "Accept-Encoding": "gzip, deflate, br", "Accept-Language": "en-GB,en;q=0.9,en-US;q=0.8", "X-Forwarded-Proto": "http", "X-Forwarded-Port": "3000"}, "requestContext": {"accountId": "123456789012", "apiId": "1234567890", "http": {"method": "GET", "path": "/", "protocol": "HTTP/1.1", "sourceIp": "127.0.0.1", "userAgent": "Custom User Agent String"}, "requestId": "f5eb497b-d980-463c-89b6-3a0a6535dedf", "routeKey": "GET /", "stage": "$default", "time": "18/Aug/2022:15:43:30 +0000", "timeEpoch": 1660837410, "domainName": "localhost", "domainPrefix": "localhost"}, "body": "", "pathParameters": {}, "stageVariables": null, "isBase64Encoded": false}
2022-08-18 15:43:36,832 | Found one Lambda function with name 'AddTransactionFunctionsDefaultGenerated'
2022-08-18 15:43:36,832 | Invoking AddTransaction::AddTransaction.Functions_Default_Generated::Default (dotnet6)
2022-08-18 15:43:36,832 | No environment variables found for function 'AddTransactionFunctionsDefaultGenerated'
2022-08-18 15:43:36,832 | Environment variables overrides data is standard format
2022-08-18 15:43:36,832 | Loading AWS credentials from session with profile 'None'
2022-08-18 15:43:36,854 | Resolving code path. Cwd=/workspaces/Database modeling/AddTransaction/src/AddTransaction, CodeUri=/workspaces/Database modeling/AddTransaction/src/AddTransaction
2022-08-18 15:43:36,854 | Resolved absolute path to code is /workspaces/Database modeling/AddTransaction/src/AddTransaction
2022-08-18 15:43:36,855 | Code /workspaces/Database modeling/AddTransaction/src/AddTransaction is not a zip/jar file
2022-08-18 15:43:36,907 | Skip pulling image and use local one: public.ecr.aws/sam/emulation-dotnet6:rapid-1.53.0-x86_64.

2022-08-18 15:43:36,907 | Mounting /workspaces/Database modeling/AddTransaction/src/AddTransaction as /var/task:ro,delegated inside runtime container
2022-08-18 15:43:42,065 | Starting a timer for 30 seconds for function 'AddTransactionFunctionsDefaultGenerated'
START RequestId: 77b24c3c-5223-4629-8922-071947cf1c59 Version: $LATEST
Error: .NET binaries for Lambda function are not correctly installed in the /var/task directory of the image when the image was built. The /var/task directory is missing the required .deps.json file.
18 Aug 2022 15:43:42,454 [ERROR] (rapid) Init failed error=Runtime exited with error: exit status 105 InvokeID=
Error: .NET binaries for Lambda function are not correctly installed in the /var/task directory of the image when the image was built. The /var/task directory is missing the required .deps.json file.
END RequestId: 1a079aad-a1c6-4af8-ad0c-d7889a568bff
REPORT RequestId: 1a079aad-a1c6-4af8-ad0c-d7889a568bff  Init Duration: 0.18 ms  Duration: 1119.09 ms    Billed Duration: 1120 ms        Memory Size: 256 MB     Max Memory Used: 256 MB
2022-08-18 15:43:45,511 | Cleaning all decompressed code dirs
2022-08-18 15:43:45,512 | Unable to find Click Context for getting session_id.
2022-08-18 15:43:45,512 | Invalid lambda response received: Lambda response must be valid json
2022-08-18 15:43:45 127.0.0.1 - - [18/Aug/2022 15:43:45] "GET / HTTP/1.1" 502 -
2022-08-18 15:43:46 127.0.0.1 - - [18/Aug/2022 15:43:46] "GET /favicon.ico HTTP/1.1" 403 -

Expected result:

The API should work

Workaround

The API DOES work when running commands sam build -t serverless.template sam local start-api (note that in second command template path is not supplied, but everything works as it should)

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

SAM CLI, version 1.53.0 OS: Linux - docker dotnet 6 development container

jfuss commented 2 years ago

@EinarasGar Thanks for the issue, however this is expected. When you provide --template, you are telling the command to use that template. When using build, a template is generated within ./aws-sam/build which points to all the built artifacts. So the workaround is actually is the right path here.

Leaving open for now in case there are follow ups.

EinarasGar commented 2 years ago

Thank you for a quick reply @jfuss . Apologies for invalid issue. However, I am confused. In what case would supplying template path would be a correct action? Maybe this should be documented better? Thank you

jfuss commented 2 years ago

@EinarasGar No need to apologize. We would much rather the community cut an issue than worry about it being "invalid".

In what case would supplying template path would be a correct action? Maybe this should be documented better?

I thought we added something in this area, I know this is a gotcha that has gotten customers before. I will check this deeper to see what we can do here. Suggestions are welcome, either in docs or in the CLI output. Knowing where you would look and where/what you would expect is always helpful info.

On the "when to supply" part: If you name the template in the "auto discoverable" name (template.yaml, template.yml, or template.json), you should never need to supply the template. If you have a custom template name, and use sam build, only providing -t to sam build is all you need. If you aren't using sam build, then all commands would be needed. I feel there is some case with package and deploy but can't specifically recall.

I am going to assign this to myself, as a reminder to come back once I do a deeper dive with the docs. So would just ask to leave this open, until we have clear messaging on these cases.

EinarasGar commented 2 years ago

@jfuss, I looked around a bit more and you are right that there is documentation about it which does state that the --template parameter is to be used within the build directory. It does make sense.

That being said, it is counter intuitive for a new user learning about the toolkit. The build command expects a template of a project The start-api command expects template of build artifacts I would expect that in case that the user has custom template, the -t argument shall be used with all commands involving a template. The last thing I would expect is that template path should be where the build directories are.

The sam cli help command also does state that it references build artifacts

AWS SAM template which references built
artifacts for resources in the template. (if
applicable)  [default:
template.[yaml|yml|json]]

but again, as for a new user learning about toolkit, I was not aware that build command creates a seperate template and places it in build artifacts folder.

That being said, I think the best course of action would be to check if the build artifacts are located in the same folder as the template. For instance, if the template expects functions AddTransactionFunctionsDefaultGenerated, AddTransactionFunctionsAddGenerated .... , then check if cwd of template contains folders AddTransactionFunctionsDefaultGenerated, AddTransactionFunctionsAddGenerated. If the folders are not found, the start-api command should print a warning message that there are missing files and state that the provided template should be in the same directory as other build files.

Right now the start-api command just tries to mount folders based on the current working dir, and if the folders are not there, it still tries to run programs resulting in errors.

So to reiterate, the documentation does say correct things, its just not very intuitive how to use -t argument for a new user.