aws / aws-sam-cli

CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM
https://aws.amazon.com/serverless/sam/
Apache License 2.0
6.52k stars 1.17k forks source link

Bug: Invoking a lambda function is giving me an error 'cannot find module' even though module exists #5415

Open kkchu791 opened 1 year ago

kkchu791 commented 1 year ago

After building and starting the local sam server, I'm noticing after I run my lambda GetBlocksByDateRangeFunction, I get this error in the console.

Invalid lambda response received: Invalid API Gateway Response Keys: {'trace', 'errorMessage', 'errorType'} in {'errorType': 'Runtime.ImportModuleError', 'errorMessage': "Error: Cannot find
module 'something'\nRequire stack:\n- /var/runtime/index.mjs", 'trace': ["Runtime.ImportModuleError: Error: Cannot find module 'something'", 'Require stack:', '- /var/runtime/index.mjs', '
at _loadUserApp (file:///var/runtime/index.mjs:997:17)', '    at async UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1032:21)', '    at async start
(file:///var/runtime/index.mjs:1195:23)', '    at async file:///var/runtime/index.mjs:1201:1']}

Steps to reproduce:

I'm not sure how I got this behavior to happen.

Observed result:

2023-06-25 11:58:45,415 | Constructed Event 1.0 to invoke Lambda. Event: {'version': '1.0', 'httpMethod': 'GET', 'body': None, 'resource': '/kevin', 'requestContext': {'resourceId': '123456',
'apiId': '1234567890', 'resourcePath': '/kevin', 'httpMethod': 'GET', 'requestId': '04e1e7ee-a920-462e-ac8d-3589db2ff535', 'accountId': '123456789012', 'stage': 'Prod', 'identity': {'apiKey':
None, 'userArn': None, 'cognitoAuthenticationType': None, 'caller': None, 'userAgent': 'Custom User Agent String', 'user': None, 'cognitoIdentityPoolId': None, 'cognitoAuthenticationProvider':
None, 'sourceIp': '127.0.0.1', 'accountId': None}, 'extendedRequestId': None, 'path': '/kevin', 'protocol': 'HTTP/1.1', 'domainName': '127.0.0.1:3001', 'requestTimeEpoch': 1687719488,
'requestTime': '25/Jun/2023:18:58:08 +0000'}, 'queryStringParameters': None, 'multiValueQueryStringParameters': None, 'headers': {'User-Agent': 'PostmanRuntime/7.32.3', 'Accept': '*/*',
'Postman-Token': '6f0a0ea9-75de-4839-a840-947f37c83019', 'Host': '127.0.0.1:3001', 'Accept-Encoding': 'gzip, deflate, br', 'Connection': 'keep-alive', 'X-Forwarded-Proto': 'http',
'X-Forwarded-Port': '3001'}, 'multiValueHeaders': {'User-Agent': ['PostmanRuntime/7.32.3'], 'Accept': ['*/*'], 'Postman-Token': ['6f0a0ea9-75de-4839-a840-947f37c83019'], 'Host':
['127.0.0.1:3001'], 'Accept-Encoding': ['gzip, deflate, br'], 'Connection': ['keep-alive'], 'X-Forwarded-Proto': ['http'], 'X-Forwarded-Port': ['3001']}, 'pathParameters': None,
'stageVariables': None, 'path': '/kevin', 'isBase64Encoded': False}
2023-06-25 11:58:45,419 | Found one Lambda function with name 'GetBlocksByDateRangeFunction'
2023-06-25 11:58:45,424 | Invoking src/handlers/something.somethingHandler (nodejs18.x)
2023-06-25 11:58:45,425 | No environment variables found for function 'GetBlocksByDateRangeFunction'
2023-06-25 11:58:45,426 | Resolving code path. Cwd=/Users/kirkchu/pairup/ourtracks-node-api/.aws-sam/build,
CodeUri=/Users/kirkchu/pairup/ourtracks-node-api/.aws-sam/build/GetBlocksByDateRangeFunction
2023-06-25 11:58:45,427 | Resolved absolute path to code is /Users/kirkchu/pairup/ourtracks-node-api/.aws-sam/build/GetBlocksByDateRangeFunction
2023-06-25 11:58:45,427 | Code /Users/kirkchu/pairup/ourtracks-node-api/.aws-sam/build/GetBlocksByDateRangeFunction is not a zip/jar file
2023-06-25 11:58:45,465 | Using local image: public.ecr.aws/lambda/nodejs:18-rapid-x86_64.

2023-06-25 11:58:45,466 | Mounting /Users/kirkchu/pairup/ourtracks-node-api/.aws-sam/build/GetBlocksByDateRangeFunction as /var/task:ro,delegated, inside runtime container
2023-06-25 11:58:46,369 | Starting a timer for 3 seconds for function 'GetBlocksByDateRangeFunction'
START RequestId: 56b06cbc-5a48-4570-886f-f9e589e8b906 Version: $LATEST
2023-06-25T18:58:46.658Z    undefined   ERROR   Uncaught Exception  {"errorType":"Runtime.ImportModuleError","errorMessage":"Error: Cannot find module 'something'\nRequire stack:\n- /var/runtime/index.mjs","stack":["Runtime.ImportModuleError: Error: Cannot find module 'something'","Require stack:","- /var/runtime/index.mjs","    at _loadUserApp (file:///var/runtime/index.mjs:997:17)","    at async UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1032:21)","    at async start (file:///var/runtime/index.mjs:1195:23)","    at async file:///var/runtime/index.mjs:1201:1"]}
25 Jun 2023 18:58:46,687 [ERROR] (rapid) Init failed error=Runtime exited with error: exit status 129 InvokeID=
2023-06-25T18:58:46.957Z    undefined   ERROR   Uncaught Exception  {"errorType":"Runtime.ImportModuleError","errorMessage":"Error: Cannot find module 'something'\nRequire stack:\n- /var/runtime/index.mjs","stack":["Runtime.ImportModuleError: Error: Cannot find module 'something'","Require stack:","- /var/runtime/index.mjs","    at _loadUserApp (file:///var/runtime/index.mjs:997:17)","    at async UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1032:21)","    at async start (file:///var/runtime/index.mjs:1195:23)","    at async file:///var/runtime/index.mjs:1201:1"]}
END RequestId: 191afcfa-7165-4e0d-a437-efa1c55904c9
REPORT RequestId: 191afcfa-7165-4e0d-a437-efa1c55904c9  Init Duration: 0.94 ms  Duration: 567.76 ms Billed Duration: 568 ms Memory Size: 128 MB Max Memory Used: 128 MB
2023-06-25 11:58:47,270 | Cleaning all decompressed code dirs
2023-06-25 11:58:47,272 | Lambda returned empty body!
2023-06-25 11:58:47,273 | Invalid lambda response received: Invalid API Gateway Response Keys: {'errorType', 'trace', 'errorMessage'} in {'errorType': 'Runtime.ImportModuleError',
'errorMessage': "Error: Cannot find module 'something'\nRequire stack:\n- /var/runtime/index.mjs", 'trace': ["Runtime.ImportModuleError: Error: Cannot find module 'something'", 'Require
stack:', '- /var/runtime/index.mjs', '    at _loadUserApp (file:///var/runtime/index.mjs:997:17)', '    at async UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1032:21)', '
at async start (file:///var/runtime/index.mjs:1195:23)', '    at async file:///var/runtime/index.mjs:1201:1']}

Expected result:

I'm expecting it to not return an error and not to say Cannot Find Module since the module does exist.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

Mac

  1. OS: MacOS Big Sur
  2. sam --version: SAM CLI, version 1.88.0
  3. AWS region: us-west-2
# Paste the output of `sam --info` here

{
  "version": "1.88.0",
  "system": {
    "python": "3.11.4",
    "os": "macOS-11.2.2-x86_64-i386-64bit"
  },
  "additional_dependencies": {
    "docker_engine": "20.10.23",
    "aws_cdk": "Not available",
    "terraform": "Not available"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}```

`Add --debug flag to command you are running`
kkchu791 commented 1 year ago

Approaches I've tried:

  1. Deleted all docker images and containers involving sam
  2. Change the name of the resource and it began to work. When I switched it back to the original name, I got the error.
  3. Updated Sam-Cli to the latest version
sriram-mv commented 1 year ago

Thanks for creating this issue, can you share your template and how your code is structured?

Change the name of the resource and it began to work. When I switched it back to the original name, I got the error.

This is super interesting, do you see any changes in the artifacts within .aws-sam/ directory once you change the name of the resource?

kkchu791 commented 1 year ago

template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Runtime: nodejs18.x
    Timeout: 180
    Architectures: x86_64
    Environment:
      Variables:
        DB_HOST: process.env.DB_HOST
        DB_USER: process.env.DB_USER
        DB_PASSWORD: process.env.DB_PASSWORD
        DB_DATABASE: process.env.DB_DATABASE
        DB_PORT: process.env.DB_PORT
  Api:
    Cors:
      AllowMethods: "'*'"
      AllowHeaders: "'*'"
      AllowOrigin: "'*'"

Resources:
  GetBlocksByDateRangeFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: src/handlers/blocks/get-blocks-by-date-range.getBlocksByDateRangeHandler
      Runtime: nodejs18.x
      Architectures:
        - x86_64
      Events:
        Api:
          Type: Api
          Properties:
            Path: /blocks
            Method: GET

  GetOpenBlocksByDateRangeFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: ./src/handlers/blocks/get-open-blocks-by-date-range.getOpenBlocksByDateRangeHandler
      Architectures:
        - x86_64
      Events:
        Api:
          Type: HttpApi
          Properties:
            Path: /blocks/open
            Method: GET

  CreateBlockFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: src/handlers/blocks/create-block.createBlockHandler
      Architectures:
        - x86_64
      Events:
        Api:
          Type: HttpApi
          Properties:
            Path: /blocks
            Method: POST

  # UpdateBlockFunction:
  #   Type: AWS::Serverless::Function
  #   Properties:
  #     Handler: src/handlers/blocks/update-block.updateBlockHandler
  #     Architectures:
  #       - x86_64
  #     Events:
  #       Api:
  #         Type: HttpApi
  #         Properties:
  #           Path: /blocks/{id}
  #           Method: PUT

Outputs:
  WebEndpoint:
    Description: API Gateway endpoint URL for Prod stage
    Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"
kkchu791 commented 1 year ago
Screen Shot 2023-06-28 at 6 29 23 PM
kkchu791 commented 1 year ago

I don't see to many changes in the artifacts for .aws-sam besides the function name changing to the new one I changed it to. See in screenshots that the function name is now getBlockssByDateRangeFunction instead of getBlocksByDateRangeFunction.

Screen Shot 2023-06-28 at 6 32 17 PM Screen Shot 2023-06-28 at 6 34 33 PM
dineshnagumothu commented 1 year ago

Facing the same issue on M1 Mac. Started initially well, then suddenly got the error with no code changes.

Changed the name of the resource and starts working again.

cunneen commented 1 year ago

I also have this problem (also an M1 Mac), but unlike @dineshnagumothu , renaming the resource didn't work for me.

I strongly suspect the issue is with the lambda runtime harness within the docker image; The error I get is :

node:internal/modules/cjs/loader:1031
  throw err;
  ^

Error: Cannot find module '/var/runtime/index.js'
    at Function._resolveFilename (node:internal/modules/cjs/loader:1028:15)
    at Function._load (node:internal/modules/cjs/loader:873:27)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:22:47 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}
22 Sep 2023 08:49:50,052 [ERROR] (rapid) Init failed error=Runtime exited with error: exit status 1 InvokeID=

But if I open a terminal session to the Docker container (which is left running), the /var/runtime/ folder contains an index.mjs file but not an index.js file.

The docker image seems to be this one:

public.ecr.aws/sam/emulation-nodejs16.x:rapid-1.50.0-x86_64

cunneen commented 1 year ago

In my case above, I discovered that I had an outdated version of the SAM CLI that was installed via homebrew, in addition to the one I'd installed via the MacOS package installer.

After uninstalling both, and installing the latest (v1.97.0), debugging my lambda function works nicely for me.

ldu2 commented 1 year ago

I’m running into a similar issue on my Mac, OS Ventura 13.6. It happens to all three hello-world ts apps, nodejs14, 16 and 18.

One interesting thing is that after starting the app (any of the three), the first hit to the /hello endpoint gives the classic Error: Cannot find module ‘app’\nRequire stack:\n-/var/runtime/index.mjs”,”stack…. Then the consecutive hits give a different error: Invalid lambda response received: Lambda response must be a valid json.

My sam cli is v1.97.0. I installed with the brew initially, same version, same error. Uninstall that and installed through pkg file. Same error.

This error didn’t happened to me on the most popular python version of the hello world app.

Please let me know if there’s any fix for this issue. It doesn’t seems like there’s any obvious issue on my side. (If you know what I mean) Also removing images and changing resources name didn’t work.

erikhuisman commented 1 year ago

This error happens to me on the sample project:

No problem invoking and run the api for the plain js version

jonaszeu commented 1 year ago

The same problem applies to me, also mac m1 ventura 13.x. Unfortunately, image build and renaming don't help me either. I also tried reinstalling CLI, but that didn't make any difference for me.

TimothyBrehm commented 1 year ago

I'm also seeing this issue. On a Mac. Making nominal code changes sometimes gets it to go away, as does changing the resource name.

--Edit with more details-- This was happening primarily with my authorizer function, but also happened with other functions. Reinstalling the SAM CLI appears to have fixed it. I was also seeing errors with my functions not shutting down in docker I was also experiencing network issues (looks like sam local start-api likes to phone home when kicking off?

GeeWee commented 1 year ago

Also seeing this error on Mac M1 with this starter template. I previously also got it on the plain JS template, but not after updating the CLI to the latest version.

A workaround for me seems to be running sam build manually before running sam local start-api - then it seems to work

keeganstothert commented 11 months ago

Same error on mac (m2) for me the api runs fine for some random amount of time then blows up with this error and requires me to kill and re-run sam local start-api to get it working again

hnnasit commented 11 months ago

Hi, I've tried to reproduce the issue using different init templates (nodejs18.x, ts app) but could not reproduce the same behavior. I installed the latest SAM CLI version (1.105.0) using the native PKG installer on a M1 Ventura 13.5.2 machine. Does the issue still occur in the latest version? If so, can you provide the steps/sample app to reproduce it for us to investigate this further? sam build needs to be run for a ts project to transpile the code to JS before running sam local suite of commands. I'm trying to understand why does it not build the lambda function. Here is my sam --info output:

{
  "version": "1.105.0",
  "system": {
    "python": "3.8.13",
    "os": "macOS-13.5.2-arm64-arm-64bit"
  },
  "additional_dependencies": {
    "docker_engine": "24.0.6",
    "aws_cdk": "2.103.1 (build 3bb19ac)",
    "terraform": "1.5.0"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}
vini2001 commented 10 months ago

I am on M1 Ventura 13.5.1 and was having the same issues described above. I had SAM CLI 1.106.0 installed through the GUI package installer, tried most or all the suggested approaches and nothing worked. I had the issue even with the starter hello world template.

In my case, what ended up working for me was completely uninstalling SAM CLI and reinstalling it using brew. The installed version was the same (1.106.0). Not sure if what did the job was installing using brew or if the issue was with the GUI SAM CLI package installer I had used previously.

Hope this is useful for someone else.

ldu2 commented 8 months ago

I’m running into a similar issue on my Mac, OS Ventura 13.6. It happens to all three hello-world ts apps, nodejs14, 16 and 18.

One interesting thing is that after starting the app (any of the three), the first hit to the /hello endpoint gives the classic Error: Cannot find module ‘app’\nRequire stack:\n-/var/runtime/index.mjs”,”stack…. Then the consecutive hits give a different error: Invalid lambda response received: Lambda response must be a valid json.

My sam cli is v1.97.0. I installed with the brew initially, same version, same error. Uninstall that and installed through pkg file. Same error.

This error didn’t happened to me on the most popular python version of the hello world app.

Please let me know if there’s any fix for this issue. It doesn’t seems like there’s any obvious issue on my side. (If you know what I mean) Also removing images and changing resources name didn’t work.

Round back to this after a couple months. It’s working for me now after I reinstalled sam cli to 1.110.0 through brew. I also reinstalled Docker to the most up to date version, but I didn’t end up using the image version of the example instead I used the zip version. I’m using Hello World Node 18 sample. I didn’t test other Node versions sample since I only needed Node 18 to work for now. Hope this helps for others.

developerKumar commented 7 months ago

I got it working by following the steps:

  1. sam build
  2. sam local start-api

This issue occurring when using hello world example with typescript.

cjmyles commented 6 months ago

I'm not sure if this is useful to anyone but I had a problem similar to this. I took an existing Typescript Node application I created, and tried to retrofit it so I could run it as an AWS Lambda function. The purpose of the application is to listen for new files being uploaded to a specific bucket, then process the file. The app uses tsc to compile the TypeScript to JavaScript and output the files to a build folder.

My tsconfig.json file looks like this:

{
    "compilerOptions": {
        "module": "CommonJS",
        "esModuleInterop": true,
        "target": "ES2017",
        "outDir": "./build",
        "strict": true
    },
    "include": ["src/**/*.ts"],
    "exclude": ["node_modules"]
}

I installed the AWS SAM CLI from the AWS website rather than using Homebrew (as AWS no longer maintains the Homebrew version), and Docker.

I then created a template.yaml file in the root (bucket name redacted for illustration purposes):

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Resources:
  MyFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.handler
      Runtime: nodejs20.x
      CodeUri: ./build
      Events:
        S3Event:
          Type: S3
          Properties:
            Bucket: [my-bucket-name]
            Events: s3:ObjectCreated:*

I then modified my index.ts file to export a handler method:

import { S3Event, Context } from "aws-lambda";

export const handler = async (event: S3Event, context: Context): Promise<{ statusCode: number; body: string; }> => {

I also created a event.json file that I could use to simulate a new PUT request on the bucket (using the sam generate-event command):

{
    "Records": [
        {
            "eventVersion": "2.0",
            "eventSource": "aws:s3",
            "awsRegion": "ap-southeast-2",
            "eventTime": "1970-01-01T00:00:00.000Z",
            "eventName": "ObjectCreated:Put",
            "userIdentity": {
                "principalId": "EXAMPLE"
            },
            "requestParameters": {
                "sourceIPAddress": "127.0.0.1"
            },
            "responseElements": {
                "x-amz-request-id": "EXAMPLE123456789",
                "x-amz-id-2": "EXAMPLE123/5678abcdefghijklambdaisawesome/mnopqrstuvwxyzABCDEFGH"
            },
            "s3": {
                "s3SchemaVersion": "1.0",
                "configurationId": "testConfigRule",
                "bucket": {
                    "name": "[my-bucket-name]",
                    "ownerIdentity": {
                        "principalId": "EXAMPLE"
                    },
                    "arn": "arn:aws:s3:::[my-bucket-name]"
                },
                "object": {
                    "key": "[my-key-name]",
                    "size": 1024,
                    "eTag": "0123456789abcdef0123456789abcdef",
                    "sequencer": "0A1B2C3D4E5F678901"
                }
            }
        }
    ]
}

Finally, I built my code and ran sam local invoke --event event.json (I originally thought I had to run the sam start-api command, but since I'm not using API Gateway, this method is more appropriate). This is where I'd experience the following error:

{"errorType":"Runtime.ImportModuleError","errorMessage":"Error: Cannot find module 'node-xlsx'"...

After some trial and error here's what I discovered:

Obviously, copying package.json manually to the build folder isn't a long term solution - so I'll either need to write a script for that as part of the build process, or my project setup isn't quite right. I've been a little lazy with reading the docs on how to set up a TypeScript project with AWS Lambda/SAM so I'll try and find an example Hello World style repository to see how they do it. This is all still a bit of a work in progress, but I thought this might at least help others facing a similar issue.

Edit: Creating a Hello World example (the TypeScript version) using the AWS SAM CLI then modifying it for my needs seemed to fix the issue. See https://aws.amazon.com/blogs/compute/building-typescript-projects-with-aws-sam-cli/ on how to do this.

jghaines commented 5 months ago

I've raised a (possibly tangentially) related issue #7143 BuildMethod: esbuild does not include bundled files where non-source files bundled in an npm library are not included in a typescript/esbuild bundle. A straightforward reproduction is included.

davemanroth commented 4 months ago

I got it working by following the steps:

  1. sam build
  2. sam local start-api

This issue occurring when using hello world example with typescript.

This did it for me, thank you @developerKumar!