Nested lambda handler references are not resolved as expected

ilanc commented 4 years ago

I'm unable to invoke an httpApi lambda when I deploy it using sls framework. However if I simply comment out the framework config ({org, app}) in the serverless.yml and re-deploy (i.e. using vanilla sls), then I am able to invoke the lambda successfully.

For the broken sls framework version I see this in the logs:

Error: Cannot find module './handler.main.js'

Problem cause:

my .js is actually called handler.js not handler.main.js
it appears that vanilla sls handles dotted notation in the function handler name differently to the sls framework version

Background info:

handler.js is a bundle created by webpack and exports a single function (i.e. exports["main"] = ...).

webpack.config.js has:

output: {
  path: path.join(__dirname, "dist"),
  filename: "[name].js",
  libraryTarget: "commonjs",
  library: "main",
},

My serverless.yml references this in the functions section like so: handler: handler.main.post (where post is the name of a function which I export)

serverless.yml

```yaml service: lambda-pdf # org: ilanc # app: lambda-pdf provider: name: aws runtime: nodejs12.x region: eu-west-2 stage: prod deploymentBucket: name: spike-serverless-deploy functions: convert: description: lambda version of @spike/pdf memorySize: 512 timeout: 5 handler: handler.main.post # can't use `export default` from webpacked .ts files events: - httpApi: method: POST path: / ```

sls deploy (framework version) output

``` Serverless: Load command interactiveCli Serverless: Load command config Serverless: Load command config:credentials Serverless: Load command config:tabcompletion Serverless: Load command config:tabcompletion:install Serverless: Load command config:tabcompletion:uninstall Serverless: Load command create Serverless: Load command install Serverless: Load command package Serverless: Load command deploy Serverless: Load command deploy:function Serverless: Load command deploy:list Serverless: Load command deploy:list:functions Serverless: Load command invoke Serverless: Load command invoke:local Serverless: Load command info Serverless: Load command logs Serverless: Load command metrics Serverless: Load command print Serverless: Load command remove Serverless: Load command rollback Serverless: Load command rollback:function Serverless: Load command slstats Serverless: Load command plugin Serverless: Load command plugin Serverless: Load command plugin:install Serverless: Load command plugin Serverless: Load command plugin:uninstall Serverless: Load command plugin Serverless: Load command plugin:list Serverless: Load command plugin Serverless: Load command plugin:search Serverless: Load command config Serverless: Load command config:credentials Serverless: Load command rollback Serverless: Load command rollback:function Serverless: Load command upgrade Serverless: Load command uninstall Serverless: Load command login Serverless: Load command logout Serverless: Load command generate-event Serverless: Load command test Serverless: Load command dashboard Serverless: Load command output Serverless: Load command output:get Serverless: Load command output:list Serverless: Load command param Serverless: Load command param:get Serverless: Load command param:list Serverless: Load command studio Serverless: Load command dev platform-sdk fetching: POST https://api.serverless.com/core/tenants/ilanc/applications/lambda-pdf/profileValue Serverless: Invoke deploy Serverless: Invoke package Serverless: Invoke aws:common:validate Serverless: Invoke aws:common:cleanupTempDir platform-sdk fetching: GET https://api.serverless.com/core/tenants/ilanc/applications/lambda-pdf platform-sdk fetching: GET https://api.serverless.com/core/meta Serverless: Packaging service... Serverless: Excluding development dependencies... Serverless: Invoke aws:package:finalize Serverless: [AWS sts 200 1.6s 0 retries] getCallerIdentity({}) platform-sdk fetching: POST https://api.serverless.com/malt/destinations/create Serverless: Invoke aws:common:moveArtifactsToPackage Serverless: Invoke aws:common:validate Serverless: [AWS s3 200 1.178s 0 retries] getBucketLocation({ Bucket: 'spike-serverless-deploy' }) Serverless: Safeguards Processing... Serverless: Safeguards Results: Summary -------------------------------------------------- passed - no-secret-env-vars passed - no-unsafe-wildcard-iam-permissions passed - framework-version passed - allowed-stages passed - allowed-regions passed - allowed-runtimes Serverless: Safeguards Summary: 6 passed, 0 warnings, 0 errors Serverless: Invoke aws:deploy:deploy Serverless: [AWS cloudformation 200 1.339s 0 retries] describeStacks({ StackName: 'lambda-pdf-prod' }) Serverless: [AWS s3 200 0.653s 0 retries] listObjectsV2({ Bucket: 'spike-serverless-deploy', Prefix: 'serverless/lambda-pdf/prod' }) Serverless: [AWS s3 200 0.627s 0 retries] headObject({ Bucket: 'spike-serverless-deploy', Key: 'serverless/lambda-pdf/prod/1592825346115-2020-06-22T11:29:06.115Z/compiled-cloudformation-template.json' }) Serverless: [AWS s3 200 0.636s 0 retries] headObject({ Bucket: 'spike-serverless-deploy', Key: 'serverless/lambda-pdf/prod/1592825346115-2020-06-22T11:29:06.115Z/lambda-pdf.zip' }) Serverless: [AWS lambda 200 0.755s 0 retries] getFunction({ FunctionName: 'lambda-pdf-prod-convert' }) Serverless: [AWS sts 200 1.205s 0 retries] getCallerIdentity({}) Serverless: Uploading CloudFormation file to S3... Serverless: [AWS s3 200 0.799s 0 retries] putObject({ Body: , Bucket: 'spike-serverless-deploy', Key: 'serverless/lambda-pdf/prod/1592825625601-2020-06-22T11:33:45.601Z/compiled-cloudformation-template.json', ContentType: 'application/json', Metadata: { filesha256: 'iAxKa8p/MiMXr9c2Z14/r4sFVAZ6BUzvPRYH8ODEsrs=' } }) Serverless: Uploading artifacts... Serverless: Uploading service lambda-pdf.zip file to S3 (1.58 MB)... Serverless: [AWS s3 200 5.095s 0 retries] putObject({ Body: , Bucket: 'spike-serverless-deploy', Key: 'serverless/lambda-pdf/prod/1592825625601-2020-06-22T11:33:45.601Z/lambda-pdf.zip', ContentType: 'application/zip', Metadata: { filesha256: 'zEyGnjq91H/z5PQkwWQ+3PmYfgT3Mf9IEMI0cXjhOFs=' } }) Serverless: Validating template... Serverless: [AWS cloudformation 200 1.237s 0 retries] validateTemplate({ TemplateURL: 'https://s3.amazonaws.com/spike-serverless-deploy/serverless/lambda-pdf/prod/1592825625601-2020-06-22T11:33:45.601Z/compiled-cloudformation-template.json' }) Serverless: Updating Stack... Serverless: [AWS cloudformation 200 1.118s 0 retries] updateStack({ StackName: 'lambda-pdf-prod', Capabilities: [ 'CAPABILITY_IAM', 'CAPABILITY_NAMED_IAM', [length]: 2 ], Parameters: [ [length]: 0 ], TemplateURL: 'https://s3.amazonaws.com/spike-serverless-deploy/serverless/lambda-pdf/prod/1592825625601-2020-06-22T11:33:45.601Z/compiled-cloudformation-template.json', Tags: [ { Key: 'STAGE', Value: 'prod' }, [length]: 1 ] }) Serverless: Checking Stack update progress... Serverless: [AWS cloudformation 200 2.455s 0 retries] describeStackEvents({ StackName: 'arn:aws:cloudformation:eu-west-2:366730223589:stack/lambda-pdf-prod/4036d1a0-b471-11ea-9f1b-021ebf3a767c' }) .Serverless: [AWS cloudformation 200 2.649s 0 retries] describeStackEvents({ StackName: 'arn:aws:cloudformation:eu-west-2:366730223589:stack/lambda-pdf-prod/4036d1a0-b471-11ea-9f1b-021ebf3a767c' }) Serverless: [AWS cloudformation 200 2.219s 0 retries] describeStackEvents({ StackName: 'arn:aws:cloudformation:eu-west-2:366730223589:stack/lambda-pdf-prod/4036d1a0-b471-11ea-9f1b-021ebf3a767c' }) Serverless: [AWS cloudformation 200 2.494s 0 retries] describeStackEvents({ StackName: 'arn:aws:cloudformation:eu-west-2:366730223589:stack/lambda-pdf-prod/4036d1a0-b471-11ea-9f1b-021ebf3a767c' }) Serverless: [AWS cloudformation 200 3s 0 retries] describeStackEvents({ StackName: 'arn:aws:cloudformation:eu-west-2:366730223589:stack/lambda-pdf-prod/4036d1a0-b471-11ea-9f1b-021ebf3a767c' }) ................Serverless: [AWS cloudformation 200 1.952s 0 retries] describeStackEvents({ StackName: 'arn:aws:cloudformation:eu-west-2:366730223589:stack/lambda-pdf-prod/4036d1a0-b471-11ea-9f1b-021ebf3a767c' }) ..Serverless: [AWS cloudformation 200 2.73s 0 retries] describeStackEvents({ StackName: 'arn:aws:cloudformation:eu-west-2:366730223589:stack/lambda-pdf-prod/4036d1a0-b471-11ea-9f1b-021ebf3a767c' }) Serverless: [AWS cloudformation 200 2.064s 0 retries] describeStackEvents({ StackName: 'arn:aws:cloudformation:eu-west-2:366730223589:stack/lambda-pdf-prod/4036d1a0-b471-11ea-9f1b-021ebf3a767c' }) .... Serverless: Stack update finished... Serverless: Invoke aws:info Serverless: [AWS cloudformation 200 0.801s 0 retries] describeStacks({ StackName: 'lambda-pdf-prod' }) Serverless: [AWS cloudformation 200 0.824s 0 retries] listStackResources({ StackName: 'lambda-pdf-prod', NextToken: undefined }) Service Information service: lambda-pdf stage: prod region: eu-west-2 stack: lambda-pdf-prod resources: 13 api keys: None endpoints: POST - https://abcdefghij.execute-api.eu-west-2.amazonaws.com/ functions: convert: lambda-pdf-prod-convert layers: None Serverless: Invoke aws:deploy:finalize Serverless: [AWS s3 200 1.014s 0 retries] listObjectsV2({ Bucket: 'spike-serverless-deploy', Prefix: 'serverless/lambda-pdf/prod' }) Serverless: Publishing service to the Serverless Dashboard... Serverless: [AWS sts 200 1.756s 0 retries] getCallerIdentity({}) Serverless: [AWS cloudformation 200 0.705s 0 retries] describeStacks({ StackName: 'lambda-pdf-prod' }) platform-sdk fetching: POST https://api.serverless.com/core/tenants/ilanc/applications/lambda-pdf/services/lambda-pdf/stages/prod/regions/eu-west-2/deployments Serverless: Successfully published your service to the Serverless Dashboard: https://dashboard.serverless.com/tenants/ilanc/applications/lambda-pdf/services/lambda-pdf/stage/prod/region/eu-west-2 (local, /spike/v8) ilan@ilan:/spike/v8/priv/stack/lambda-pdf/dist$ ```

sls deploy (vanilla version) output

``` Serverless: Load command interactiveCli Serverless: Load command config Serverless: Load command config:credentials Serverless: Load command config:tabcompletion Serverless: Load command config:tabcompletion:install Serverless: Load command config:tabcompletion:uninstall Serverless: Load command create Serverless: Load command install Serverless: Load command package Serverless: Load command deploy Serverless: Load command deploy:function Serverless: Load command deploy:list Serverless: Load command deploy:list:functions Serverless: Load command invoke Serverless: Load command invoke:local Serverless: Load command info Serverless: Load command logs Serverless: Load command metrics Serverless: Load command print Serverless: Load command remove Serverless: Load command rollback Serverless: Load command rollback:function Serverless: Load command slstats Serverless: Load command plugin Serverless: Load command plugin Serverless: Load command plugin:install Serverless: Load command plugin Serverless: Load command plugin:uninstall Serverless: Load command plugin Serverless: Load command plugin:list Serverless: Load command plugin Serverless: Load command plugin:search Serverless: Load command config Serverless: Load command config:credentials Serverless: Load command rollback Serverless: Load command rollback:function Serverless: Load command upgrade Serverless: Load command uninstall Serverless: Load command login Serverless: Load command logout Serverless: Load command generate-event Serverless: Load command test Serverless: Load command dashboard Serverless: Load command output Serverless: Load command output:get Serverless: Load command output:list Serverless: Load command param Serverless: Load command param:get Serverless: Load command param:list Serverless: Load command studio Serverless: Load command dev Serverless: Invoke deploy Serverless: Invoke package Serverless: Invoke aws:common:validate Serverless: Invoke aws:common:cleanupTempDir Serverless: Packaging service... Serverless: Excluding development dependencies... Serverless: Invoke aws:package:finalize Serverless: Invoke aws:common:moveArtifactsToPackage Serverless: Invoke aws:common:validate Serverless: [AWS s3 200 0.982s 0 retries] getBucketLocation({ Bucket: 'spike-serverless-deploy' }) Serverless: Invoke aws:deploy:deploy Serverless: [AWS cloudformation 200 1.245s 0 retries] describeStacks({ StackName: 'lambda-pdf-prod' }) Serverless: [AWS s3 200 0.725s 0 retries] listObjectsV2({ Bucket: 'spike-serverless-deploy', Prefix: 'serverless/lambda-pdf/prod' }) Serverless: [AWS s3 200 0.615s 0 retries] headObject({ Bucket: 'spike-serverless-deploy', Key: 'serverless/lambda-pdf/prod/1592823993487-2020-06-22T11:06:33.487Z/compiled-cloudformation-template.json' }) Serverless: [AWS s3 200 0.624s 0 retries] headObject({ Bucket: 'spike-serverless-deploy', Key: 'serverless/lambda-pdf/prod/1592823993487-2020-06-22T11:06:33.487Z/lambda-pdf.zip' }) Serverless: [AWS lambda 200 0.736s 0 retries] getFunction({ FunctionName: 'lambda-pdf-prod-convert' }) Serverless: [AWS sts 200 1.658s 0 retries] getCallerIdentity({}) Serverless: Uploading CloudFormation file to S3... Serverless: [AWS s3 200 0.757s 0 retries] putObject({ Body: , Bucket: 'spike-serverless-deploy', Key: 'serverless/lambda-pdf/prod/1592825346115-2020-06-22T11:29:06.115Z/compiled-cloudformation-template.json', ContentType: 'application/json', Metadata: { filesha256: '7fdekQC70GMEszfkyVGPkF/yV4vxLO0Y7aBQj3Xrqxw=' } }) Serverless: Uploading artifacts... Serverless: Uploading service lambda-pdf.zip file to S3 (1.46 MB)... Serverless: [AWS s3 200 2.964s 0 retries] putObject({ Body: , Bucket: 'spike-serverless-deploy', Key: 'serverless/lambda-pdf/prod/1592825346115-2020-06-22T11:29:06.115Z/lambda-pdf.zip', ContentType: 'application/zip', Metadata: { filesha256: 'G1QbVABfnNM0VX90a85c+6jEcsmB2WWinQn3GAj9a8Q=' } }) Serverless: Validating template... Serverless: [AWS cloudformation 200 1.038s 0 retries] validateTemplate({ TemplateURL: 'https://s3.amazonaws.com/spike-serverless-deploy/serverless/lambda-pdf/prod/1592825346115-2020-06-22T11:29:06.115Z/compiled-cloudformation-template.json' }) Serverless: Updating Stack... Serverless: [AWS cloudformation 200 0.977s 0 retries] updateStack({ StackName: 'lambda-pdf-prod', Capabilities: [ 'CAPABILITY_IAM', 'CAPABILITY_NAMED_IAM', [length]: 2 ], Parameters: [ [length]: 0 ], TemplateURL: 'https://s3.amazonaws.com/spike-serverless-deploy/serverless/lambda-pdf/prod/1592825346115-2020-06-22T11:29:06.115Z/compiled-cloudformation-template.json', Tags: [ { Key: 'STAGE', Value: 'prod' }, [length]: 1 ] }) Serverless: Checking Stack update progress... Serverless: [AWS cloudformation 200 2.615s 0 retries] describeStackEvents({ StackName: 'arn:aws:cloudformation:eu-west-2:366730223589:stack/lambda-pdf-prod/4036d1a0-b471-11ea-9f1b-021ebf3a767c' }) .Serverless: [AWS cloudformation 200 2.585s 0 retries] describeStackEvents({ StackName: 'arn:aws:cloudformation:eu-west-2:366730223589:stack/lambda-pdf-prod/4036d1a0-b471-11ea-9f1b-021ebf3a767c' }) Serverless: [AWS cloudformation 200 2.469s 0 retries] describeStackEvents({ StackName: 'arn:aws:cloudformation:eu-west-2:366730223589:stack/lambda-pdf-prod/4036d1a0-b471-11ea-9f1b-021ebf3a767c' }) Serverless: [AWS cloudformation 200 2.371s 0 retries] describeStackEvents({ StackName: 'arn:aws:cloudformation:eu-west-2:366730223589:stack/lambda-pdf-prod/4036d1a0-b471-11ea-9f1b-021ebf3a767c' }) .... Serverless: Stack update finished... Serverless: Invoke aws:info Serverless: [AWS cloudformation 200 0.917s 0 retries] describeStacks({ StackName: 'lambda-pdf-prod' }) Serverless: [AWS cloudformation 200 1.319s 0 retries] listStackResources({ StackName: 'lambda-pdf-prod', NextToken: undefined }) Service Information service: lambda-pdf stage: prod region: eu-west-2 stack: lambda-pdf-prod resources: 9 api keys: None endpoints: POST - https://abcdefghij.execute-api.eu-west-2.amazonaws.com/ functions: convert: lambda-pdf-prod-convert layers: None Serverless: Invoke aws:deploy:finalize Serverless: [AWS s3 200 0.763s 0 retries] listObjectsV2({ Bucket: 'spike-serverless-deploy', Prefix: 'serverless/lambda-pdf/prod' }) Serverless: Run the "serverless" command to setup monitoring, troubleshooting and testing. ```

Installed version

Framework Core: 1.72.0
Plugin: 3.6.13
SDK: 2.3.1
Components: 2.30.12

Additional:

webpack.config.js

```js const path = require("path"); const webpack = require("webpack"); const functionSrcFile = "handler"; module.exports = { target: "node", // mode: "production", mode: "development", optimization: { usedExports: true, }, entry: { [functionSrcFile]: ["./build/main/" + functionSrcFile], // will create ../dist/{functionSrcFile}.js }, output: { path: path.join(__dirname, "dist"), filename: "[name].js", libraryTarget: "commonjs", library: "main", }, module: { rules: [ { test: /\.js?$/, exclude: /node_modules/, loader: "babel-loader", options: { presets: [ [ "@babel/env", { targets: { node: "12", }, }, ], ], }, }, ], }, resolve: { extensions: [".mjs", ".js"], }, externals: ["aws-sdk"], // aws-sdk included in lambda - no need to bundle it };

medikoo commented 4 years ago

@ilanc can you prepare a clean reproduction test case, that doesn't involve a Webpack? It might that issue is in your webpack configuration and that's out of scope of this project.

ilanc commented 4 years ago

howdy - the problem is the UserFunction.js code inside the serverless framework. I just logged on to try and get the callstack from the dashboard.serverless.com but it's hidden (as of 10 mins ago? with a message about upgrading serverless enterprise plugin?).

It looked like the sls framework injects it's own handler which then needs to find and require your handler and run it. I'm pretty sure that's where the problem is. If UserFunction.js see a handler with dotted notation then it need to strip out the parts and look for files in /tmp until it finds a match

i.e. for handler.main.post it should look for:

/tmp/handler.js => then call main.post() => this is what I need (and what sls vanilla / lambda framwork appears to already do)
/tmp/handler.main.js => then call post()
/tmp/handler.main.post.js => then call default() i guess?

medikoo commented 4 years ago

@ilanc for general support please use https://forum.serverless.com/ or https://www.serverless.com/slack

This issue tracker is purely for bug reports or feature requests, and those should be reported with full respect to template remarks (e.g. no webpack, external plugins etc. involved)

ilanc commented 4 years ago

Hi @medikoo, this is a valid bug report. The serverless framework has different behaviour to the vanilla sls - behaviour that will prevent non-default export handler.js functions from being called AFAICT - i.e. a bug.

Ignore the comments about the dashboard I just mentioned this because I was going to send you the stack trace from the dashboard which would have made it obvious where the bug lies.

I'm well acquainted with the various serverless offerings as well as the forum, this is the correct place for this bug report.

medikoo commented 4 years ago

Hi @medikoo, this is a valid bug report. The serverless framework has different behaviour to the vanilla sls - behaviour that will prevent non-default export handler.js functions from being called AFAICT - i.e. a bug.

@ilanc can you prove it with reproducible test case that doesn't involve webpack?

ilanc commented 4 years ago

sure

ilanc commented 4 years ago

Here you go: https://github.com/ilanc/bug-demo

ilanc commented 4 years ago

@medikoo you'll need to follow the instructions in the readme to reproduce + edit the framework tenant settings for your account.

medikoo commented 4 years ago

Thank you @ilanc

It appears that dashboard plugin doesn't handle nested handler references properly

/cc @astuyve @scouredimage

ilanc commented 4 years ago

:+1:

ilanc commented 4 years ago

Hi @medikoo - I see that I am assigned to this. Are you awaiting a pull request from me? or am I supposed to comment on the linked PR from @astuyve?

medikoo commented 4 years ago

Hi @medikoo - I see that I am assigned to this. Are you awaiting a pull request from me?

@ilanc I've assigned you initially as post was not clear to me. I've removed the assignment.

Still if you know how to fix it, PR is definitely welcome

or am I supposed to comment on the linked PR from @astuyve?

Which PR you mean?

ilanc commented 4 years ago

Thanks @medikoo. Ignore my comment about the PR - I saw that there was "1 PR" in the tabs at the top of the page and assumed that it was linked. But it's actually just a list of all currently active PRs.

serverless / dashboard-plugin

Nested lambda handler references are not resolved as expected #441