Non flat directory structure

[andrewoh531]

Is it possible to have a non-flat directory structure? I'm trying to put my functions under:

src
  |---functions
        |---hello.js

I've updated my serverless.yml configuration's handler to src/functions/hello.handler. I'm able to run it locally but when I deploy it and then attempt to execute it I get the error:

Unable to import module 'src/functions/hello': Error

[jangerhofer]

I can't figure out whether it's a Webpack configuration problem or an issue with the serverless-webpack plugin, though I am leaning toward the former. Where I include exports from files beneath the handler root directory, webpack does not include those files in the build generated by serverless deploy and serverless webpack --out dist.

I'll keep experimenting and report back if I can find a solution. In the meantime, if anyone happens to have a suggestion, please do advise! :)

[andrewoh531]

I spent a bit of time on it on the weekend and I also think it's the former. I can create a function under src/functions/ and it will run when deployed if there aren't other imports. As soon as I add an import it seems to fail when deployed.

[jayair]

@andrewoh531 Thanks for investigating. Let me take a look and figure out what is going on.

[fwang]

@andrewoh531 @jangerhofer I couldn't reproduce with my test setup. Here's what I have:

serverless.yml

  test_deep:
    handler: controllers/deep/test_deep.main
    events:
      - http:
          path: test_deep_with_import
          method: get

controllers/deep/test_deep.js

import cool from 'cool-ascii-faces';

export const main = async (event, context, callback) => {
  callback(null, {
    statusCode: 200,
    body: JSON.stringify({
      status: cool()
    })
  });
};

Deploy:

$ sls deploy
Serverless: Bundling with Webpack...
Time: 1833ms
                            Asset     Size  Chunks             Chunk Names
    controllers/deep/test_deep.js  4.85 kB       0  [emitted]  controllers/deep/test_deep
Serverless: Packing external modules: source-map-support@^0.4.14, babel-runtime@^6.23.0, cool-ascii-faces@1.3.4
...

Request:

$ curl https://twilwzvoe7.execute-api.us-east-1.amazonaws.com/dev/test_deep_with_import
{"status":"ლ(^o^ლ)"}

Let me know if you guys are seeing other wise.

[andrewoh531]

Been playing around with it a bit more. I'm not sure how @jangerhofer reproduced the issue but issue seems to be related to importing specific modules. Importing a module from the handler is fine for most cases, it's just specific module import that it errors on. Also it's only reproducible when deployed. Invoking it locally is fine.

I have a very simple handler and trying to import dynogels-promisifed seems to cause the error. Do you know of any potential issues or limitations with webpack or babel when importing modules? I maintain dynogels-promisified and have used it on various different projects without any issues. In fact I've used it in a previously deployed version of Lambda (using Apex) using ES6 without any issues.

Below is a snippet of my code:

// Line below causes error even though it's not used. 
import dynogels from 'dynogels-promisified';

export function handler(event, ctx, cb) {

  const response = {
    statusCode: 200,
    body: JSON.stringify({
      message: 'yo',
      input: event,
    }),
  };

  cb(null, response);
}

[andrewoh531]

Looking into this further it appears to be related to peer dependencies. dynogels-promisified has a peer dependency on dynogels which wasn't being packed as an external dependency. If I explicitly include an import statement for dynogels that module is packed as a dependency otherwise it's not, even though it's included in package.json.

Is this an issue with serverless-webpack or with how webpack.config.js is configured?

[jayair]

@andrewoh531 I looked into it a bit.

There are some specific things serverless-webpack is doing that is causing this. We are using the externals config option and the webpackIncludeModules option. You can read more about it here.

However, your case of specifying a dependency in package.json without an explicit import should still work.

All modules stated in externals will be excluded from bundled files. If an excluded module is stated as dependencies in package.json, it will be packed into the Serverless artifact under the node_modules directory.

Can you share the handler function that is causing a problem? And your package.json?

[andrewoh531]

Hi jayair, yes I did see the reference on it in serverless-webpack. Haven't tested whether it works properly without using the webpack configuration provided by your generator to isolate the problem as I've been using the workaround.

The handler and package.json are below. Nothing particular interesting:

// Line below causes error even though it's not used. 
import dynogels from 'dynogels-promisified';

export function handler(event, ctx, cb) {

  const response = {
    statusCode: 200,
    body: JSON.stringify({
      message: 'yo',
      input: event,
    }),
  };

  cb(null, response);
}

{
  "name": "users",
  "version": "1.0.0",
  "description": "A starter project for the Serverless Framework with ES7 support",
  "main": "handler.js",
  "scripts": {
    "test": "NODE_ENV=test AWS_REGION=ap-southeast-2 mocha --recursive ./src/tests/ --compilers js:babel-core/register --opts ./src/tests/mocha.opts"
  },
  "author": "",
  "license": "MIT",
  "dependencies": {
    "babel-runtime": "^6.23.0",
    "dynogels": "^8.0.0",
    "dynogels-promisified": "^1.0.4",
    "joi": "^10.5.0",
    "node-uuid": "^1.4.8",
    "source-map-support": "^0.4.14"
  },
  "devDependencies": {
    "babel-core": "^6.24.0",
    "babel-loader": "^6.4.1",
    "babel-plugin-transform-runtime": "^6.23.0",
    "babel-preset-es2015": "^6.24.0",
    "babel-preset-stage-3": "^6.24.1",
    "chai": "^3.5.0",
    "dynalite": "^1.2.0",
    "js-yaml": "^3.8.2",
    "lambda-tester": "^2.8.1",
    "mocha": "^3.4.1",
    "serverless-webpack": "^1.0.0-rc.4",
    "webpack": "^2.3.1",
    "webpack-node-externals": "^1.5.4"
  }
}

[jayair]

@andrewoh531 Thanks I'll try it out. It's interesting because you don't really need to specify the externals option if you are not using the AWS SDK. Can you try removing that from your webpack.config.js?

[andrewoh531]

R u talking about removing webpack-node-externals? That didn't work:

  Error --------------------------------------------------

     Cannot find module 'webpack-node-externals'

[jayair]

Oh not that. Try removing this line https://github.com/AnomalyInnovations/serverless-es7/blob/master/webpack.config.js#L29 and this line https://github.com/AnomalyInnovations/serverless-es7/blob/master/serverless.yml#L22.

[hammadzz]

@andrewoh531 I am having the same problem. It just kills the lambda due to that import. Can't figure out why. Runs on sls invoke local just fine. Could you share your exact line for a workaround? I tried everything at this point.

Edit: tried just dynogels plain and that does not work either. Something about it does not play nice with web pack I guess.

[andrewoh531]

@hammadzz sorry for the late reply. It's been awhile but I think I ended up scrapping ES7. I wanted to use async/await but eventually decided the trouble wasn't worth it.

[hammadzz]

@andrewoh531 I don't think it was webpack babel setup. Turns out something else was whack with the lambda itself. After increasing the memory to 512mb it kind of flushed out and worked. It seems Lambda had frozen up and it wasn't just the memory issue. I would delete it and recreate with more memory and try.

In the end dynogels-promisified worked for me in lambda with es6/es7 syntax (transpiled ofcourse)