[1.16.x/1.17.x] node_modules is included in the package file

[yonathan06]

I just updated to serverless v1.17, and the plugin stopped working (the function size was 28mb) after downgrading to v1.16, the plugin was back (function size 68kb)

[goncaloneves]

Thanks for reporting @yonathan06. I will have a look at this in the next couple of days.

[staceymoore]

I can confirm that this started happening as of Serverless 1.16. The node_modules directory is included in the deployment package.

[goncaloneves]

@yonathan06 @staceymoore I just performed tests on my environment on SLS 1.17 and it's working fine. I was before in 1.16.x.

1. Can you share your service serverless.yml. Could you pinpoint what is breaking in your configuration?
2. What is the optimize configuration you are using?
3. Is optimize plugin being called on deploy command?

If more people are having this issue please come in and share what you have.

[staceymoore]

@goncaloneves my plugin config is super simple:

optimize:
    global: true

It is being called on deploy and it is optimizing the file. However, entire packages in node_modules are included in the deploy package. I suspect that this may have something to do with changes in Serverless 1.16 that prevent dev dependencies from being included in a deploy.

To recreate the issue, make sure you have some node modules installed in your project. The complete module will get bundled in with each function in the node_modules folder even if it is not used in a require within the function. Hope that helps.

[goncaloneves]

@staceymoore I have been testing here and I got to your problem by setting package individually to false.

I checked what's inside the _optimize folder and they are just the Browserified files without node_modules. But then inside .serverless the package has the parent directories. As you can see in the debug output below Optimize is setting the globs correctly in all functions package options.

Looks to me SLS package is embedding somehow the parent folder without following the functions package globs. I had a look in SLS issues and seems there are some issues open currently about packaging. I didn't go into depth in the extent.

Could you try setting package to individually and check results.

In my test case the optimize setup:

Optimize: debug {
  "functions": [
    {
      "bundle": "/test/_optimize/warmup/_warmup/index.js",
      "handlerOriginal": "_warmup/index.warmUp",
      "handlerOptimize": "_optimize/warmup/_warmup/index.warmUp",
      "package": {
        "exclude": [
          "**"
        ],
        "include": [
          "_optimize/warmup/**"
        ]
      }
    },
    {
      "bundle": "/test/_optimize/function1/index.js",
      "handlerOriginal": "function1/index.handler",
      "handlerOptimize": "_optimize/function1/index.handler",
      "package": {
        "exclude": [
          "**"
        ],
        "include": [
          "_optimize/function1/**"
        ]
      }
    },
    {
      "bundle": "/test/_optimize/function2/index.js",
      "handlerOriginal": "function2/index.handler",
      "handlerOptimize": "_optimize/function2/index.handler",
      "package": {
        "exclude": [
          "**"
        ],
        "include": [
          "_optimize/function2/**"
        ]
      }
    }
  ],
  "options": {
    "debug": true,
    "exclude": [
      "aws-sdk"
    ],
    "external": [],
    "externalPaths": {},
    "extensions": [],
    "global": true,
    "includePaths": [],
    "ignore": [],
    "minify": true,
    "plugins": [],
    "prefix": "_optimize",
    "presets": [
      [
        "/test/node_modules/babel-preset-env/lib/index.js",
        {
          "targets": {
            "node": "6.10"
          }
        }
      ]
    ]
  }
}

[goncaloneves]

Adding more information to this, probably this is related https://github.com/serverless/serverless/issues/3869.

(Edit): What can be done on the optimize side is enforce package individually to be true all times. But that removes functionality for some people even though now is broken.

What are your thoughts? I can release a patch to put some tape on this. 🤔

[staceymoore]

@goncaloneves I already have package individually set to true as follows:

package:
  individually: true

Looks like Serverless is working on this in https://github.com/serverless/serverless/pull/3889

Also, I attempted doing an explicit exclude as follows, but that didn't work for me either.

package:
  individually: true
  exclude:
    - node_modules/**

[goncaloneves]

@staceymoore so not even with individually to true. 😟

Set debugto true and tell me what you see inside the _optimize folder? You should see inside the paths with your handler bundles with nothing else.

[staceymoore]

@goncaloneves , that is correct. The _optimize directory only contains the optimized bundles, but each zipped deployment package in the .serverless directory also has the node_modules directory, including aws-sdk.

[goncaloneves]

@staceymoore yes, those are the same results I have. Serverless package globs aren't working properly. Even though in my case I don't get this issue with individual package turned on.

From the plugin standpoint, it looks like it's working fine. The final .serverless packaging from the _optimize handlers is the framework responsibility. Let's hope for a quick fix.

[yonathan06]

Sorry for joining this late This is happening in the same way for me - in the _optimize plugin there is the bundled file, but there is also all of the node_modules in the .serverless folder

[goncaloneves]

I am releasing a fix for when individually is false to set SLS global include and exclude paths. Please try the new release and let me know the results.

[yonathan06]

@goncaloneves now it's working, thanks!!