[FEAT] nx-serverless - add ability to exclude modules from generated package.json

[jamesgroat]

I'd like to be able to exclude modules from the serverless deploy package. I have layers setup that include a lot of modules that I use across multiple functions.

I'd like to be able to specify excluded modules in the build options. There is a forceExcludes parameter passed to getProdModules that is always passed in as an empty array []. https://github.com/flowaccount/nx-plugins/blob/0efe67a6a5fa089820afc99ca44e65b785ca1842/libs/nx-serverless/src/utils/normalize.ts#L236

In addition forceExcludes is only used to avoid throwing an error if a runtime dependency is in devDependencies. https://github.com/flowaccount/nx-plugins/blob/0efe67a6a5fa089820afc99ca44e65b785ca1842/libs/nx-serverless/src/utils/normalize.ts#L314

I propose that we add forceExcludes to build options and not add those modules to the created package.json.

Check which provider is affected: [X] AWS [X] Azure [X] Google Cloud Platform

Check which framework is affected: [] Angular [] Nodejs [X] Serverless [] Lambda [] Infrastructure as a code

Additional context I've tested deploys w/ the modules that I have in my layer removed from the package.json and it works as expected. Doing this takes my individual function bundle sizes down from 12.8mb each to a few kb.

[hoang-innomize]

I am also looking to see a way to force exclude packages that we managed in the layers. Our package size is going bigger (around 20-30mb) and we have around 5-10 functions per service, so we really need this feature.

[hoang-innomize]

@jamesgroat are you able to share code snippets?

[wickstargazer]

hmm ... we can do so by adding it into angular.json or workspace.json and use it as parameters? or would you guys rather manage it from serverless.yaml ?

[hoang-innomize]

I think adding to serverless.yml is better (the serverless-webpack plugin applies the same approach), it to avoid all services that are affected when modifying the angular.json or workspace.json.

By the way, what is your idea on how to turn off the packager in the nx-serverless and use another (i.e. serverless-webpack)?

[wickstargazer]

hmm the reason i built this packagr is to make it work with nx and serverless-webpack obsolete.

I think a better approach is how to allow custom webpack configurations using the same packagr?

The webpack part is actually using the angular builders, and packagr is just used to prune the packages to make it small and robust. 😄

[hoang-innomize]

Custom webpack.config.js file is already supported now by nx-serverless. but we want to add some customization per service (on serverless.yml file) such as forceExclude, forceInclude, toggle package.individually option (seems not to work now)

[wickstargazer]

I think that can be fix in the packagr itself ... Rather do it that way ... Lets draft it out, I will have a session or two this month to re-work on the bugs/enhancements that are there in the repo

[wickstargazer]

 // TODO: issue #48
        return getProdModules(
          externals,
          packageJson,
          originPackageJsonPath,
          [],
          dependencyGraph,
          verbose
        );

Have to get the forceexludes from the serverless config here in depcheck.ts and webpack.stats.ts

[jon-gourley]

Will this be implemented soon? This is an essential feature IMO and sounds like a simple fix. Would be great to have an option for the build builder to exclude a list of packages. I've had to manually edit normalize.js in order to exclude chrome-aws-lambda from one my serverless apps that uses this plugin.

This is an awesome plugin btw 🙏

[AtlasRW]

About this forceExcludes option, it would be great to be able to exclude all node_modules in one line, maybe a glob pattern string array which would permit to do it like node_modules/* (rootDir must be the root of the workspace) or maybe a specific option about node_modules/package.json dependencies would be a great addition.

Most importantly, an option like this in serverless.yml would probably cause problems with serverless-offline as it is based on the same build target, and would not be able to access the layer as it is local.

Using an option in the angular.json/workspace.json seems like a possible fix, if this hypothetical option (an excludeNodeModules boolean for this example) is put in the build target options, it would be possible to base the serve and deploy on different build configurations :

{
  "build": {
    "executor": "@flowaccount/nx-serverless:build",
    "configurations": {
      "dev-offline": {
        "excludeNodeModules": false
      },
      "production-offline": {
        "excludeNodeModules": false
      },
      "production-no-deps": {
        "excludeNodeModules": true
      }
    }
  },
  "serve": {
    "executor": "@flowaccount/nx-serverless:offline",
    "configurations": {
      "dev": {
        "buildTarget": "myApp:build:dev-offline"
      },
      "production": {
        "buildTarget": "myApp:build:production-offline"
      }
    }
  },
  "deploy": {
    "executor": "@flowaccount/nx-serverless:deploy",
    "options": {
      "buildTarget": "myApp:build:production-no-deps"
    }
  }
}