Nx & nestjs@6.0.0 errors using "ng build --prod"

[ZenSoftware]

I'm submitting a...

[x] Bug report

Current behavior

Nx project with a bare-bones nestjs@6.0.0 app does not compile when using the --prod flag.

Does not compile

• ng build --prod

Compiles

• ng serve
• ng build without the --prod flag
• nestjs@5.7.4 works with the --prod flag

Expected behavior

Should compile

Minimal reproduction of the problem with instructions

Nx & nestjs@6.0.0 bug reproduction

Environment

- Nest: 6.0.0
- Node: 10.15.3
- Nx: 7.7.1
- Platform: Linux

### Build log ``` $ ng build --prod Starting type checking service... Using 6 workers with 2048MB memory limit Hash: de380795417288d36cc2 Version: webpack 4.29.0 Time: 13666ms Built at: 03/17/2019 12:50:11 PM Asset Size Chunks Chunk Names main.js 1.14 MiB 0 main main.js.map 1.65 MiB 0 main Entrypoint main = main.js main.js.map [0] ./node_modules/tslib/tslib.es6.js 10.1 KiB {0} [built] [16] ./node_modules/@nestjs/common/index.js 695 bytes {0} [built] [81] ./node_modules/@nestjs/core/constants.js 486 bytes {0} [built] [88] ./apps/api/src/app/app.service.ts 395 bytes {0} [built] [145] ./node_modules/@nestjs/core/helpers/index.js 236 bytes {0} [built] [149] ./node_modules/@nestjs/core/nest-application-context.js 10.9 KiB {0} [built] [151] ./node_modules/reflect-metadata/Reflect.js 50 KiB {0} [built] [302] ./node_modules/@nestjs/core/services/index.js 236 bytes {0} [built] [307] ./node_modules/@nestjs/core/nest-application.js 14.5 KiB {0} [built] [313] ./node_modules/@nestjs/core/index.js 972 bytes {0} [built] [329] ./apps/api/src/app/app.module.ts 485 bytes {0} [built] [330] ./apps/api/src/app/app.controller.ts 913 bytes {0} [built] [331] multi ./apps/api/src/main.ts 28 bytes {0} [built] [332] ./apps/api/src/main.ts 992 bytes {0} [built] [333] ./node_modules/@nestjs/core/adapters/index.js 231 bytes {0} [built] + 555 hidden modules WARNING in ./node_modules/@nestjs/common/utils/load-package.util.js 8:39-59 Critical dependency: the request of a dependency is an expression @ ./node_modules/@nestjs/core/nest-application.js @ ./node_modules/@nestjs/core/index.js @ ./apps/api/src/main.ts @ multi ./apps/api/src/main.ts WARNING in ./node_modules/@nestjs/core/helpers/load-adapter.js 8:15-39 Critical dependency: the request of a dependency is an expression @ ./node_modules/@nestjs/core/nest-factory.js @ ./node_modules/@nestjs/core/index.js @ ./apps/api/src/main.ts @ multi ./apps/api/src/main.ts WARNING in ./node_modules/optional/optional.js 6:15-30 Critical dependency: the request of a dependency is an expression @ ./node_modules/@nestjs/core/nest-application.js @ ./node_modules/@nestjs/core/index.js @ ./apps/api/src/main.ts @ multi ./apps/api/src/main.ts ERROR in ./node_modules/@nestjs/core/nest-application.js Module not found: Error: Can't resolve '@nestjs/microservices' in '/home/z/Work/nx-nest/node_modules/@nestjs/core' @ ./node_modules/@nestjs/core/nest-application.js 148:124-156 @ ./node_modules/@nestjs/core/index.js @ ./apps/api/src/main.ts @ multi ./apps/api/src/main.ts ERROR in ./node_modules/@nestjs/core/nest-factory.js Module not found: Error: Can't resolve '@nestjs/microservices' in '/home/z/Work/nx-nest/node_modules/@nestjs/core' @ ./node_modules/@nestjs/core/nest-factory.js 56:136-168 @ ./node_modules/@nestjs/core/index.js @ ./apps/api/src/main.ts @ multi ./apps/api/src/main.ts ERROR in ./node_modules/@nestjs/common/cache/cache.providers.js Module not found: Error: Can't resolve 'cache-manager' in '/home/z/Work/nx-nest/node_modules/@nestjs/common/cache' @ ./node_modules/@nestjs/common/cache/cache.providers.js 10:116-140 @ ./node_modules/@nestjs/common/cache/cache.module.js @ ./node_modules/@nestjs/common/cache/index.js @ ./node_modules/@nestjs/common/index.js @ ./apps/api/src/app/app.module.ts @ ./apps/api/src/main.ts @ multi ./apps/api/src/main.ts ERROR in ./node_modules/@nestjs/common/pipes/validation.pipe.js Module not found: Error: Can't resolve 'class-transformer' in '/home/z/Work/nx-nest/node_modules/@nestjs/common/pipes' @ ./node_modules/@nestjs/common/pipes/validation.pipe.js 52:119-147 @ ./node_modules/@nestjs/common/pipes/index.js @ ./node_modules/@nestjs/common/index.js @ ./apps/api/src/app/app.module.ts @ ./apps/api/src/main.ts @ multi ./apps/api/src/main.ts ERROR in ./node_modules/@nestjs/common/serializer/class-serializer.interceptor.js Module not found: Error: Can't resolve 'class-transformer' in '/home/z/Work/nx-nest/node_modules/@nestjs/common/serializer' @ ./node_modules/@nestjs/common/serializer/class-serializer.interceptor.js 33:131-159 34:8-36 @ ./node_modules/@nestjs/common/serializer/index.js @ ./node_modules/@nestjs/common/index.js @ ./apps/api/src/app/app.module.ts @ ./apps/api/src/main.ts @ multi ./apps/api/src/main.ts ERROR in ./node_modules/@nestjs/common/pipes/validation.pipe.js Module not found: Error: Can't resolve 'class-validator' in '/home/z/Work/nx-nest/node_modules/@nestjs/common/pipes' @ ./node_modules/@nestjs/common/pipes/validation.pipe.js 51:115-141 @ ./node_modules/@nestjs/common/pipes/index.js @ ./node_modules/@nestjs/common/index.js @ ./apps/api/src/app/app.module.ts @ ./apps/api/src/main.ts @ multi ./apps/api/src/main.ts ```

[kamilmysliwiec]

@FrozenPandaz can I somehow help with this issue? It seems that Nx builder is trying to bundle all dependencies, even lazy loaded ones. For instance, this require (cache.providers.ts file in common):

const cacheManager = loadPackage('cache-manager', 'CacheModule', () =>
   require('cache-manager'),
);

shouldn't be bundled if CacheModule is not being used.

[ZenSoftware]

Edit: Below is not entirely correct after further investigation. Refer to @kamilmysliwiec comment below.

After spending a few hours on this, I have found the root of the problem!
ng build --prod is equivalent to ng build --configuration=production. @angular/cli looks for the configuration named production in the angular.json file under architect.configurations. Here is the scaffolded Nest configuration produced by Nx:

"architect": {
  "configurations": {
    "production": {
      "optimization": true,
      "extractLicenses": true,
      "inspect": false,
      "fileReplacements": [
      {
        "replace": "apps/api/src/environments/environment.ts",
        "with": "apps/api/src/environments/environment.prod.ts"
      }
      ],
      "externalDependencies": "none"
    }
  }
}

I believe removing the line "externalDependencies": "none" skips the webpack tree shaking processing. It compiles fine if you remove that line from the configuration.

@FrozenPandaz Fortunately server side build sizes are not important for most scenarios. Would it be possible to make a change to Nx so that "externalDependencies": "none" is not included in the scaffolding of Nest projects? It should unblock people from upgrading to Nest v6.

[kamilmysliwiec]

So basically I have dig into this issue a little bit and I have some takeaways to share. Firstly, this issue is not caused by 6.0.0 release - it wasn't working properly in 5.x either. However, in 5.x you were able to compile your prod app unless you used ValidationPipe or CacheModule - then, it apparently stopped to work. This has changed though, because we are no longer using expressions (variables) inside require() function, but we're rather passing a loader function that has dependency defined statically.

Hence, this for instance:

const cacheManager = loadPackage('cache-manager', 'CacheModule', () =>
   require('cache-manager'),
);

leads to compilation errors, because webpack traverse modules graph and tries to resolve cache-manager dependency even though(!) CacheModule might not be used (tree-shaking is being performed later on, once bundle is created). Consequently, it breaks Nest idea of lazy require() calls, because webpack tries to load them all during the bundling process.

How to fix this? The easiest solution would be to use webpack-node-externals package to avoid bundling node_modules (bundle & tree-shake only user code). Usage is pretty simple:

externals: [nodeExternals()], 

And it should be fine in 99% cases. However, sometimes you may prefer to bundle entire application (one file, smaller size, somewhat lower memory consumption, slightly faster bootstrap time [serverless]) - in this case, you could use webpack.IgnorePlugin plugin.

 plugins: [
    ...options.plugins,
    new webpack.IgnorePlugin({
      checkResource(resource) {
        const lazyImports = ['@nestjs/microservices', '@nestjs/platform-express', 'cache-manager', 'class-validator', 'class-transformer'];
        if (!lazyImports.includes(resource)) {
          return false;
        }
        try {
          require.resolve(resource);
        } catch (err) {
          return true;
        }
        return false;
      },
    }),
  ],

This workaround would ignore whitelisted packages IF they aren't present (require.resolve() throws an error). Otherwise, it would bundle them as well. This example would work, but the problem here is that you basically have to hardcode these whitelisted packages in the actual builder (Nx node builder) in this case. Perhaps, there is a way to just conditionally provide additional plugin to determine webpack behavior determine on the framework which is being used. CC @FrozenPandaz

[ZenSoftware]

Edit: The following is a messy solution. Simply setting "externalDependencies": "all" in the angular.json file does the same thing. Nice find @xmlking! ^_^

---

Thanks @kamilmysliwiec for all the hard work. The community loves you for it ^_^

My build process is a bit more convoluted than I was hoping for, but I ended up using @kamilmysliwiec recommendation to use webpack-node-externals.

• Install @angular-builders/custom-webpack via npm i -D @angular-builders/custom-webpack
• Install webpack-node-externals via npm i -D webpack-node-externals
• Creat a webpack.partial.js at the root of your project with the following:

var nodeExternals = require('webpack-node-externals');

module.exports = {
  externals: [nodeExternals()]
};

• Create a build configuration in angular.json file. (Note that this is presuming your Nest app in Nx is named "api")

...

"architect": {
  "custom": {
    "builder": "@angular-builders/custom-webpack:server",
    "options": {
      "customWebpackConfig": {
        "path": "./webpack.partial.js"
      },
      "outputPath": "dist/apps/api",
      "main": "apps/api/src/main.ts",
      "tsConfig": "apps/api/tsconfig.app.json"
    },
    "configurations": {
      "production": {
        "optimization": true,
        "fileReplacements": [
          {
            "replace": "apps/api/src/environments/environment.ts",
            "with": "apps/api/src/environments/environment.prod.ts"
          }
        ]
      }
    }
  }
}

...

• Create a small npm script: ng run api:custom --configuration=production to build the Nest app without node_modules
• Change your Dockerfile to install npm dependencies (if you are using Docker that is)

COPY package*.json ./
RUN npm ci --production

[xmlking]

looks like nx already support excluding some or all node modules from bundling externalDependencies: 'all' | 'none' | string[]; here

for my case "externalDependencies": "all" worked.

[drakenfly]

What if I want to build a dockerfile for the Backend? I don't want all installs of my package.json in the docker image, so I think bundling the nest.js' dependencies should be possible.

FROM node:10 as builder

ENV WORK_DIR /usr/src/app

RUN mkdir ${WORK_DIR}
WORKDIR ${WORK_DIR}

COPY package.json package-lock.json ${WORK_DIR}/

RUN [ "npm", "install" ]

COPY . ${WORK_DIR}

RUN [ "npm", "run", "build", "--" , "--project=backend", "--prod" ]
# RUN [ "npm", "run", "build", "--" , "--project=frontend", "--prod" ]

FROM node:10

ENV WORK_DIR /usr/src/app
WORKDIR ${WORK_DIR}

COPY --from=builder ${WORK_DIR}/dist/apps/backend ${WORK_DIR}
# COPY --from=builder ${WORK_DIR}/dist/apps/frontend ${WORK_DIR}/assets
EXPOSE 3333
CMD [ "node", "main.js" ]

Starting the resulting container will result in this output:

docker run -it -p 8080:3333 test-container
internal/modules/cjs/loader.js:584
    throw err;
    ^

Error: Cannot find module 'tslib'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:582:15)
    at Function.Module._load (internal/modules/cjs/loader.js:508:25)
    at Module.require (internal/modules/cjs/loader.js:637:17)
    at require (internal/modules/cjs/helpers.js:22:18)
    at Object.<anonymous> (/usr/src/app/main.js:91:18)
    at __webpack_require__ (/usr/src/app/main.js:20:30)
    at Module.<anonymous> (/usr/src/app/main.js:231:63)
    at __webpack_require__ (/usr/src/app/main.js:20:30)
    at Object.<anonymous> (/usr/src/app/main.js:222:18)
    at __webpack_require__ (/usr/src/app/main.js:20:30)

"externalDependencies": "none" seems like the option I need, but if fails the build. Anybody got an idea how to handle this?

[FrozenPandaz]

Thanks for the debugging @kamilmysliwiec.

We have the externalDependencies: 'all' | 'none' | string[] option for this. (Uses node-webpack-externals) We figured a good setting in production would be to bundle all dependencies (externalDependencies = 'none') (shipping lighter Docker containers/ electron applications). We can turn this off for NestJS applications if it does not work. If you would still like to bundle, you can provide a list of externalDepencies = [] which are not bundled. And then only add those specific node_modules.

We also accept a webpackConfig option (in Nx 7.7). You can pass in a .js file which exports a function which returns a webpack config.. looks like the following:

/**
 * @param config is the config Nx generates
 * @param context some context about the task
 **/
module.exports = (config, context) {
  // extend, mutate, create a new config (whatever you want)  
  return config;
}

For the root of the issue @kamilmysliwiec:

I feel like (not sure) dependencies/ tree-shaking isn't what is expected. If the user does not use '@nestjs/microservices', it shouldn't be in the bundle at all. In Angular I think the major facilitator of this is NgModule imports. If you don't import HttpClientModule, no Http code will be brought into the compilation. Can you think of where in the source code those imports are brought in and how it can be avoided? Of course, it may be a breaking change so it's too late for v6 :(.

[kamilmysliwiec]

@FrozenPandaz

I feel like (not sure) dependencies/ tree-shaking isn't what is expected. If the user does not use '@nestjs/microservices', it shouldn't be in the bundle at all. In Angular I think the major facilitator of this is NgModule imports. If you don't import HttpClientModule, no Http code will be brought into the compilation. Can you think of where in the source code those imports are brought in and how it can be avoided? Of course, it may be a breaking change so it's too late for v6 :(.

See my response here, especially this part:

Webpack traverses modules graph and tries to resolve cache-manager dependency even though(!) CacheModule might not be used (tree-shaking is being performed later on, once bundle is created). Consequently, it breaks Nest idea of lazy require() calls, because webpack tries to load them all during the bundling process.

Basically, Nest in some cases is using require() calls lazily. For instance, if you don't use either RedisClient or RedisServer, the require('redis') expression won't be evaluated = the package won't have to be installed. This allows us to avoid creating 10 packages for every existing transport strategy just to put 2 classes in there. Also, if someone doesn't use redis strategy, he won't be forced to install redis.

Another, more important example is ValidationPipe which dynamically loads both class-validator and class-transformer. Validation is considered as a crucial feature and is being using among lots of applications. However, we cannot put these 2 libraries into internal dependencies because they also expose decorators to developers that have to be used from outside Nest internal scope. Hence, we have to keep them as peer dependencies. In this case, we would have to again extract ValidationPipe (1 class) to a separate package.

The solution that I have included in my post (with webpack.IgnorePlugin()) generally allow you to still bundle + perform tree-shaking for everything (including lazy require() calls). It would obviously consume error messages about missing Nest-lazy-loaded packages (if they ARE actually missing), but it shouldn't be a big deal since this build is being used in prod mode only(correct?). Also, during the tree-shaking step, webpack would get rid of this code that is not being used anyway so it wouldn't affect the final bundle either.

So reassuming, if we still want to have tree-shaking + bundling, the only thing is to put this webpack.IgnorePlugin() solution (described here) as a custom webpack configuration (available in Nx 7.7, correct?) only for Nest apps.

[xmlking]

testing with nx webpackConfig option, https://github.com/xmlking/ngx-starter-kit/blob/develop/angular.json#L255

var IgnorePlugin = require('webpack').IgnorePlugin;

module.exports = (config, context) => {
  // extend, mutate, create a new config (whatever you want)

  config.plugins = [
    ...config.plugins,
    new IgnorePlugin({
      checkResource(resource) {
        const lazyImports = [
          '@nestjs/microservices',
          '@nestjs/platform-express',
          'cache-manager',
          'class-validator',
          'class-transformer',
        ];
        if (!lazyImports.includes(resource)) {
          return false;
        }
        try {
          require.resolve(resource);
        } catch (err) {
          return true;
        }
        return false;
      },
    }),
  ];

  return config;
};

@FrozenPandaz is this correct way to customize webpack ?

[FrozenPandaz]

I created a PR to revert the change for now. I still suggest people look into bundling in their dependencies but I guess it's hard to provide something that works for NestJS out of the box.

@xmlking Yes, that is correct! I recommend the following setting instead:

"externalDependencies": [
     "@nestjs/microservices",
     "@nestjs/platform-express",
     "class-transformer",
     "class-validator",
     "cache-manager"
]

@kamilmysliwiec Maybe the webpack team can help suggest a better way to handle such module references.

[iangregsondev]

Actually removing the external externalDependencies, and adding all and none - still no go, so i decided to add the packages it was complaining about, it was all going well but then I had to add @nestjs/microservices and then mqtt because it complained that was missing and then i run into this.

It appears there is a hashbank in the js file and webpack doesn't understand it.

ERROR in ./libs/nest-correlation/node_modules/mqtt/mqtt.js 1:0
Module parse failed: Unexpected character '#' (1:0)
You may need an appropriate loader to handle this file type.
> #!/usr/bin/env node
| 'use strict';
| /*
 @ ./libs/nest-correlation/node_modules/@nestjs/microservices/server/server-mqtt.js 13:70-85
 @ ./libs/nest-correlation/node_modules/@nestjs/microservices/server/index.js
 @ ./libs/nest-correlation/node_modules/@nestjs/microservices/index.js
 @ ./libs/nest-correlation/node_modules/@nestjs/core/nest-application.js
 @ ./libs/nest-correlation/node_modules/@nestjs/core/index.js
 @ ./libs/nest-correlation/src/lib/correlation.service.ts
 @ ./libs/nest-correlation/src/index.ts
 @ multi ./libs/nest-correlation/src/index.ts

I am stuck, anyone have a workaround, i tried everything above but no go.

[iangregsondev]

Also tried @FrozenPandaz suggestion of

"externalDependencies": [
     "@nestjs/microservices",
     "@nestjs/platform-express",
     "class-transformer",
     "class-validator",
     "cache-manager"
]

and this produces :-(

Starting type checking service...
Using 6 workers with 2048MB memory limit

<--- Last few GCs --->

[90119:0x104800000]    82233 ms: Mark-sweep 1387.4 (1426.2) -> 1386.9 (1428.2) MB, 569.9 / 0.0 ms  (average mu = 0.149, current mu = 0.014) allocation failure scavenge might not succeed
[90119:0x104800000]    82791 ms: Mark-sweep 1390.2 (1428.2) -> 1389.6 (1430.7) MB, 550.5 / 0.0 ms  (average mu = 0.085, current mu = 0.012) allocation failure scavenge might not succeed

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x24b8556dbe3d]
Security context: 0x17ad2359e6e1 <JSObject>

[iangregsondev]

@kamilmysliwiec @FrozenPandaz Tried another solution previously mentioned but I get the following error passing in a webpackconfig to nrwl.

require(...) is not a function
TypeError: require(...) is not a function

      "architect": {
        "build": {
          "builder": "@nrwl/builders:node-build",
          "options": {
            "outputPath": "dist/libs/nest-correlation",
            "main": "libs/nest-correlation/src/index.ts",
            "tsConfig": "libs/nest-correlation/tsconfig.lib.json",
            "assets": [],
            "webpackConfig": "webpackConfig.js"
          },

var nodeExternals = require("webpack-node-externals")

module.exports = {
  externals: [nodeExternals()]
}

Any help anyone, I am not able to build anything. If all else fails - I suppose i need to manually build each library ... rather than use ng .

[FrozenPandaz]

The change has been reverted in Nx 7.8.0. If you would like to bundle your nest application, you can look into providing an Array of modules to remain external. The update does not run any migrations so if you are still running into issues, remove externalDependencies in the producftion configuration.

[FazioNico]

got the same issue... and working with Nx workspace to manage multiples nestjs application.. Any way to bundle all deps in dist folder of each nestjs application? Or a way to build correct package.json files with only required deps of each nestjs application??

[kamilmysliwiec]

@FazioNico https://github.com/nestjs/nest/issues/1706#issuecomment-475922745

[servrox]

Stumbled upon this issue when I tried to get webpack.server.config.js work in my nx workspace with a seperated nest app for ssr.. (i was missing ContextReplacementPlugin -> Demo+Explanation) Maybe this helps someone.

[idhard]

i'm trying to bundle my apps with webpack to reduce package size as i'm using aws lambda functions and i don't manage to make it work using the suggested solution ....
i'm adding the @nest/graphql module to the app and there are quite lot warning and errors when building the app :

WARNING in ./node_modules/@nestjs/core/helpers/load-adapter.js 8:39-63
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/@nestjs/core/nest-factory.js
 @ ./node_modules/@nestjs/core/index.js
 @ ./src/sls.ts

WARNING in ./node_modules/@nestjs/common/utils/load-package.util.js 8:39-59
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/@nestjs/core/nest-application.js
 @ ./node_modules/@nestjs/core/index.js
 @ ./src/sls.ts

WARNING in ./node_modules/optional/optional.js 6:11-26
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/@nestjs/core/nest-application.js
 @ ./node_modules/@nestjs/core/index.js
 @ ./src/sls.ts

WARNING in ./node_modules/express/lib/view.js 81:13-25
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/express/lib/application.js
 @ ./node_modules/express/lib/express.js
 @ ./node_modules/express/index.js
 @ ./src/sls.ts

WARNING in ./node_modules/type-graphql/dist/helpers/loadResolversFromGlob.js 10:46-63
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/type-graphql/dist/utils/buildSchema.js
 @ ./node_modules/type-graphql/dist/utils/index.js
 @ ./node_modules/type-graphql/dist/index.js
 @ ./node_modules/@nestjs/graphql/dist/graphql-schema-builder.js
 @ ./node_modules/@nestjs/graphql/dist/graphql.factory.js
 @ ./node_modules/@nestjs/graphql/dist/index.js
 @ ./node_modules/@nestjs/graphql/index.js
 @ ./src/app.module.ts
 @ ./src/sls.ts

WARNING in ./node_modules/merge-graphql-schemas/dist/index.esm.js 3782:21-31
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/@nestjs/graphql/dist/graphql-types.loader.js
 @ ./node_modules/@nestjs/graphql/dist/index.js
 @ ./node_modules/@nestjs/graphql/index.js
 @ ./src/app.module.ts
 @ ./src/sls.ts

WARNING in ./node_modules/ts-morph/node_modules/typescript/lib/typescript.js 94814:19-45
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/ts-morph/dist/typescript/public.js
 @ ./node_modules/ts-morph/dist/main.js
 @ ./node_modules/@nestjs/graphql/dist/graphql-ast.explorer.js
 @ ./node_modules/@nestjs/graphql/dist/index.js
 @ ./node_modules/@nestjs/graphql/index.js
 @ ./src/app.module.ts
 @ ./src/sls.ts

WARNING in ./node_modules/subscriptions-transport-ws/node_modules/ws/lib/buffer-util.js
Module not found: Error: Can't resolve 'bufferutil' in '/Users/odifisIH/Sites/cashmetrics/packages/core/node_modules/subscriptions-transport-ws/node_modules/ws/lib'
 @ ./node_modules/subscriptions-transport-ws/node_modules/ws/lib/buffer-util.js
 @ ./node_modules/subscriptions-transport-ws/node_modules/ws/lib/receiver.js
 @ ./node_modules/subscriptions-transport-ws/node_modules/ws/index.js
 @ ./node_modules/subscriptions-transport-ws/dist/server.js
 @ ./node_modules/subscriptions-transport-ws/dist/index.js
 @ ./node_modules/apollo-server-core/dist/ApolloServer.js
 @ ./node_modules/apollo-server-core/dist/index.js
 @ ./node_modules/apollo-server-express/dist/index.js
 @ ./node_modules/@nestjs/graphql/dist/graphql.factory.js
 @ ./node_modules/@nestjs/graphql/dist/index.js
 @ ./node_modules/@nestjs/graphql/index.js
 @ ./src/app.module.ts
 @ ./src/sls.ts

WARNING in ./node_modules/subscriptions-transport-ws/node_modules/ws/lib/validation.js
Module not found: Error: Can't resolve 'utf-8-validate' in '/Users/odifisIH/Sites/cashmetrics/packages/core/node_modules/subscriptions-transport-ws/node_modules/ws/lib'
 @ ./node_modules/subscriptions-transport-ws/node_modules/ws/lib/validation.js
 @ ./node_modules/subscriptions-transport-ws/node_modules/ws/lib/receiver.js
 @ ./node_modules/subscriptions-transport-ws/node_modules/ws/index.js
 @ ./node_modules/subscriptions-transport-ws/dist/server.js
 @ ./node_modules/subscriptions-transport-ws/dist/index.js
 @ ./node_modules/apollo-server-core/dist/ApolloServer.js
 @ ./node_modules/apollo-server-core/dist/index.js
 @ ./node_modules/apollo-server-express/dist/index.js
 @ ./node_modules/@nestjs/graphql/dist/graphql.factory.js
 @ ./node_modules/@nestjs/graphql/dist/index.js
 @ ./node_modules/@nestjs/graphql/index.js
 @ ./src/app.module.ts
 @ ./src/sls.ts

ERROR in ./node_modules/chokidar/node_modules/fsevents/fsevents.node 1:0
Module parse failed: Unexpected character '�' (1:0)
You may need an appropriate loader to handle this file type.
(Source code omitted for this binary file)
 @ ./node_modules/chokidar/node_modules/fsevents/fsevents.js 13:15-41
 @ ./node_modules/chokidar/lib/fsevents-handler.js
 @ ./node_modules/chokidar/index.js
 @ ./node_modules/@nestjs/graphql/dist/graphql-definitions.factory.js
 @ ./node_modules/@nestjs/graphql/dist/index.js
 @ ./node_modules/@nestjs/graphql/index.js
 @ ./src/app.module.ts
 @ ./src/sls.ts

this is my webpack config :

const webpack = require('webpack');
const path = require('path');
const nodeExternals = require('webpack-node-externals');

module.exports = {
  mode: 'production',
  entry: './src/sls.ts',
  target: 'node',
  module: {
    rules: [
      {
        test: /\.tsx?$/,
        use: 'ts-loader',
        exclude: /node_modules/
      }
    ]
  },
  resolve: {
    extensions: [ '.mjs','.tsx', '.ts', '.js' ]
  },
  // externals: [nodeExternals()],
  plugins: [
    new webpack.IgnorePlugin({
      checkResource(resource) {
        const lazyImports = ['@nestjs/microservices', '@nestjs/platform-express','@nestjs/grahpql', 'cache-manager', 'class-validator', 'class-transformer', 'graphql'];
        if (!lazyImports.includes(resource)) {
          return false;
        }
        try {
          require.resolve(resource);
        } catch (err) {
          return true;
        }
        return false;
      },
    }),
  ],
  output: {
    libraryTarget: 'commonjs',
    filename: 'build.js',
    path: path.resolve(__dirname, 'dist')
  }
};

im just using the graphql example from nestJS and is worth mentioning that without adding '.mjs' to the extensions list webpack will also complain: https://github.com/graphql/graphql-js/issues/1272

thanks !

[tayambamwanza]

looks like nx already support excluding some or all node modules from bundling externalDependencies: 'all' | 'none' | string[]; here

for my case "externalDependencies": "all" worked.

Where do you insert this? I'm struggling to find the option.

An example snippet of surrounding code would help a lot.

[marcus-sa]

@tayambamwanza

"architect": {
  "configurations": {
    "production": {
      "optimization": true,
      "extractLicenses": true,
      "inspect": false,
      "fileReplacements": [
        {
          "replace": "apps/api/src/environments/environment.ts",
          "with": "apps/api/src/environments/environment.prod.ts"
        }
      ],
      "externalDependencies": "none" <--- HERE
    }
  }
}

Under architect.configurations.NAME.externalDependencies

[asirko]

I would like to deploy a standalone bundle without any node_modules.

If I understand correctly, the intended way of doing so would be to add "externalDependencies": "none". But it leads me to the same errors that ZenSoftware described. (I am using nest v6.2.4 in a Nx project).

Can someone confirm my following understanding of the work around suggested above:

• exclude problematic dependencies https://github.com/nestjs/nest/issues/1706#issuecomment-478145526
• then provide a minimalist package.json to install requiered node_modules

Did I miss something or is there any plan to correct the "externalDependencies": "none" error?

[anissouissi]

@FrozenPandaz , I tried the solution you proposed in your comment. Unfortunately, it dosen't work. I get this error: $ ng build --prod Schema validation failed with the following errors: Data path "" should NOT have additional properties(externalDependencies).

[ghost]

Come on, any updates on this issue? This is really urgent...

[ZenSoftware]

I have just published a repo which demonstrates how one can bundle Nest and containerize it for Docker. It attempts to address the issues that most people are having here. You can find it here, cheers!

💥 Nest 🔰 Webpack 🔰 Docker 💥

[ghost]

I really don't like the solution with lazyImports or externals. Is there no way on nestjs-side to fix this issue?

[ZenSoftware]

I'm just a humble member of the Nest community. Though, I do think we need to sympathize with the complexity of integrating many 3rd party systems together (such as Nest does), and how there can be problems that may not have eloquent solutions. Bundling and tree shaking code has been something the industry has been struggling with since the inception of modern Javascript (ES6+). Webpack and Rollup seem to be the current front running technologies of the industry. Historically, there was no exiting Javascript framework that managed to solve the bundling/tree shaking problem, due to the fact there were various competing Javascript module systems (Commonjs vs AMD vs ES6). Furthermore, there are framework specific module systems, such as Google's Angular module system, which is what Nest's own module system is inspired after. The Angular team took on the challenge to design a module system that would allow for the construction of a deterministic abstract syntax tree (AST). It contains the information needed to appropriately tree shake out unused code and determine what should actually be bundled. The industry has come a long way in a short amount of time, but there are still long standing problems to be resolved.

This of course leads us to the issue of this thread. The problem lies deeper than just at the framework level (Nest), or at the bundling level (Webpack). Therefore, we should be sympathetic to the fact that we may need some explicit configuration for the time being. The industry as a whole needs more time to mature. Until then, all of us require some proficiency with configuring bundling systems like Webpack.

[ghost]

In a lambda-context it would be really cool to use Nestjs, even for functions which do not expose a REST-API, but listen on different event-triggers...and for these lambdas we would only need @nestjs/common and @nestjs/core to create the nest-context via NestFactory.createApplicationContext(...);. But the bundling-process of these lambdas should be easier - without any further configuration. With webpack i can create this super-ugly configuration above. With parcel I do not even have the possbility to bundle a lambda - unless I add all these additional (unnecessary) dependencies above...Maybe someone can help?

[DmitryEfimenko]

@ZenSoftware Thanks for providing an example of bundling a Nest app. Is there a chance you could provide a similar example in the context of NX workspace? There's a bit of added complexity that has to do with specifying configuration via angular.json builders there that I don't quite understand yet.

[bradtaniguchi]

I created a PR to revert the change for now. I still suggest people look into bundling in their dependencies but I guess it's hard to provide something that works for NestJS out of the box.

@xmlking Yes, that is correct! I recommend the following setting instead:

"externalDependencies": [
     "@nestjs/microservices",
     "@nestjs/platform-express",
     "class-transformer",
     "class-validator",
     "cache-manager"
]

This approach worked for me, but I noticed my app failed when running with the following error:

[Nest] 8880   - 09/11/2019, 6:41 PM   [PackageLoader] No driver (HTTP) has been selected. In order to take advantage of the default driver, please, ensure to install the "@nestjs/platform-express" package ($ npm install @nestjs/platform-express).

I had to remove the @nestjs/platform-express package from the list as I needed it to be included in the bundle (duh hehe), but after that it worked. :)

Just wanted to point out the error I ran into in-case someone was trying to get this to work.

[ZenSoftware]

@DmitryEfimenko For a clean install of an Nx project, using the Angular + Nest schematic, most people seem to be using the "externalDependencies" configuration provided by the "builder": "@nrwl/node:build" in the angular.json file. The options are externalDependencies: 'all' | 'none' | string[];

I myself am having difficulties getting bundling working for our Nest app managed with Nx. The reason is due to the fact that we have 108 npm packages, and we're planning on adding more. Webpack is failing to cope with the complexity of the Javascript ecosystem. Ultimately, the lowest denominator is correctness. So we are excluding all of our dependencies and installing them in the Docker image instead. I spent the last 2 days trying to get bundling working. I couldn't produce reliable builds given the number of 3rd party libraries we are using. The builds seem to be missing modules here and there. At any rate, I've determined that the maintenance of the bundling process is not worth it. Ensuring correctness is more important, as to allow for the integration of new 3rd party libraries with agility. Bundling sort of becomes a bottleneck. Our Kubernetese pods are not as lightweight as they could possibly be, but computer science is full of trade offs... 😅

I published 💥 Nest 🔰 Webpack 🔰 Docker 💥 as a bare minimum example of what is involved in manually setting up a bundling system, without the complexities of many 3rd party libraries. It's actually not that much configuration. I think it is perfectly appropriate to invest in effort in getting bundling working. Stateless micro services are the first thing that come to mind. Small Docker image sizes are necessary for that purpose. You want to have minimal footprint to efficiently spin up replicas of Kubernetes pods to meet compute demands, and dispose of them efficiently.

[mledour]

I tried with externalDependencies but I've got this error:

class NestApplication extends nest_application_context_1.NestApplicationContext {
TypeError: Class extends value undefined is not a constructor or null
    at Module../node_modules/@nestjs/core/nest-application.js

"externalDependencies": [
     "@nestjs/microservices",
     "@nestjs/platform-express",
     "class-transformer",
     "class-validator",
     "cache-manager"
]

[vadistic]

I still have problem with @nest 6.8.2 using bare, cli-generated monorepo + webpack: true

Silly question: what kind of config you're taking about with those externalDependencies: []. I've always stayed on react side of things and I'm not not familliar with nx/ng. It's angular.json config file? Can I set it in nest-cli.json? I'm only asking coz googling quite failed me :)

The solution with ignore plugin works, but there are 2 additional submodule imports that need ignoring (@nestjs/microservices/microservices-module & @nestjs/websockets/socket-module)

Pasting for posterity :smile:

// webpack.config.js

module.exports = config => {

  const ignorePlugin = new webpack.IgnorePlugin({
    checkResource(resource) {
      const lazyImports = [
        '@nestjs/microservices',
        // ADD THIS
        '@nestjs/microservices/microservices-module',
        '@nestjs/websockets',
        // AND THIS
        '@nestjs/websockets/socket-module',
        '@nestjs/platform-express',
        'cache-manager',
        'class-validator',
        'class-transformer',
      ];

      if (!lazyImports.includes(resource)) {
        return false;
      }
      try {
        require.resolve(resource);
      } catch (err) {
        return true;
      }
      return false;
    },
  });

    config.plugins = config.plugins.concat(ignorePlugin);
    return config
}

[METACEO]

I'm hitting the same error message as @mledour with the below project environment. I've experienced little-to-no problems with my SSR Angular app and it's main.js bundle file but I'm hitting problems trying to effectively build my Nest app into something just as easy to deploy.

Any direction towards anything that works would be greatly appreciated!

$ yarn nx report
yarn run v1.21.1
$ nx report

>  NX  Report complete - copy this into the issue template

  @nrwl/angular : 8.9.0
  @nrwl/cli : 8.9.0
  @nrwl/cypress : 8.9.0
  @nrwl/eslint-plugin-nx : Not Found
  @nrwl/express : Not Found
  @nrwl/jest : 8.9.0
  @nrwl/linter : 8.9.0
  @nrwl/nest : 8.9.0
  @nrwl/next : Not Found
  @nrwl/node : 8.9.0
  @nrwl/react : Not Found
  @nrwl/schematics : Not Found
  @nrwl/tao : 8.9.0
  @nrwl/web : Not Found
  @nrwl/workspace : 8.9.0
  typescript : 3.5.3

Done in 0.98s.

[focux]

As @vadistic, I'm still running into this issue in a cli generated monorepo. For some reason, it works well when using nest build outside of Docker, but when I'm trying to build a Docker image and I run nest build, I get errors like this one:

ERROR in ../node_modules/@google-cloud/firestore/build/src/v1beta1/firestore_client.js
Module not found: Error: Can't resolve './firestore_client_config' in '/opt/node_modules/@google-cloud/firestore/build/src/v1beta1'

[kamilmysliwiec]

To recap and finally close this issue.

In many real-world scenarios (depending on what libraries are being used), you should not bundle Node.js applications (not only NestJS applications) with all dependencies (external packages located in the node_modules folder). Although this may make your docker images smaller (due to tree-shaking), somewhat reduce the memory consumption, slightly increase the bootstrap time (which is particularly useful in the serverless environments), it won't work in combination with many popular libraries commonly used in the ecosystem.

For instance, if you try to build NestJS (or just express) application with MongoDB, you will see the following error in your console:

Error: Cannot find module './drivers/node-mongodb-native/connection'
at webpackEmptyContext

Why? Because mongoose depends on mongodb which depends on kerberos (C++) and node-gyp.

Similarly, if you try to build an application which consists of Angular, Express, and Postgres (with, let's say, TypeORM as ORM), you will see this error:

ERROR in ./node_modules/pg/lib/native/client.js
Module not found: Error: Can't resolve 'pg-native'

Why? Because pg depends on pg-native which depends on node-libpq (low level C++ native bindings to PostgreSQL).

Lots of popular Node.js packages that are "required" to build a typical REST/GraphQL API depend on native bindings. Hence, you can't bundle them and produce a single executable JS file (which is easy for Front-End apps like Angular).

If, despite everything I wrote, you still want to bundle dependencies, AND YOU USE NX there are 2 ways:

• use plugin to ignore certain packages https://github.com/nestjs/nest/issues/1706#issuecomment-474514484
• use externalDependencies and define a list of packages that shouldn't be bundled (and bundle everything else) https://github.com/nestjs/nest/issues/1706#issuecomment-478145526

Nest CLI (when webpack mode is enabled "webpack": true) will ignore all node_modules dependencies by default. However, if despite everything I wrote, you still want to bundle dependencies, there is a one solution:

• use a custom webpack configuration and add a plugin to ignore certain packages https://github.com/nestjs/nest/issues/1706#issuecomment-540315239

If you're lucky enough (libraries that you depend on don't rely on native bindings) and you follow the above instructions, you should still be able to bundle your application.