@nrwl/node:node executor does not support importing ESM libs

[owen26]

Current Behavior

Error [ERR_REQUIRE_ESM]: require() of ES Module from not supported.

Expected Behavior

Npm libs bundled with ESM only module should be supported.

Steps to Reproduce

• Create a new NX workspace and generate a @nrwl/node application.
• Add an NPM package that is only bundled with ESM, e.g. yarn add @keycloak/keycloak-admin-client
• Import this package in main.ts
• Run nx serve
• Observe the error message listed in the current behaviour section.
• Run nx build
• Observe no errors, build is successful.

Failure Logs

Error [ERR_REQUIRE_ESM]: require() of ES Module from not supported.

Environment

Node : 16.15.0 OS : darwin arm64 yarn : 1.22.19

nx : 14.1.1 @nrwl/angular : Not Found @nrwl/cypress : 14.1.1 @nrwl/detox : Not Found @nrwl/devkit : 14.1.1 @nrwl/eslint-plugin-nx : 14.1.1 @nrwl/express : 14.1.9 @nrwl/jest : 14.1.1 @nrwl/js : 14.1.1 @nrwl/linter : 14.1.1 @nrwl/nest : Not Found @nrwl/next : Not Found @nrwl/node : 14.1.9 @nrwl/nx-cloud : Not Found @nrwl/nx-plugin : Not Found @nrwl/react : 14.1.1 @nrwl/react-native : Not Found @nrwl/schematics : Not Found @nrwl/storybook : 14.1.1 @nrwl/web : 14.1.1 @nrwl/workspace : 14.1.1 typescript : 4.6.4 rxjs : 6.6.7

Possibly related issue: https://github.com/nrwl/nx/issues/10296

[klerick]

+1, i have the same problem

[stomvi]

Check out #10565 and #10414. Waiting for good news.

[edsonprecoder]

I have the same import KcAdminClient from '@keycloak/keycloak-admin-client'; const kcAdminClient = new KcAdminClient( { baseUrl: process.env.KEYCLOAK_URL, realmName: process.env.REALM_NAME } );

Error [ERR_REQUIRE_ESM]: require() of ES Module C:\.....

[davidgabrichidze]

same issue: can't import nanoid 4.0, which has only ESM support. I'm using @nrwl/nest 14.1.7

[moatorres]

Same here. Right now my whole "Nx experience" feels like product owners went all crazy with plugins, "build anything" capabilities, and left behind the simplicity and truly greatness of nx: its tools for managing development directories.

[bilalshaikh42]

We are running into this same issue with nanoid 4.0 and other ESM only libraries. This is a major blocker for us!

[jmls]

any updates ? Hitting the same issue

[nicolabello]

Do we know if this will be fixed any time soon? Any timeline?

[BugGambit]

Bump

[Mrhoho]

Any updates?

[quaelin]

I just had to update one of my dependencies, which is now ESM-only. I have yet to find a way to make it work with nx, and I'm wondering if there is even a viable workaround for this?

[ChoSeoHwan]

I have same issue

[gaurang171]

I have same issue. I can use dynamic import but that requires await and so restructure of the code. Running project with esbuild bundler works but it has issue related to debugger. https://github.com/nrwl/nx/issues/14556 just in case anyone wants to use esbuild

[dylandjian]

Hello! Any update on this? Still can't find a way to make this work 😓

[sancelot]

It is difficult to ask support for this subject without providing a sample repository to work with. you should at least use typescript 4.7

I made a node application and used this import and it worked .

import KcAdminClient from '@keycloak/keycloak-admin-client'; const kcAdminClient = new KcAdminClient( { baseUrl: process.env.KEYCLOAK_URL, realmName: process.env.REALM_NAME } );

my environment : Node : 16.18.0 OS : win32 x64 npm : 9.2.0

nx : 15.6.3 @nrwl/angular : Not Found @nrwl/cypress : 15.6.3 @nrwl/detox : Not Found @nrwl/devkit : 15.6.3 @nrwl/esbuild : 15.6.3 @nrwl/eslint-plugin-nx : 15.6.3 @nrwl/expo : Not Found @nrwl/express : Not Found @nrwl/jest : 15.6.3 @nrwl/js : 15.6.3 @nrwl/linter : 15.6.3 @nrwl/nest : Not Found @nrwl/next : Not Found @nrwl/node : 15.6.3 @nrwl/nx-cloud : Not Found @nrwl/nx-plugin : Not Found @nrwl/react : 15.6.3 @nrwl/react-native : Not Found @nrwl/rollup : Not Found @nrwl/schematics : Not Found @nrwl/storybook : Not Found @nrwl/web : Not Found @nrwl/webpack : 15.6.3 @nrwl/workspace : 15.6.3 @nrwl/vite : 15.6.3 typescript : 4.8.4

Local workspace plugins:

Community plugins:

[alejandrombc]

Same here with nanoid version 4.0.1 :/, any news?

[sancelot]

Same here with nanoid version 4.0.1 :/, any news?

@alejandrombc nobody can solve your problem, if you don't provide sample code repository AND environment . As I wrote it works.

That depends on your project setup. something is missing or badly setted up .....

[owen26]

As I write this comment the issue still persists with NX 15.7.2 and TypeScript 4.9.5.

Same for both @keycloak/keycloak-admin-client@21.0.0 and nanoi@4.0.1.

Also out of curiosity, I tried the new serve executor @nrwl/js:node / build executor @nrwl/esbuild:esbuild and the result is the same.

I'm not sure @alejandrombc why you cannot reproduce the issue because there is nothing really custom setup. You get the error straight away with all default settings.

If I'm getting more feedback of not replicable I will consider spending some time to create a repo. Although I doubt how useful will that be since everything is default NX settings.

[shayank]

same issue with nanoid 4.0.2. I updated the whole dependencies (nx migrate --run-migrations) and still have the error.

[christopher-caldwell]

@sancelot I made a reproduction repo showing the issue (for me at least) https://github.com/christopher-caldwell/esm-require

[caioquirino]

Tested @christopher-caldwell against the new 16.0.0-beta.1 and still fails with the same error:

[ watch ] build succeeded, watching for changes...
Debugger listening on ws://localhost:9229/e5b6f3bd-c0db-4a6f-b2b6-766685931683
For help, see: https://nodejs.org/en/docs/inspector

/home/caio/github/christopher-caldwell/esm-require/node_modules/@nx/js/src/executors/node/node-with-require-overrides.js:16
        return originalLoader.apply(this, arguments);
                              ^
Error [ERR_REQUIRE_ESM]: require() of ES Module /home/caio/github/christopher-caldwell/esm-require/dist/packages/api/main.js from /home/caio/github/christopher-caldwell/esm-require/node_modules/@nx/js/src/executors/node/node-with-require-overrides.js not supported.
Instead change the require of main.js in /home/caio/github/christopher-caldwell/esm-require/node_modules/@nx/js/src/executors/node/node-with-require-overrides.js to a dynamic import() which is available in all CommonJS modules.
    at Module._load (/home/caio/github/christopher-caldwell/esm-require/node_modules/@nx/js/src/executors/node/node-with-require-overrides.js:16:31)
    at Object.<anonymous> (/home/caio/github/christopher-caldwell/esm-require/node_modules/@nx/js/src/executors/node/node-with-require-overrides.js:19:1)

[danielmahon]

My current solution for an esm module built from esbuild is to update node-with-require-overrides.js and use patch-package https://github.com/nrwl/nx/pull/10414#issuecomment-1369314024

// node-with-require-overrides.js line 19

// require(fileToRun);
import(fileToRun).catch((error) => {
  console.error('Error importing main module:', error);
});

[nicu-chiciuc]

Here's another repro with the steps taken from the offical docs for nx (without any changes + installing chalk): https://github.com/nicu-chiciuc/repro-nx-chalk

At this point, I'm thinking about switching from Nx to a different stack, since I've been stuck on these issues for several weeks.

[Kolahzary]

Any updates on this issue? Are there any workarounds to just make it work?

We're having the same issue with turkey-neighbourhoods@^3

• https://github.com/muratgozel/turkey-neighbourhoods/issues/9

[binaryartifex]

i have a barely instantiated nestjs app and a node library that exports a planetscale database connection. as soon as anything is imported from the node library i get this....

F:\...\product-suite\node_modules\.pnpm\@nx+js@16.5.2_@swc-node+register@1.4.2_@swc+core@1.3.69_nx@16.5.2_typescript@5.1.6\node_modules\@nx\js\src\executors\node\node-with-require-overrides.js:18
        return originalLoader.apply(this, arguments);
                              ^
Error [ERR_REQUIRE_ESM]: require() of ES Module F:\...\product-suite\node_modules\.pnpm\@planetscale+database@1.8.0\node_modules\@planetscale\database\dist\index.js from F:\...\haven-world-tours\product-suite\dist\apps\web-server\main.js not supported.
Instead change the require of index.js in F:\client-projects\haven-world-tours\product-suite\dist\apps\web-server\main.js to a dynamic import() which is available in all CommonJS modules.

just brought my whole initial startup deployment to a crashing halt...

[mattfysh]

@sancelot I made a reproduction repo showing the issue (for me at least) https://github.com/christopher-caldwell/esm-require

I was able to get this working by changing the build target options. use "format": ["cjs"] and "bundle": true This will produce CommonJS output in the /dist/packages/api directory

[dmastag]

@mattfysh where do you change this ?

[mattfysh]

@dmastag look for the project.json file in the subfolder, you'll find these options in the build target, e.g.

https://github.com/christopher-caldwell/esm-require/blob/255b3250325ab03616931d15c34115f73f300bfc/packages/api/project.json#L14-L15

[dmastag]

Thanks @mattfysh

Sadly this still doesn't work on my project when I tried it 😞

I am using the latest "@nx/node": "16.5.5" So that might be the problem

[christopher-caldwell]

Somehow, I was able to get it working with the newer executors. https://github.com/christopher-caldwell/esm-require/tree/working

[dmastag]

Interesting, I am using webtorrent and the new executioners sadly don't work for me. Even though I can see that #10414 is executed.

I put the whole empty project at https://github.com/dmastag/esm-error-webtorrent

I do see the require-override being called, but still its not working. Also tried to use dynamic import, but webtorrent is still not working

[Mydayyy]

Same issue with the ESM-only library "file-type". Bummed to see this is an issue for several years now.

[mattfysh]

@Mydayyy can you upload a repro to github and I'll take a look

[Mydayyy]

Hey,

sure. You can find it here: https://github.com/Mydayyy/nx-nestjs-esm/tree/master

Readme details the (fairly simple) steps to reproduce it. As far as I can see the issues stems from the fact that the import of "file-type" is transpiled to a require. I checked one of my other projects where I am using the same library and the import is transpiled differently, but I did not yet dig deeper.

The repo was created with the nx workspace creation with the nest preset

npx create-nx-workspace nx-nestjs-esm --preset=nest
..
npm install file-type

[mattfysh]

@Mydayyy this one has me beat 😮‍💨

I tried a bunch of different things to try and get the output to support dynamic imports, but nothing worked

• tsconfig.app.json: set module to "esnext"
• webpack.config.js: set output.environment.dynamicImport to true

Then updated the source code to this:

const { fileTypeFromFile } = await import(/* webpackIgnore: true */ 'file-type');

And yet even with these changes (and a few others I tried) something in the nx/nest/webpack/ts stack is stubbornly transpiling the dynamic import.

Is there any way you can take the nest starter template and replace webpack with esbuild, or does nest require the use of the webpack bundler?

[Mydayyy]

Yea, I was not able to find a solution either (except that ugly workaround which I commented out in my repro repo). If I ever find a solution here when circle back to the issue I'll keep you posted, for now I guess its the ugly workaround and waiting for better esm support.

Is there any way you can take the nest starter template and replace webpack with esbuild, or does nest require the use of the webpack bundler?

Its a bit more elaborate. I think its possible but requires a decent amount of custom configuration. Currently webpack is baked into the template and not configurable

[isunilmekala]

It got fixed by including "find-cache-dir": "^3.3.2" package into the project.

[woppa684]

It got fixed by including "find-cache-dir": "^3.3.2" package into the project.

Yeah, had the same thing today. More and more libraries will switch to ESM only and Nx will break more and more and ugly workarounds like this have to be created... Where can I find the roadmap regarding Nx and ESM? Our company has to invest more and more into keeping the stuff running.

[morgan-wild]

One more with noble/ed25519 which is described as ' Pure ESM, can be imported without transpilers'.

That is so sad because this library is perfect for my project...

Any news?

[beeman]

One more with noble/ed25519 which is described as ' Pure ESM, can be imported without transpilers'.

That is so sad because this library is perfect for my project...

Any news?

Have you tried the option described here? https://github.com/christopher-caldwell/esm-require/blob/255b3250325ab03616931d15c34115f73f300bfc/packages/api/project.json#L14-L15

I believe that's what made it work for me when using noble/ed25519.

[morgan-wild]

Hi @beeman

I tried just now, and I still have the same issue.

I noticied we don't use the same executor, maybe because I use NestJS.

[danielmahon]

@morgan-wild if you don't NEED webpack then use esbuild as you should just need to set the format option to esm, set createPackageJson and create a partial package.json with type=module. Make sure you update your tsconfigs to use esm as well. Last time I got esm to work with webpack I had to override the webpack config with ALOT of seemingly redundant options. And working with external libs was a pain. Also haven't checked in a few months but I don't think NestJS supports esm yet. So that might be a blocker for you unless there is a workaround there too.

(Replying on mobile, sorry if I'm missing additional context)

[morgan-wild]

Thank you for your reply. I hope NestJS will support it soon because I have a nice project made with it.

For my issue I finally used another library but not great as noble/ed25519.

[dmastag]

I got my webtorrent to finally work in nestjs

Its based on https://github.com/nrwl/nx/issues/18974#issue-1877876549 But had to be slightly changed

Sharing the Code, hopefully this can help someone :

import { Injectable, OnModuleInit } from '@nestjs/common';

@Injectable()
export class AppService implements OnModuleInit {
  private client: any;

  async onModuleInit(): Promise<void> {
    const { default: WebTorrent } = await (Function('return import("webtorrent")')() as Promise<typeof import('webtorrent')>);
    this.client = new WebTorrent({ tracker: false, maxConns: 3 });
  }

  getData(): { message: string } {
    return { message: 'Hello API' };
  }
}

Nodejs will spit out some warnings, but it works "for now"

[markabrahams]

I encountered this same problem when running my project that includes the trading-signal npm library (version 4.0.0):

/home/mark/myrepo/node_modules/@nx/js/src/executors/node/node-with-require-overrides.js:18
        return originalLoader.apply(this, arguments);
                              ^
Error [ERR_REQUIRE_ESM]: require() of ES Module /home/mark/myrepo/node_modules/trading-signals/dist/index.js from /home/mark/myrepo/dist/apps/myapp/main.js not supported.
Instead change the require of index.js in /home/mark/myrepo/dist/apps/myapp/main.js to a dynamic import() which is available in all CommonJS modules.

For reference my npx nx report is:

mark@mymachine:~/myrepo$ npx nx report
<...snip...>

   Node   : 20.5.1
   OS     : linux-x64
   npm    : 10.4.0

   nx                 : 17.3.1
   @nx/js             : 17.3.1
   @nx/jest           : 17.3.1
   @nx/linter         : 17.3.1
   @nx/eslint         : 17.3.1
   @nx/workspace      : 17.3.1
   @nx/devkit         : 17.3.1
   @nx/esbuild        : 17.3.1
   @nx/eslint-plugin  : 17.3.1
   @nx/node           : 17.3.1
   @nrwl/tao          : 17.3.1
   @nx/webpack        : 17.3.1
   typescript         : 5.3.3

The trading-signals library switched to ESM imports in version 4.0.0.

My searching found this thread and I attempted these solutions from it:

1. https://github.com/nrwl/nx/issues/11335#issuecomment-1518373862 from @danielmahon

The structure of my file node_modules/@nx/js/src/executors/node/node-with-require-overrides.js seemed different and so I couldn't make the same code substitution in this file, so this solution didn't work for my case.

2. https://github.com/nrwl/nx/issues/11335#issuecomment-1652139636 from @christopher-caldwell

I studied the repo for what might have made the working branch work, but couldn't reproduce the success! @christopher-caldwell - were you able to pinpoint the actual fix in your working branch?

3. https://github.com/nrwl/nx/issues/11335#issuecomment-1754091042 from @beeman

I tried to switch executor from webpack to esbuild, and "format" from "cjs" to "esm". I encountered a condition where local imports only worked with a ".js" file extension, and could not work out how to fix this, so I couldn't get this to work. (@danielmahon also mentioned this solution here: https://github.com/nrwl/nx/issues/11335#issuecomment-1756172475)

4. https://github.com/nrwl/nx/issues/11335#issuecomment-1764875067 from @dmastag

This looked NestJS-specific, and I was unsure if this translated to a more generalised solution (e.g. I don't use NestJS)

5. Rolling back to a previous version of the trading-signals npm (3.7.0) before its ESM refactor.

This produced a working solution, albeit a non-ideal one. Any solution that sacrifices potential module updates or fixes - particularly security fixes - is not ideal by any stretch!

So the time spent trying different solutions to this problem has probably been around 4 hours. I'm not sure if this would be a typical developer experience or not in this case?!?

So this raised for me some genuine questions about what seems to me to be a serious shortcoming in the Nx tooling such that ESM modules are not usable (this may be too strong a statement, but it fits my current experience - would be over the moon to be proven wrong though if I'm missing an obvious solution!). And this has been a documented and unfixed problem for around 18 months now. Although there is a healthy interest in this thread, there doesn't seem to be an urgency to fix.

(a) Are there not many ESM libraries out there, so that this is not hitting many developers yet and therefore not that important yet? (despite a general consensus that ESM-style imports are preferred over CommonJS-style going forward) (b) Are developers hitting these problems and rolling back their third-party library versions to pre-ESM versions? (like I did! That seems bad though!) (c) Are developers finding other workarounds, or getting some of the above solutions (or other solutions) to this working?

And none of this is a slight on Nx developers - I see 722 open issues as I type this, so hands will be very full out there, prioritisation will obviously be a necessity, and further I'm unfortunately in no position to understand let alone develop a potential PR for this! Just trying to get a bit of mental clarity on the state of play here!

Sorry for the long write-up - congratulations if you made it to the end! :-)

[beeman]

So this raised for me some genuine questions about what seems to me to be a serious shortcoming in the Nx tooling such that ESM modules are not usable (

I think this might be a bit of a too-simple view of things, I see this as an issue with the ecosystem at large.

In my case, I use my Nx to build APIs that use Nest.js, and that framework doesn't support it. The reason: fastify also doesn't support it because it's immature.

So there's a chain of dependencies that make this an issue. Nx merely reflects this.

I managed to get ESM working using the third fix in your list but that was not a Nest.js project. So it can work, it depends on what you want. Not a lot that Nx can do here imo.

[markabrahams]

Thanks @beeman for your insight - it does help to get this perspective on things and is much appreciated!

A few thoughts:

I think this might be a bit of a too-simple view of things

Yes, my view of things might be simple and simply off-base even?!? Hence my line of questioning :-)

To me, this Nx error:

Error [ERR_REQUIRE_ESM]: require() of ES Module

suggests that Nx code (@nx/js/src/executors/node/node-with-require-overrides.js) is erroneously trying to require an ESM module (it even tells you that!) instead doing the correct thing and importing it.

This is nothing to do with Nest.js per se, or really any specific framework or library. It's to do with the general case of libraries that export ESM-style. Specifically that Nx code isn't correctly importing libraries that export ESM-style. That seems to be an Nx bug!

it depends on what you want

My wants are quite simple. I want: (1) An Nx setup where a import statements in my source code work for third party libraries that use CommonJS exports and for those that use ESM exports. (2) I don't want unexpected surprises like this error (it was an unexpected and unwanted surprise!) I don't even care about my transpiled output format, as long as it runs correctly and I can get source-map debugging going with it. (3) And I want to minimise the amount of time required to get to such an Nx setup - for me and for developers everywhere! :-)

Not a lot that Nx can do here

Here's where I'd have a slightly different take on it. A healthy number of developers have contributed text or emoji-love to comments in this thread, suggesting there is at least a reasonable interest in this topic. The Nx developers (hi!) at the very least could: (a) clarify their position on this issue (e.g. Is it an Nx bug? Is it a configuration problem? Is it due to a bug/shortcoming in an Nx dependency like esbuild? etc.), and (b) also give developers a steer on a solution.

I feel like I'm floundering a bit here, and I'm wondering if others are as well?

[binaryartifex]

I've got a ground-breaking idea, instead of NX constantly bring out more and more new feature's, maybe start triaging issues that are older than 12 months and FIX THEM FIRST. You can add all the new hotness you want, but what bloody good is it when folks get legitimately blocked on issues like this? ill show myself out...

[joeflateau]

I've landed on this method of importing esm libs:

export const dynamicImport = new Function(
  'specifier',
  'return import(specifier)'
) as <T = never>(specifier: string) => Promise<T>;

usage:

await dynamicImport<typeof import('thumbhash')>('thumbhash');

would be cool to not have to do so, but it works for now

it's also how nx does it internally: https://github.com/nrwl/nx/blob/7a7cbeca44e44ff62ae3ab120bccb0850e09884c/packages/js/src/executors/node/node-with-require-overrides.ts#L5

[danielsharvey]

Interested in peoples thoughts. I've run across issues which seem similar to the above.

My specific issue relating to loading buildable libraries. My observation is that the @nx/js:tsc executor creates temporary tsconfig.json files to reference the dist/** folders (built libraries) of referenced buildable libraries that are dependencies when running tsc on the requested project.

Example temporary tsconfig.json:

  ...
  "compilerOptions": {
    "paths": {
      ...
      "@my-project/cli-lib": [
        "dist/libs/cli-lib" // <-- added by Nx to reference built library
      ]
    }
  },

What it looks like Nx does is traverse the build task graph to collate the dependency references needed for this process.

There is node_modules/@nx/js/src/executors/node/node-with-require-overrides.js which is similar (though I've had troubles with it) but this does not handle ESM.

I've put together an approach which uses a custom executor node-executor which:

1. Uses the build task graph (via buildTarget similar to @nx/js:node) to extract these dependencies to the built libraries (typically in dist/),

2. Configure a custom Node Module loader (see https://nodejs.org/api/module.html#hooks) which supports CommonJS and ESM module resolution for imports.

3. Run the selected project using the custom loader.

I'm interested in peoples thoughts and happy to share my executor if it would be useful.

Note that the way dependencies are gathered has some sharp edges and is changing - see NX_BUILDABLE_LIBRARIES_TASK_GRAPH which seems to the intended path forward (see here, here and here).