search

nrwl / nx

Smart Monorepos · Fast CI
https://nx.dev
MIT License
23.81k stars 2.38k forks source link

@nrwl/node application not transpiling to esm #10296

Open JonNode28 opened 2 years ago

JonNode28 commented 2 years ago

Current Behavior

@nrwl/node application does not transpile to esm so esm-only packages (like node-fetch) fail.

Expected Behavior

Should transpile to esm

Steps to Reproduce

  1. npx create-nx-workspace
  2. Select "app"
  3. yarn add -D @nrwl/node
  4. nx g @nrwl/node:application esm-app
  5. yarn add node-fetch (an esm only package)
  6. import node-fetch in to apps/esm-app/src/main.ts
  7. change "module" in apps/esm-app/tsconfig.app.json to 'esnext'
  8. change "target" compiler option in tsconfig.base.json to 'esnext'
  9. yarn build
  10. dist/apps/esm-app/main.js should not use require() but does

main.js contains the following at the top

/******/    var __webpack_modules__ = ({

/***/ "node-fetch":
/***/ ((module) => {

module.exports = require("node-fetch");

/***/ })

/******/    });

Repro: https://github.com/JonNode28/nx-esm-repro

Failure Logs

Running yarn start produces the following error:

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /Users/xxx/Projects/tmp/nx-esm-repro/node_modules/node-fetch/src/index.js
require() of ES modules is not supported.
require() of /Users/xxx/Projects/tmp/nx-esm-repro/node_modules/node-fetch/src/index.js from /Users/xxx/Projects/tmp/nx-esm-repro/dist/apps/esm-app/main.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /Users/xxx/Projects/tmp/nx-esm-repro/node_modules/node-fetch/package.json.

    at new NodeError (internal/errors.js:322:7)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1102:13)
    at Module.load (internal/modules/cjs/loader.js:950:32)
    at Function.Module._load (internal/modules/cjs/loader.js:790:12)
    at Function.Module._load (/Users/xxx/Projects/tmp/nx-esm-repro/packages/node/src/executors/node/node-with-require-overrides.ts:16:27)
    at Module.require (internal/modules/cjs/loader.js:974:19)
    at require (internal/modules/cjs/helpers.js:101:18)
    at Object.node-fetch (/Users/xxx/Projects/tmp/nx-esm-repro/dist/apps/esm-app/webpack:/nx-esm-repro/external commonjs "node-fetch":1:1)
    at __webpack_require__ (/Users/xxx/Projects/tmp/nx-esm-repro/dist/apps/esm-app/webpack:/nx-esm-repro/webpack/bootstrap:19:1)
    at /Users/xxx/Projects/tmp/nx-esm-repro/dist/apps/esm-app/main.js:84:68
jrfornes commented 2 years ago

I had a similar error after running nx run nxt-e2e:e2e --verbose after migrating to 13.2.X

Must use import to load ES Module: /Users/some-project/node_modules/@angular/compiler-cli/bundles/index.js
require() of ES modules is not supported.
require() of /Users/some-project/node_modules/@angular/compiler-cli/bundles/index.js from /Users/some-project/node_modules/**@angular-builders/custom-webpack**/node_modules/@angular-devkit/build-angular/src/webpack/configs/common.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /Users/some-project/node_modules/@angular/compiler-cli/package.json.

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /Users/some-project/node_modules/@angular/compiler-cli/bundles/index.js

I was missing upgrading one of my project dependencies (non-controlled by NX) to an Angular 13 supportable version. I fixed the error by upgrading the package.

sfsr12 commented 2 years ago

Have the same issue trying to use normalize-url

Seems to be the same issue as: #7872

github-actions[bot] commented 2 years ago

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs. If we missed this issue please reply to keep it active. Thanks for being a part of the Nx community! 🙏

itPiligrim commented 2 years ago

Up🔝

gaurang171 commented 1 year ago

Same for me

keyp-dev commented 1 year ago

Same for me

caioquirino commented 1 year ago

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs. If we missed this issue please reply to keep it active. Thanks for being a part of the Nx community! 🙏

Please annoy us via e-mail to keep it active haha

In this case:

Same for me +1

radiovisual commented 1 year ago

Same thing is happening to me. Sending unicorn emojis to keep the bot from marking this issue as stale: 🦄🦄🦄🦄

gaurang171 commented 1 year ago

does any one knows what was the last working version? may be its better to downgrade

suneox commented 1 year ago

Up🔝

aitorllj93 commented 1 year ago

tried running the webpack builder with target: "es2021" but I got this error:

NX Property 'target' does not match the schema. 'es2021' should be one of node,web.

m3bwc commented 1 year ago

Up

DaSchTour commented 1 year ago

Instead of sending comments with up vote on the issue with 👍🏼🙄

caioquirino commented 1 year ago

Instead of sending comments with up vote on the issue with 👍🏼🙄

@DaSchTour does that work for this repo?

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs. If we missed this issue please reply to keep it active. Thanks for being a part of the Nx community! 🙏

timur-svoboda commented 1 year ago

Same for me

nidomiro commented 1 year ago

I'm currently in the exploration phase of migration nestjs-apps in a nx monorepo to esm, and here are my findings so far:

my webpack-config:

const { composePlugins, withNx } = require('@nrwl/webpack');
const util = require('util');
const nodeExternals = require('webpack-node-externals');

// Nx plugins for webpack.
const configFunc = composePlugins(withNx(), (config) => {
  // Update the webpack config as needed here.
  // e.g. `config.plugins.push(new MyPlugin())`
  config.resolve.extensionAlias = {
    ...config.resolve.extensionAlias,
    '.js': ['.ts', '.js'],
    '.mjs': ['.mts', '.mjs'],
  };
  return config;
});

const configFuncWithOverrides = (config, ctx) => {
  return configFunc(config, ctx).then((config) => {

    /*
    Changes here copied from https://github.com/nrwl/nx/issues/7872#issuecomment-997460397
    */
    config.externalsPresets = {
      node: true,
    };

    config.output = {
      ...config.output,
      module: true,
      libraryTarget: 'module',
      chunkFormat: 'module',
      library: {
        type: 'module',
      },
      environment: {
        module: true,
      },
    };

    config.experiments = {
      ...config.experiments,
      outputModule: true,
    };

    config.externals = nodeExternals({
      importType: 'module',
    });

    return config;
  });
};

module.exports = configFuncWithOverrides;
nidomiro commented 1 year ago

Ok, the error I had had nothing to do with the config above, it works. The error was caused because I had a composite: true inside my ts-config somehow.

Building the app and running it works, but serve does not right now.

nidomiro commented 1 year ago

for serve the problem ultimately lies here, as node-with-require-overrides is only executable in commonJS: https://github.com/nrwl/nx/blob/master/packages/js/src/executors/node/node.impl.ts#LL137C47-L137C74

You can find a working esm version with nestJS and therefore webpack here: https://github.com/nidomiro/nx-commonjs-esm-migration/tree/main/apps/nest-esm. The key changes are:

ottobolyos commented 1 year ago

Thanks, @nidomiro! :pray: I did everything from your list of changes, except for patching @nrwl/js and in my case it works as expected, although I use @nx/nest@16.2.1 with webpack@5.83.1, @nestjs/core@9.1.1.

xendarboh commented 1 year ago

Working to get my entire project to esm, almost there...

I have an app generated with nx g @nx/node:app --framework=express that failed to :serve (:build worked) with errors that led me to this issue. The errors began when an imported lib (within the same nx workspace) changed from cjs to esm. Tried many config tweaks, nada.

Since it was a simple express app, I checked to see if a generated @nx/express:app worked without the issue, and it did. In my context, applying the difference to the existing project was a matter of changing the build executor from using @nx/esbuild to @nx/webpack. Now it builds and serves.

npm i --save-dev @nx/webpack

Change build target in apps/myapp/project.json:

    "build": {
      "executor": "@nx/webpack:webpack",
      "outputs": ["{options.outputPath}"],
      "defaultConfiguration": "production",
      "options": {
        "target": "node",
        "compiler": "tsc",
        "outputPath": "dist/apps/myapp",
        "main": "apps/myapp/src/main.ts",
        "tsConfig": "apps/myapp/tsconfig.app.json",
        "assets": ["apps/myapp/src/assets"],
        "isolatedConfig": true,
        "webpackConfig": "apps/myapp/webpack.config.js"
      },
      "configurations": {
        "development": {},
        "production": {}
      }
    },

Add apps/myapp/webpack.config.js:

const { composePlugins, withNx } = require('@nx/webpack');

// Nx plugins for webpack.
module.exports = composePlugins(withNx(), (config) => {
  // Update the webpack config as needed here.
  // e.g. `config.plugins.push(new MyPlugin())`
  return config;
});

I thought all was fine then very strangely(!) the error was back, (maybe from a deeper cleaning process, but no code changes). This time, hopefully finally, it was fixed by adjusting the order of imports to put the lib (from the nx workspace) after external packages from node_modules.

 import express from 'express';
-import { payloadFromBase58 } from '@myworkspace/mylib';
 import { Encoding, Field, Poseidon, PrivateKey, Signature } from 'snarkyjs';
+import { payloadFromBase58 } from '@myworkspace/mylib';
Kolahzary commented 1 year ago

Same issue here:

namdien177 commented 1 year ago

for serve the problem ultimately lies here, as node-with-require-overrides is only executable in commonJS: master/packages/js/src/executors/node/node.impl.ts#LL137C47-L137C74

This is still an issue even in the new version 16.5.0.

After playing with the tsconfig and executor of nx a bit, I think this is the main issue. Until the node-with-require-overrides support both CJS and ESM, it's painful to use ESM packages in Express app.

laneme commented 1 year ago

No status yet?

patryksmolarz commented 1 year ago

About node-fetch.

Hello! Maybe it's a little workaround but at this moment I don't have any other idea how to solve that. I had the same problem with node-fetch and ESModules. I'm using NestJS with Next. Three apps as monorepo and I migrate to @nrwl > @nx (16.8.1) packages. Solution is to use node-fetch2 instead of node-fetch But step by step:

  1. tsconfig.base.json:

    {
"compileOnSave": false,
"compilerOptions": {
"rootDir": ".",
"sourceMap": true,
"declaration": false,
"moduleResolution": "node",
"emitDecoratorMetadata": true,
"experimentalDecorators": true,
"importHelpers": true,
"target": "es2015",
"module": "esnext",
"lib": ["es2020", "dom"],
"skipLibCheck": true,
"skipDefaultLibCheck": true,
"baseUrl": ".",
"paths": {
  "@morphais/my-new-lib": ["libs/my-new-lib/src/index.ts"],
  "@morphais/shared/domain": ["libs/shared/domain/src/index.ts"]
}
},
"exclude": ["node_modules", "tmp"]
}

  2. webpack.config.js of my nestjs app

    
const { composePlugins, withNx } = require('@nx/webpack');

// Nx plugins for webpack. module.exports = composePlugins(withNx(), (config) => { // Update the webpack config as needed here. // e.g. config.plugins.push(new MyPlugin()) return config; });


3. tsconfig.json of my nestjs app

{ "extends": "../../tsconfig.base.json", "files": [], "include": [], "references": [ { "path": "./tsconfig.app.json" }, { "path": "./tsconfig.spec.json" } ], "compilerOptions": { "esModuleInterop": true } }

4. tsconfig.app.json of my nestjs app

{ "extends": "./tsconfig.json", "compilerOptions": { "outDir": "../../dist/out-tsc", "module": "commonjs", "types": ["node"], "emitDecoratorMetadata": true, "target": "es2021" }, "exclude": ["jest.config.ts", "src//*.spec.ts", "src/*/.test.ts"], "include": ["src//*.ts"] }

5. project.json of my nestjs app

{ "name": "backend", "$schema": "../../node_modules/nx/schemas/project-schema.json", "sourceRoot": "apps/backend/src", "projectType": "application", "targets": { "build": { "executor": "@nx/webpack:webpack", "outputs": ["{options.outputPath}"], "defaultConfiguration": "production", "options": { "target": "node", "compiler": "tsc", "outputPath": "dist/apps/backend", "main": "apps/backend/src/main.ts", "tsConfig": "apps/backend/tsconfig.app.json", "assets": ["apps/backend/src/assets"], "isolatedConfig": true, "webpackConfig": "apps/backend/webpack.config.js" }, "configurations": { "production": {} } }, "serve": { "executor": "@nrwl/js:node", "defaultConfiguration": "development", "options": { "buildTarget": "backend:build" }, "configurations": { "development": { "buildTarget": "backend:build:development" }, "production": { "buildTarget": "backend:build:production" } } }, "lint": { "executor": "@nx/linter:eslint", "outputs": ["{options.outputFile}"], "options": { "lintFilePatterns": ["apps/backend/*/.ts"] } }, "test": { "executor": "@nx/jest:jest", "outputs": ["{workspaceRoot}/coverage/apps/backend"], "options": { "jestConfig": "apps/backend/jest.config.js", "passWithNoTests": true } } }, "tags": [] }



As you can see nothing special with ts files and webpack. When I started to use `esModuleInterop` (because I hadn't this before) then I had to change all imports, for example dayjs `import * as dayjs from 'dayjs'` to `import dayjs from dayjs`. Error still occurred, but then I used node-fetch2 and then all start work. I hope this helps you.
fauzulkc commented 1 year ago

for serve the problem ultimately lies here, as node-with-require-overrides is only executable in commonJS: https://github.com/nrwl/nx/blob/master/packages/js/src/executors/node/node.impl.ts#LL137C47-L137C74

You can find a working esm version with nestJS and therefore webpack here: https://github.com/nidomiro/nx-commonjs-esm-migration/tree/main/apps/nest-esm. The key changes are:

This is working. 👏

github-actions[bot] commented 1 month ago

This issue has been automatically marked as stale because it hasn't had any activity for 6 months. Many things may have changed within this time. The issue may have already been fixed or it may not be relevant anymore. If at this point, this is still an issue, please respond with updated information. It will be closed in 21 days if no further activity occurs. Thanks for being a part of the Nx community! 🙏

nidomiro commented 1 month ago

not stale, even if ignored😉