In node with esbuild and esm, the output results do not include the libs

wizardnet972 commented 1 year ago

Current Behavior

I create a nodejs app with esbuild and esm.

In the project, I have imports from libs.

When nx builds the node project it not add the mapping code to the output files (_resolveFilename) or don't add them to package.json.

The results are the libs is missing from the node runtime.

Expected Behavior

like cjs format, should be handle the imports from the libs.

GitHub Repo

No response

Steps to Reproduce

create workspace
create node app with esbuild
change the format to esm

Nx Report

yarn nx report
yarn run v1.22.19
>  NX  Falling back to ts-node for local typescript execution. This may be a little slower.
  - To fix this, ensure @swc-node/register and @swc/core have been installed

 >  NX   Report complete - copy this into the issue template

   Node   : 18.13.0
   OS     : darwin arm64
   yarn   : 1.22.19
   Hasher : Native

   nx                 : 16.2.1
   @nx/js             : 16.2.1
   @nx/jest           : 16.2.1
   @nx/linter         : 16.2.1
   @nx/workspace      : 16.2.1
   @nx/devkit         : 16.2.1
   @nx/esbuild        : 16.2.1
   @nx/eslint-plugin  : 16.2.1
   @nx/node           : 16.2.1
   @nrwl/tao          : 16.2.1
   @nx/vite           : 16.2.1
   typescript         : 5.0.4
   ---------------------------------------
   Community plugins:
   @nx/rspack : 16.1.2

✨  Done in 1.10s.



### Failure Logs

_No response_

### Operating System

- [X] macOS
- [ ] Linux
- [ ] Windows
- [ ] Other (Please specify)

### Additional Information

_No response_

wizardnet972 commented 1 year ago

Another issue related to this, when I create @nx/js lib with esbuild, nx compile by default to esm and because the node is compile with cjs it can't import this lib. if I change the node app to work as esm then back to the main issue - the lib can't be found because esm doesn't handle the libs inside the project. also it not wrap them with __toEsm.

halfbakedsneed commented 1 year ago

Hitting this same issue. Existed in 15.x too. Seems like there's a special case for cjs that handles lib imports. I assume there's a reason it's not enabled for esm?:

https://github.com/nrwl/nx/blob/master/packages/esbuild/src/executors/esbuild/lib/build-esbuild-options.ts#L69

samratarmas commented 1 year ago

Another issue related to this, when I create @nx/js lib with esbuild, nx compile by default to esm and because the node is compile with cjs it can't import this lib. if I change the node app to work as esm then back to the main issue - the lib can't be found because esm doesn't handle the libs inside the project. also it not wrap them with __toEsm.

Currently you cannot import a @nx/js library inside a @nx/node app. You can verify this on an empty workspace by creating a node app and a js library both with esbuild as bundler and try to share code between them. @AgentEnder @jaysoo @FrozenPandaz @vsavkin

firxworx commented 1 year ago

@samratarmas and others with related problems:

I have am using shared @nx/js libs in my nx workspace right now in both vite/react front-end (esm) and node/express back-end (cjs) apps. The trick: build both cjs + esm so everything is happy.

I would prefer all esm but as you have discovered node generator and nx serve (among others...) breaks with esm and nx has not addressed these issues to date.

My solution is as follows:

context: all of my projects libs are buildable my "build" target in project.json uses the @nx/esbuild:esbuild executor

in package.json of your lib strip module etc so that instead of you overriding his value it will be generated during build
- the package.json of my shared libs only specify name, version, and sideEffects
in project.json under targets.build.options
- set "generatePackageJson": true
- set "format": ["esm", "cjs"]

Note how project.json can override your tsconfig. You can even go all-out and specify custom options direct to esbuild by adding a targets.build.options.esbuildOptions object that take absolute precedence.

A potentially useful capability with esbuildOptions that might help depending on your project/deploy requirements is the ability to set outExtension in case you need .mjs (otherwise Nx will choose js for esm and cjs for cjs).

Troubleshooting:

review project.json in its entirety and ensure that you have no other configurations that will override the above (and if you do, modify them accordingly).
do a fresh build then review your dist/ output to make sure entrypoint files is being created for each of cjs and esm and that the generated package.json has identified the correct ones for module and main
make sure your consuming apps are clear about "commonjs" or "module" in their respective configs

I hope this helps you, it took some time to land on this solution. I couldn't find anything helpful in the docs.

IMHO shared libraries across front-end and back-end are a key reason to use Nx and I think it should Just Work^TM out of the box

hugonteifeh commented 1 year ago

Obviously this is not how it should be, but setting bundle to true in the esbuild config in project.json seems to work for now.

samratarmas commented 1 year ago

As mentioned in previous comments, the support for ESM in @nx/node is not there, however if you tell esbuild to bundle everything to one file then your compiled code is all in one entry main.js file so there are no ESM imports to worry about.

firxworx commented 1 year ago

@hugonteifeh and @samratarmas I agree eliminating imports entirely with bundle: true is a logical and valid solution to eliminating import problems... I have an important caveat to share regarding tree shaking.

One of the key reasons why I chose to build both cjs + esm for the shared @nx/js libraries in my project and why I suggested it above was to preserve the ability tof downstream bundlers to effectively optimize builds.

In terms of cutting down my React bundle sizes, the combination of declaring sideEffects in package.json (ideally false if you can help it) and providing ESM for downstream bundlers proved effective in my Nx project (including with the stock configuration of Vite+React).

Otherwise beware of unused/dead code from shared libraries ending up in your front-end builds. You could end up with an entire library/package's code (and potentially all of its dependencies! imagine all of @faker-js/faker or something) mashed into your bundle even if you only imported a tiny few-lines-of-code helper function from a shared utility library.

Size wasn't a huge concern for me on the back-end (though it may be for some) however on the front and it caused some big red flags with absurd bundle sizes in React apps until I addressed it.

(if anyone from nrwl/nx is reading this: please support using ESM everywhere)

chmoder commented 1 year ago

From a brand new project (node app + JS lib), if I build my depended libs then run serve it works. The other solution is to set the field bundle to true in my app project.json.

npx nx run my-js-lib:build --skip-nx-cache=true
npx nx run my-node-svc:serve

Is there a way to build all the depended libs when running serve?

github-actions[bot] commented 6 months ago

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs. If we missed this issue please reply to keep it active. Thanks for being a part of the Nx community! 🙏

Tiedye commented 5 months ago

Is this solved now? Or is the workaround all that is available

github-actions[bot] commented 4 months ago

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.

nrwl / nx