@embroider/webpack building forever

[c0rydoras]

Problem

I can't serve or build any embroider projects anymore (embroider versions tested 2.x, 3.x), its stuck at ⠋ building... [@embroider/webpack]. I also tried different package managers (npm, pnpm, yarn) all resulting in the same state of ⠋ building... [@embroider/webpack] forever. This started after a system upgrade, since then i have tried these things:

• reinstall node, npm etc. 2x
• restarted my system 3x
• deleting all caches and node_modules 2x

System details

OS: archlinux NPM: 9.8.0 yarn: 1.22.19 pnpm: 8.6.7 node: 16, 18 and 20 ember-source: 4.12 and 5 ember-cli: 4.12 and 5 ember-data: 4.11, 4.12 and 5

[NullVoxPopuli]

Any chance you have a reproduction repo that demonstrates this problem?

[BobrImperator]

@NullVoxPopuli I happen to have one :) You can check out this ember-simple-auth branch https://github.com/mainmatter/ember-simple-auth/pull/2604 I'm currently migrating the addon to V2 and also wanted to add tests for embroider specifically, but it fails for both embroider scenarios I've created.

Running ESA is pretty straightforward:

• clone the repo
• pnpm install
• pnpm install && pnpm run --filter test-app test:one embroider-lts-4.12 or pnpm install && pnpm run --filter test-app test:one embroider-ember-5

Bear in mind that the embroider-lts-4.12 scenario defines an environment variable ESA_TEST_EMBROIDER that enables Embroider in the ember-cli-build.js.

It does seem to at least throw an error on CI https://github.com/mainmatter/ember-simple-auth/actions/runs/5562444913/jobs/10160768826?pr=2604

[BobrImperator]

I seem to have fixed this. Now it just throws and doesn't hang. Turns out that pnpm was using an incorrect node version 20.*, pinning to 18.16.1 fixed the hangup.

[c0rydoras]

@NullVoxPopuli to reproduce the problem simply create a new ember app with ember-cli and the --embroider flag, then try starting it with yarn start and it will hang at the building... [@embroider/webpack] forever

[BobrImperator]

@C0rydoras is it possible that you have installed nodejs and pnpm with a package manager like pacman or yay? If so pnpm could end up using a wrong version of nodejs.

In this case I'd recommend uninstalling pnpm and nodejs i.e. yay -R nodejs pnpm and install pnpm with an installation script instead https://pnpm.io/installation

Please see if there's any warning in your console at the very start.

[c0rydoras]

@BobrImperator yes, but the problem does not only occur when using pnpm but also when using yarn or npm,

also yes when using nodejs (newest version of node) I get this warning

WARNING: Ember CLI v4.12.1 is not tested against Node v20.4.0. See "https://github.com/ember-cli/ember-cli/blob/master/docs/node-support.md" to find out which version of Node is best to use.

I do normally use nodejs-lts-hydrogen (v18 lts)

[NullVoxPopuli]

to reproduce the problem simply

I ran

❯ npx ember-cli@latest new test-embroider --embroider --yarn

and got success

``` ❯ npx ember-cli@latest new test-embroider --embroider --yarn Need to install the following packages: ember-cli@5.1.0 Ok to proceed? (y) y installing app Ember CLI v5.1.0 ✨ Creating a new Ember app in /home/nullvoxpopuli/Development/tmp/test-embroider: create .editorconfig create .ember-cli create .eslintignore create .eslintrc.js create .github/workflows/ci.yml create .prettierignore create .prettierrc.js create .stylelintignore create .stylelintrc.js create .template-lintrc.js create .watchmanconfig create README.md create app/app.js create app/components/.gitkeep create app/controllers/.gitkeep create app/helpers/.gitkeep create app/index.html create app/models/.gitkeep create app/router.js create app/routes/.gitkeep create app/styles/app.css create app/templates/application.hbs create config/ember-cli-update.json create config/environment.js create config/optional-features.json create config/targets.js create ember-cli-build.js create .gitignore create package.json create public/robots.txt create testem.js create tests/helpers/index.js create tests/index.html create tests/integration/.gitkeep create tests/test-helper.js create tests/unit/.gitkeep 🚧 Installing packages... This might take a couple of minutes. yarn: Installed dependencies 🎥 Initializing git repository. Git: successfully initialized. 🎉 Successfully created project test-embroider. 👉 Get started by typing: $ cd test-embroider $ yarn start Happy coding! ~/Development/tmp took 1m2s ❯ cd test-embroider/ test-embroider on  main ❯ yarn start yarn run v1.22.19 $ ember serve Build successful (6517ms) – Serving on http://localhost:4200/ Slowest Nodes (totalTime >= 5%) | Total (avg) --------------------------------------------------------+-------------- @embroider/webpack (1) | 3898ms Babel: @ember/test-helpers (1) | 761ms ```

My environment info:

❯ ember -v
ember-cli: 5.1.0
node: 18.16.0
os: linux x64

❯ which node
/home/nullvoxpopuli/.volta/bin/node

❯ which yarn
/home/nullvoxpopuli/.volta/bin/yarn

❯ yarn --version
1.22.19

❯ node --version
v18.16.0

❯ pnpm --version
8.6.5

And with pnpm

``` ❯ EMBER_CLI_PNPM=true npx ember-cli@latest new test-embroider --embroider --pnpm installing app Ember CLI v5.1.0 ✨ Creating a new Ember app in /home/nullvoxpopuli/Development/tmp/test-embroider: create .editorconfig create .ember-cli create .eslintignore create .eslintrc.js create .github/workflows/ci.yml create .prettierignore create .prettierrc.js create .stylelintignore create .stylelintrc.js create .template-lintrc.js create .watchmanconfig create README.md create app/app.js create app/components/.gitkeep create app/controllers/.gitkeep create app/helpers/.gitkeep create app/index.html create app/models/.gitkeep create app/router.js create app/routes/.gitkeep create app/styles/app.css create app/templates/application.hbs create config/ember-cli-update.json create config/environment.js create config/optional-features.json create config/targets.js create ember-cli-build.js create .gitignore create package.json create public/robots.txt create testem.js create tests/helpers/index.js create tests/index.html create tests/integration/.gitkeep create tests/test-helper.js create tests/unit/.gitkeep 🚧 Installing packages... This might take a couple of minutes. pnpm: Installed dependencies 🎥 Initializing git repository. Git: successfully initialized. 🎉 Successfully created project test-embroider. 👉 Get started by typing: $ cd test-embroider $ pnpm start Happy coding! ~/Development/tmp took 13s ❯ cd test-embroider/ test-embroider on  main ❯ pnpm start > test-embroider@0.0.0 start /home/nullvoxpopuli/Development/tmp/test-embroider > ember serve Build successful (6763ms) – Serving on http://localhost:4200/ Slowest Nodes (totalTime >= 5%) | Total (avg) --------------------------------------------------------+-------------- @embroider/webpack (1) | 3948ms Babel: @ember/test-helpers (1) | 867ms ```

However, when I switch to Node 20, I experience the same hangup you reported. :tada:

[c0rydoras]

@NullVoxPopuli what linux distro?

[NullVoxPopuli]

❯ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.2 LTS
Release:    22.04
Codename:   jammy

❯ uname -a
Linux hephaestus 5.19.0-41-generic #42~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 18 17:40:00 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

[c0rydoras]

Installing the package managers & node with volta fixed the problem :D thank you

[void-mAlex]

it we actually have a problem on node 20 we should keep this issue open

[void-mAlex]

so been looking into this a bit and discovered something I really wasn't expecting setting JOBS=0 env var makes the build work on nodejs >= 20 added bonus is it's it takes the same time to compile (or less - your results may vary :) )

[NullVoxPopuli]

How I reproduced in a separate embroider app:

• clone: https://github.com/universal-ember/ember-primitives/
• pnpm i
• pnpm build
• cd docs-app
• volta pin node@20
• pnpm start (stuck on building)
• ctrl+c
• JOBS=0 pnpm start build completes

System info (native ubuntu)

❯ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 23.04
Release:    23.04
Codename:   lunar

❯ uname -a
Linux GhostLinux 6.2.0-27-generic #28-Ubuntu SMP PREEMPT_DYNAMIC Wed Jul 12 22:39:51 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

[ef4]

Confirmed that the above reproduction does not happen on Mac.

[void-mAlex]

reproduced with Ubuntu 22 running in WSL on windows 10

[SolPier]

reproduced with Debian 11:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 11 (bullseye)
Release:    11
Codename:   bullseye
$ node --version
v20.5.1

(not using 'volta' but 'n' for node versions)

[jakesjews]

I think the issue is caused by https://github.com/webpack-contrib/thread-loader/issues/191. Setting the environment variable UV_USE_IO_URING=0 fixes my builds

[mansona]

@void-mAlex @NullVoxPopuli can we confirm that @jakesjews's suggestion fixes ember-primitives too? I'm just trying to make sure that we are confident that this is the same issue 👍

[NullVoxPopuli]

Confirm, UV_USE_IO_URING=0 pnpm start resolves the issue for ember-primitives (after being configured for node 20)

[void-mAlex]

my next question is what is that variable and what does it do

[jakesjews]

Node 20.3.0 updated libuv which added support for using linux io_uring in the filesystem instead of the thread pool. Setting UV_USE_IO_URING=0 disables io_uring. I'm not sure if its a bug with the io_uring stuff or if the performance improvement exposed some sort of race condition in a library somewhere

[void-mAlex]

issue has now been reported on node 18.18 (and NOT happening on 18.17) -some of these are versions that GH actions run on

[void-mAlex]

I've confirmed this is resolved in node 20.8.0

[mansona]

It looks like Node 18.18.1 is working through release now too: https://github.com/nodejs/node/pull/50066

[ef4]

We can close this when 18.18.1 is released and we confirm that it's fixing this.

[void-mAlex]

closing as 18.18.1 is out with a fix for the issue and is available in github ci as well