search

embroider-build / embroider

Compiling Ember apps into spec-compliant, modern Javascript.
MIT License
330 stars 137 forks source link

`Build Error (PackagerRunner) in node_modules\@glimmer\tracking\index.js` #1132

Closed mkszepp closed 1 year ago

mkszepp commented 2 years ago

Ember: 4.1.0 Embroider 1.2.0 OS: Win 10

Sometime after file changes the build fails.

The only fix is to stop and restart ember serve. This bug cames randomly after x file changes

We use ember-cached-decorator-polyfill because @cached doesn't work with the latest embroider version

Maybe related: https://github.com/embroider-build/embroider/issues/1086#issuecomment-1044028028

[@embroider/webpack]assets by status 11.5 MiB [cached] 6 assets
Entrypoint assets/my-app.js [big] 9.96 MiB = chunk.a014068d97d15b716a73.js 6.45 MiB chunk.738f12f02191cd08114e.js 3.5 MiB chunk.a38afc09352e8ed55d0b.js 13.1 KiB
Entrypoint assets/test.js [big] 10.9 MiB = chunk.a014068d97d15b716a73.js 6.45 MiB chunk.2522794c3224ecc4d25b.js 341 KiB chunk.738f12f02191cd08114e.js 3.5 MiB chunk.e198f43b9a52d72b6f1e.js 636 KiB
cached modules 7.8 MiB (javascript) 14.4 KiB (runtime) [cached] 1831 modules
javascript modules 78 bytes
  ./node_modules/@glimmer/tracking/index.js 39 bytes [built] [1 error]
  ./node_modules/@glimmer/tracking/primitives/cache.js 39 bytes [built] [1 error]

ERROR in ./node_modules/@glimmer/tracking/index.js
Module build failed (from ../../../../../../../Projects/rg/my-app/node_modules/thread-loader/dist/cjs.js):
Thread Loader (Worker 0)
ENOENT: no such file or directory, open '$TMPDIR\embroider\a1864e\node_modules\@glimmer\tracking\index.js'
    at PoolWorker.fromErrorObj (C:\Projects\rg\my-app\node_modules\thread-loader\dist\WorkerPool.js:346:12)    at C:\Projects\rg\my-app\node_modules\thread-loader\dist\WorkerPool.js:219:29
    at mapSeries (C:\Projects\rg\my-app\node_modules\neo-async\async.js:3625:14)
    at PoolWorker.onWorkerMessage (C:\Projects\rg\my-app\node_modules\thread-loader\dist\WorkerPool.js:173:34)
    at C:\Projects\rg\my-app\node_modules\thread-loader\dist\WorkerPool.js:146:14
    at Socket.onChunk (C:\Projects\rg\my-app\node_modules\thread-loader\dist\readBuffer.js:40:9)
    at Socket.emit (events.js:400:28)
    at Socket.emit (domain.js:475:12)
    at Socket.Readable.read (internal/streams/readable.js:504:10)
    at Socket.read (net.js:638:39)
 @ ./models/relationships-tracker.js 11:0-44 401:85-92 408:80-87 415:97-104 422:90-97 429:98-105
 @ ./assets/my-app.js 608:13-59

ERROR in ./node_modules/@glimmer/tracking/primitives/cache.js
Module build failed (from ../../../../../../../Projects/rg/my-app/node_modules/thread-loader/dist/cjs.js):
Thread Loader (Worker 0)
ENOENT: no such file or directory, open '$TMPDIR\embroider\a1864e\node_modules\@glimmer\tracking\primitives\cache.js'
    at PoolWorker.fromErrorObj (C:\Projects\rg\my-app\node_modules\thread-loader\dist\WorkerPool.js:346:12)    at C:\Projects\rg\my-app\node_modules\thread-loader\dist\WorkerPool.js:219:29
    at mapSeries (C:\Projects\rg\my-app\node_modules\neo-async\async.js:3625:14)
    at PoolWorker.onWorkerMessage (C:\Projects\rg\my-app\node_modules\thread-loader\dist\WorkerPool.js:173:34)
    at C:\Projects\rg\my-app\node_modules\thread-loader\dist\WorkerPool.js:146:14
    at Socket.onChunk (C:\Projects\rg\my-app\node_modules\thread-loader\dist\readBuffer.js:40:9)
    at Socket.emit (events.js:400:28)
    at Socket.emit (domain.js:475:12)
    at Socket.Readable.read (internal/streams/readable.js:504:10)
    at Socket.read (net.js:638:39)
 @ ./node_modules/ember-cached-decorator-polyfill/index.js 1:0-75 17:44-55 18:11-19
 @ ./services/session-user.js 13:0-57 183:78-84 183:255-261
 @ ./assets/my-app.js 686:13-52

webpack 5.69.0 compiled with 2 errors in 1964 ms
Build Error (PackagerRunner) in node_modules\@glimmer\tracking\index.js

Module build failed (from ../../../../../../../Projects/rg/my-app/node_modules/thread-loader/dist/cjs.js):
Thread Loader (Worker 0)
ENOENT: no such file or directory, open '$TMPDIR\embroider\a1864e\node_modules\@glimmer\tracking\index.js\index.js'
    at PoolWorker.fromErrorObj (C:\Projects\rg\my-app\node_modules\thread-loader\dist\WorkerPool.js:346:12)    at C:\Projects\rg\my-app\node_modules\thread-loader\dist\WorkerPool.js:219:29
    at mapSeries (C:\Projects\rg\my-app\node_modules\neo-async\async.js:3625:14)
    at PoolWorker.onWorkerMessage (C:\Projects\rg\my-app\node_modules\thread-loader\dist\WorkerPool.js:173:34)
    at C:\Projects\rg\my-app\node_modules\thread-loader\dist\WorkerPool.js:146:14
    at Socket.onChunk (C:\Projects\rg\my-app\node_modules\thread-loader\dist\readBuffer.js:40:9)
    at Socket.emit (events.js:400:28)
    at Socket.emit (domain.js:475:12)
    at Socket.Readable.read (internal/streams/readable.js:504:10)
    at Socket.read (net.js:638:39)

Stack Trace and Error Report: C:\Users\markus\AppData\Local\Temp/error.dump.67b55f67c1510f8b804c19899ab04ea5.log

Looks like the path is wrong? ENOENT: no such file or directory, open '$TMPDIR\embroider\a1864e\node_modules\@glimmer\tracking\index.js\index.js'

NullVoxPopuli commented 2 years ago

do you have a reproduction readily available? I've successfully used @cached in v2 addons 🤔

mkszepp commented 2 years ago

@NullVoxPopuli here a simple repo with Ember 4.1, Embroider 1.2 & @cached which doesn't work. https://github.com/mkszepp/embroider-cached

With this repo there is possible to reproduce: https://github.com/embroider-build/embroider/issues/1086#issuecomment-1044028028

When i add ember-cached-decorator-polyfill in this repo, the @cached decorator works, but i think i get as side effect after x-rebuilds the error from this issue.

With Embroider <=1.0.0 & Ember 4.1, i have got no problems with the @cached decorator since now (works everything without problems, but the polyfill is necessary otherwise it doesn't work because the fix for @cached was added in embroider 1.2.0)

NullVoxPopuli commented 2 years ago

Embroider 1.2 & @cached which doesn't work. because the fix for @cached was added in embroider 1.2.0

these statements seem to be in conflict 🤔

Looking at the changelog: https://github.com/embroider-build/embroider/blob/main/CHANGELOG.md I don't see the fix you're talking about in 1.2, but do see it in 1.1: https://github.com/embroider-build/embroider/blob/main/CHANGELOG.md#v110-2022-02-08 (I also typo'd the PR title 😅 )

I was playing around with your reproduction -- want to show my debugging process:

  1. have sourcemaps turned on, because webpack output is not human-friendly. (I'm using devtool: 'source-map')

  2. search for glimmer/tracking in chrome (or firefox) image this shows that we have @glimmer/tracking as one of our AMD modules, so that's good -- we should be able to import it.

  3. as I scroll through the search results though, I see that something is pulling in the cached-decorator-polyfill image we don't want this.

  4. running npm list ember-cached-decorator-polyfill

    ❯ npm list ember-cached-decorator-polyfill
embroider-cached@0.0.0 <repo>
└─┬ ember-data@4.1.0
  ├─┬ @ember-data/model@4.1.0
  │ └── ember-cached-decorator-polyfill@0.1.4
  └─┬ @ember-data/store@4.1.0
    └── ember-cached-decorator-polyfill@0.1.4 deduped}

    so it's from ember-data, I figure, "for testing", let's see what happens when we remove ember-data so that we get rid of the polyfill

  5. after removing ember-data and confirming that the polyfill is gone via npm list ember-cached-decorator-polyfill, the problem persists -- so that means the decorator is not a function error has nothing to do with the polyfill.

  6. I wanted to check to make sure all your @embroider/* deps were the correct version

    ❯ cat node_modules/@embroider/addon-shim/package.json | jq '.version'
"0.50.2"

    oh! that's problematic! Let's try to force everything to be v1.2.0 I kinda got lucky with checking addon-shim first, but you can check everything at once with a glob:

    ❯ cat node_modules/@embroider/*/package.json | jq '.name,.version'
"@embroider/addon-shim"
"0.50.2"
"@embroider/babel-loader-8"
"1.2.0"
"@embroider/compat"
"1.2.0"
"@embroider/core"
"1.2.0"
"@embroider/hbs-loader"
"1.2.0"
"@embroider/macros"
"1.2.0"
"@embroider/shared-internals"
"1.2.0"
"@embroider/webpack"
"1.2.0"

    So, I already kinda cheated and pinned some of these already (due to other similarly reported issues), so my results may not match yours exactly. But, This is what you'd do:

    for npm (v8+):

      "overrides": {
    "@embroider/addon-dev": "^1.2.0",   
    "@embroider/compat": "^1.2.0",
    "@embroider/core": "^1.2.0",
    "@embroider/macros": "^1.2.0"
  },

    however, I ran in to this: https://github.com/npm/cli/issues/4232 so, I can't use npm, apparetly You could use older techniques with npm, if you're stuck on npm, with something like https://www.npmjs.com/package/force-resolutions

    buuuut, for yarn (v1):

      "resolutions": {
    "@embroider/addon-dev": "^1.2.0",   
    "@embroider/compat": "^1.2.0",
    "@embroider/core": "^1.2.0",
    "@embroider/macros": "^1.2.0"
  },

    buuuuut after installing yarn, I still have addon-dev 0.50.2, something else is fishy.

  7. Why do we have an old dependency potentially throwing things off?

    ❯ yarn why @embroider/addon-dev
=> Found "@embroider/addon-shim@0.50.2"
   - "ember-welcome-page" depends on it
   - Hoisted from "ember-welcome-page#@embroider#addon-shim"

    Let's remove the welcome-page. It's not meant to live in app passed the "introduction" anyway. (for ember veterans, it can be skipped with --no-welcome during app generation)

    However, I realize now that I've been confusing @embroider/addon-dev with @embroider/addon-shim 😩

  8. Go back to step 6, except specify addon-shim instead of addon-dev.... and forget about the famous X/Y problem. (As an aside, removing ember-welcome-page does fix the versioning issue, but it's not the correct fix).

    But now I get this error: 

    Build Error (PackagerRunner) in app.js

Module not found: Error: Can't resolve '<repo>/node_modules/@embroider/core/node_modules/@babel/runtime/helpers/esm/defineProperty.js' in '$TMPDIR/embroider/afd5f2/app.js'

    So, I need to clear /tmp/embroider. But that didn't work (same module not found error about babel/runtime). So I'm going to reset / revert all my work and start over with just the overrides/resolutions addition to package.json Success! I'm back to decorator is not a function. Now let's switch to yarn, as addon-shim is still 0.50.2 Switching to yarn brings back Can't resolve (babel runtime). So.. idk what that's output. I don't think yarn does nested node_modules? so 🤷 I switch to pnpm, but the decorator is not a function issue is still there.

    What's interesting is that pnpm, seems to be better and not leaking transitive dependencies to the root node_modules

    ❯ cat node_modules/@embroider/*/package.json | jq '.name,.version'
"@embroider/compat"
"1.2.0"
"@embroider/core"
"1.2.0"
"@embroider/webpack"
"1.2.0"

    I don't know what that means for runtime, 🤷

  9. This has gotten a bit ridiculous, I need to step back a bit. Looking again at the @glimmer/tracking module: image It's aliasing @ember/-internals/metal, so let's go look at that file image It's a big list of exports assigned to some _exports variable in this module. So, let's see if cached is anywhere in this file (it's pretty big -- I think it's all of the metal package).

    This is suspicious: image image but there are more results to look at -- this could just be some initialization step for assignment later.

    Ok, here is the decorator: image So.. it does exist, and is a function.

    So now I'm going to setup some break points in that getter in @glimmer/tracking, and see what value it's looking at. image These breakpoints were never hit.

    Placing a breakpoint in application.js (where @cached is used): reveals the following: image @glimmer/tracking doesn't have the @cached getter.... but why?! which @glimmer/tracking is it looking at? clicking the FunctionLocation reveals: image _which is the same file we checked earlier to see if cached was exported...

    So now I want to debug the compat package which has the code for the supposed @cached fix... Adding two debuggers, and launching ember s with the JavaScript Debug Terminal image

    but, I don't actually need to run the dubugger (yet), because I think I found the problem): image cached is not exported from this fake file.

    Editing that to: image Fixes the issue.

    And here is a PR: https://github.com/embroider-build/embroider/pull/1135

Thanks for reporting!!!

NullVoxPopuli commented 2 years ago

In hindsight, I probably should have started with the compat code. 🙃

nightire commented 2 years ago

I hope every PR could have a step-by-step debugging / developing process like what @NullVoxPopuli did here, wonderful!

mkszepp commented 2 years ago

@NullVoxPopuli thank you for this step by step debugging & PR 🙃. Wunderful

mkszepp commented 2 years ago

fix was released in embroider 1.3

mkszepp commented 2 years ago

bad news... now with embroider 1.4 @cached decorator works without polyfill, but the error from this issues was not solved 😢

the files is really missing in TMP folder... folder when there is the bug: grafik

folder when it works: grafik

NullVoxPopuli commented 2 years ago

did you delete /tmp/embroider?

mkszepp commented 2 years ago

@NullVoxPopuli i have now manually removed the /tmp/embroider folder & i will try it

mkszepp commented 2 years ago

@NullVoxPopuli after some changes i was running in the same error 😢

NullVoxPopuli commented 2 years ago

do you have a reproduction?

mkszepp commented 2 years ago

i will try to create a repository, which i can share you... but i think, its difficult...

mkszepp commented 2 years ago

@NullVoxPopuli with new repo there is not possible to reproduce, but i have something found out: This bug is not only, when i save a file & i have found a way how can i reproduce it in my repo.

I'm working on Windows 10 x64 with VSCode. Sometimes (not always) when i open a file in VSCode (from the VSCode tree), ember starts a rebuild (i think windows is changing a timestamp in this situation and a rebuild starts) When i open files from the tree, while he is building, this error appreas. Looks like a timeing problem or file lock while create/remove... The simplest way to reproducte was, to open the complete tree and clicking step by step all files in the tree.

My repo is already a little bit larger (+- 3 secounds for a rebuild).

Maybe you have a large repository, so you can try it to reproduct it with them, otherwise i must create a repo with x-components...

NullVoxPopuli commented 2 years ago

are you using WSL-2?

mkszepp commented 2 years ago

no, at the moment i'm working directly in windows with powershell, but there is planned to switch to WSL-2, because its faster and also recommended from ember

NullVoxPopuli commented 1 year ago

I think we can close this issue for now.

If anyone has a similar error, please open a new issue <3