search

embroider-build / embroider

Compiling Ember apps into spec-compliant, modern Javascript.
MIT License
334 stars 136 forks source link

Can’t find in-repo addon on subsequent builds #768

Open sandydoo opened 3 years ago

sandydoo commented 3 years ago

I’ve got an addon with an in-repo addon that crashes on subsequent builds. The initial build goes well, but if you modify any file and cause a rebuild, it crashes.

The entire lib folder disappears from the temporary working directory after the rebuild, so, naturally, it crashes whenever anything in the pipeline tries to read the in-repo component file.

And it only happens in an addon. I couldn’t replicate this in an app.

ERROR Summary:

  - broccoliBuilderErrorStack: ModuleBuildError: Module build failed (from /Users/Sander/Programming/embroider-in-repo-repro/node_modules/thread-loader/dist/cjs.js):
Thread Loader (Worker 0)
ENOENT: no such file or directory, open '$TMPDIR/embroider/213e4c/lib/in-repo-addon/components/in-repo-component.js/in-repo-component.js'

    at /Users/Sander/Programming/embroider-in-repo-repro/node_modules/webpack/lib/NormalModule.js:316:20
    at /Users/Sander/Programming/embroider-in-repo-repro/node_modules/loader-runner/lib/LoaderRunner.js:367:11
    at /Users/Sander/Programming/embroider-in-repo-repro/node_modules/loader-runner/lib/LoaderRunner.js:182:20
    at context.callback (/Users/Sander/Programming/embroider-in-repo-repro/node_modules/loader-runner/lib/LoaderRunner.js:111:13)
    at Object.callback (/Users/Sander/Programming/embroider-in-repo-repro/node_modules/thread-loader/dist/index.js:51:7)
    at /Users/Sander/Programming/embroider-in-repo-repro/node_modules/async/internal/queue.js:101:31
    at /Users/Sander/Programming/embroider-in-repo-repro/node_modules/async/internal/onlyOnce.js:12:16
    at callback (/Users/Sander/Programming/embroider-in-repo-repro/node_modules/thread-loader/dist/WorkerPool.js:161:21)
    at /Users/Sander/Programming/embroider-in-repo-repro/node_modules/thread-loader/dist/WorkerPool.js:187:17
    at /Users/Sander/Programming/embroider-in-repo-repro/node_modules/async/internal/map.js:32:9
  - code: [undefined]
  - codeFrame: Module build failed (from /Users/Sander/Programming/embroider-in-repo-repro/node_modules/thread-loader/dist/cjs.js):
Thread Loader (Worker 0)
ENOENT: no such file or directory, open '$TMPDIR/embroider/213e4c/lib/in-repo-addon/components/in-repo-component.js/in-repo-component.js'

  - errorMessage: ../../lib/in-repo-addon/components/in-repo-component.js: Module build failed (from /Users/Sander/Programming/embroider-in-repo-repro/node_modules/thread-loader/dist/cjs.js):
Thread Loader (Worker 0)
ENOENT: no such file or directory, open '$TMPDIR/embroider/213e4c/lib/in-repo-addon/components/in-repo-component.js/in-repo-component.js'

        at PackagerRunner (@embroider/webpack)
  - errorType: Build Error
  - location:
    - column: [undefined]
    - file: ../../lib/in-repo-addon/components/in-repo-component.js
    - line: [null]
    - treeDir: [undefined]
  - message: ../../lib/in-repo-addon/components/in-repo-component.js: Module build failed (from /Users/Sander/Programming/embroider-in-repo-repro/node_modules/thread-loader/dist/cjs.js):
Thread Loader (Worker 0)
ENOENT: no such file or directory, open '$TMPDIR/embroider/213e4c/lib/in-repo-addon/components/in-repo-component.js/in-repo-component.js'

        at PackagerRunner (@embroider/webpack)
  - name: Error
  - nodeAnnotation: @embroider/webpack
  - nodeName: PackagerRunner
  - originalErrorMessage: Module build failed (from /Users/Sander/Programming/embroider-in-repo-repro/node_modules/thread-loader/dist/cjs.js):
Thread Loader (Worker 0)
ENOENT: no such file or directory, open '$TMPDIR/embroider/213e4c/lib/in-repo-addon/components/in-repo-component.js/in-repo-component.js'

  - stack: ModuleBuildError: Module build failed (from /Users/Sander/Programming/embroider-in-repo-repro/node_modules/thread-loader/dist/cjs.js):
Thread Loader (Worker 0)
ENOENT: no such file or directory, open '$TMPDIR/embroider/213e4c/lib/in-repo-addon/components/in-repo-component.js/in-repo-component.js'

    at /Users/Sander/Programming/embroider-in-repo-repro/node_modules/webpack/lib/NormalModule.js:316:20
    at /Users/Sander/Programming/embroider-in-repo-repro/node_modules/loader-runner/lib/LoaderRunner.js:367:11
    at /Users/Sander/Programming/embroider-in-repo-repro/node_modules/loader-runner/lib/LoaderRunner.js:182:20
    at context.callback (/Users/Sander/Programming/embroider-in-repo-repro/node_modules/loader-runner/lib/LoaderRunner.js:111:13)
    at Object.callback (/Users/Sander/Programming/embroider-in-repo-repro/node_modules/thread-loader/dist/index.js:51:7)
    at /Users/Sander/Programming/embroider-in-repo-repro/node_modules/async/internal/queue.js:101:31
    at /Users/Sander/Programming/embroider-in-repo-repro/node_modules/async/internal/onlyOnce.js:12:16
    at callback (/Users/Sander/Programming/embroider-in-repo-repro/node_modules/thread-loader/dist/WorkerPool.js:161:21)
    at /Users/Sander/Programming/embroider-in-repo-repro/node_modules/thread-loader/dist/WorkerPool.js:187:17
    at /Users/Sander/Programming/embroider-in-repo-repro/node_modules/async/internal/map.js:32:9

=================================================================================

Steps to reproduce

  1. Clone https://github.com/sandydoo/embroider-in-repo-repro
  2. ember s
  3. Modify/touch any file to cause a rebuild

Works on both of the Embroider optimization settings. You don‘t even need to use anything from the in-repo addon to get it to crash.

Versions

ember-cli: 3.26.1
@embroider/compat: 0.39.1
@embroider/core: 0.39.1
@embroider/test-setup: 0.39.1
@embroider/webpack: 0.39.1

Updates

  1. Setting JOBS=1 gets us a slightly more informative error:
ERROR Summary:

  - broccoliBuilderErrorStack: ModuleBuildError: Module build failed (from /Users/Sander/Programming/embroider-in-repo-repro/node_modules/babel-loader/lib/index.js):
Error: ENOENT: no such file or directory, open '$TMPDIR/embroider/213e4c/lib/in-repo-addon/components/in-repo-component.js/in-repo-component.js'
    at /Users/Sander/Programming/embroider-in-repo-repro/node_modules/webpack/lib/NormalModule.js:316:20
    at /Users/Sander/Programming/embroider-in-repo-repro/node_modules/loader-runner/lib/LoaderRunner.js:367:11
    at Array.<anonymous> (/Users/Sander/Programming/embroider-in-repo-repro/node_modules/loader-runner/lib/LoaderRunner.js:203:19)
    at Storage.finished (/Users/Sander/Programming/embroider-in-repo-repro/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:55:16)
    at ReadFileContext.<anonymous> (/Users/Sander/Programming/embroider-in-repo-repro/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:91:9)
    at ReadFileContext.callback (/Users/Sander/Programming/embroider-in-repo-repro/node_modules/graceful-fs/graceful-fs.js:123:16)
    at FSReqCallback.readFileAfterOpen [as oncomplete] (fs.js:273:13)
  - code: [undefined]
  - codeFrame: Module build failed (from /Users/Sander/Programming/embroider-in-repo-repro/node_modules/babel-loader/lib/index.js):
Error: ENOENT: no such file or directory, open '$TMPDIR/embroider/213e4c/lib/in-repo-addon/components/in-repo-component.js/in-repo-component.js'
  - errorMessage: ../../lib/in-repo-addon/components/in-repo-component.js: Module build failed (from /Users/Sander/Programming/embroider-in-repo-repro/node_modules/babel-loader/lib/index.js):
Error: ENOENT: no such file or directory, open '$TMPDIR/embroider/213e4c/lib/in-repo-addon/components/in-repo-component.js/in-repo-component.js'
        at PackagerRunner (@embroider/webpack)
  - errorType: Build Error
  - location:
    - column: [undefined]
    - file: ../../lib/in-repo-addon/components/in-repo-component.js
    - line: [null]
    - treeDir: [undefined]
  - message: ../../lib/in-repo-addon/components/in-repo-component.js: Module build failed (from /Users/Sander/Programming/embroider-in-repo-repro/node_modules/babel-loader/lib/index.js):
Error: ENOENT: no such file or directory, open '$TMPDIR/embroider/213e4c/lib/in-repo-addon/components/in-repo-component.js/in-repo-component.js'
        at PackagerRunner (@embroider/webpack)
  - name: Error
  - nodeAnnotation: @embroider/webpack
  - nodeName: PackagerRunner
  - originalErrorMessage: Module build failed (from /Users/Sander/Programming/embroider-in-repo-repro/node_modules/babel-loader/lib/index.js):
Error: ENOENT: no such file or directory, open '$TMPDIR/embroider/213e4c/lib/in-repo-addon/components/in-repo-component.js/in-repo-component.js'
  - stack: ModuleBuildError: Module build failed (from /Users/Sander/Programming/embroider-in-repo-repro/node_modules/babel-loader/lib/index.js):
Error: ENOENT: no such file or directory, open '$TMPDIR/embroider/213e4c/lib/in-repo-addon/components/in-repo-component.js/in-repo-component.js'
    at /Users/Sander/Programming/embroider-in-repo-repro/node_modules/webpack/lib/NormalModule.js:316:20
    at /Users/Sander/Programming/embroider-in-repo-repro/node_modules/loader-runner/lib/LoaderRunner.js:367:11
    at Array.<anonymous> (/Users/Sander/Programming/embroider-in-repo-repro/node_modules/loader-runner/lib/LoaderRunner.js:203:19)
    at Storage.finished (/Users/Sander/Programming/embroider-in-repo-repro/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:55:16)
    at ReadFileContext.<anonymous> (/Users/Sander/Programming/embroider-in-repo-repro/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:91:9)
    at ReadFileContext.callback (/Users/Sander/Programming/embroider-in-repo-repro/node_modules/graceful-fs/graceful-fs.js:123:16)
    at FSReqCallback.readFileAfterOpen [as oncomplete] (fs.js:273:13)

=================================================================================
  1. Updated @embroider to 0.40.0. Still reproducable.
siva-sundar commented 3 years ago

Getting same kind of error during the initial build. Maybe related https://github.com/embroider-build/embroider/issues/722

sandydoo commented 3 years ago

Progress!

So dummy apps are special in that they get an owning addon that’s always rebuilt, right?

Well, this treeInstance.sync() that gets run on rebuilds is problematic. It’s sort of nuking the working directory. My poor little in-repo addon comes along next in the queue and doesn’t get a sync(), so it’s never added back to the working directory. That’s why the lib folder goes missing on rebuilds.

Also, it’s deleting something called .embroider-reuse.json — looks important 🤷

https://github.com/embroider-build/embroider/blob/ffa2d3922558cb33ceab45b73f4cdd829f2b9905/packages/compat/src/compat-addons.ts#L136-L152

Doing a sync() on the in-repo addon is enough to fix the issue, but:

  1. Should we be ignoring more files in the TreeSync instance? e.g. .embroider-reuse.json. What else is being deleted here?
  2. Are there any other packages that need this treatment?
  3. How do we catch, specifically, the in-repo addons in this loop? They look indistinguishable from regular addons.