Deleting a file while the ember server is running throws a build error.

[colenso]

So far I've noticed this for files in app/helpers/** and app/templates/components/** but not app/components/** files.

[colenso]

Error Summary:

  - broccoliBuilderErrorStack: ModuleNotFoundError: Module not found: Error: Can't resolve './_app_/templates/components/driver-day/allocator.hbs' in '/Users/colenso/Documents/client/node_modules/.embroider/rewritten-packages/ember-cli-json-module.87b08615/node_modules/ember-cli-json-module'
    at /Users/colenso/Documents/client/node_modules/webpack/lib/Compilation.js:2022:28
    at /Users/colenso/Documents/client/node_modules/webpack/lib/NormalModuleFactory.js:817:13
    at eval (eval at create (/Users/colenso/Documents/client/node_modules/webpack/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:10:1)
    at /Users/colenso/Documents/client/node_modules/webpack/lib/NormalModuleFactory.js:275:22
    at eval (eval at create (/Users/colenso/Documents/client/node_modules/webpack/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:24:1)
    at /Users/colenso/Documents/client/node_modules/@embroider/webpack/src/webpack-resolver-plugin.js:50:29
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
  - code: [undefined]
  - codeFrame: Module not found: Error: Can't resolve './_app_/templates/components/driver-day/allocator.hbs' in '/Users/colenso/Documents/client/node_modules/.embroider/rewritten-packages/ember-cli-json-module.87b08615/node_modules/ember-cli-json-module'
  - errorMessage: assets/client.js: Module not found: Error: Can't resolve './_app_/templates/components/driver-day/allocator.hbs' in '/Users/colenso/Documents/client/node_modules/.embroider/rewritten-packages/ember-cli-json-module.87b08615/node_modules/ember-cli-json-module'
        at PackagerRunner (@embroider/webpack)
  - errorType: Build Error
  - location:
    - column: [undefined]
    - file: assets/client.js
    - line: [null]
    - treeDir: [undefined]
  - message: assets/client.js: Module not found: Error: Can't resolve './_app_/templates/components/driver-day/allocator.hbs' in '/Users/colenso/Documents/client/node_modules/.embroider/rewritten-packages/ember-cli-json-module.87b08615/node_modules/ember-cli-json-module'
        at PackagerRunner (@embroider/webpack)
  - name: Error
  - nodeAnnotation: @embroider/webpack
  - nodeName: PackagerRunner
  - originalErrorMessage: Module not found: Error: Can't resolve './_app_/templates/components/driver-day/allocator.hbs' in '/Users/colenso/Documents/client/node_modules/.embroider/rewritten-packages/ember-cli-json-module.87b08615/node_modules/ember-cli-json-module'
  - stack: ModuleNotFoundError: Module not found: Error: Can't resolve './_app_/templates/components/driver-day/allocator.hbs' in '/Users/colenso/Documents/client/node_modules/.embroider/rewritten-packages/ember-cli-json-module.87b08615/node_modules/ember-cli-json-module'
    at /Users/colenso/Documents/client/node_modules/webpack/lib/Compilation.js:2022:28
    at /Users/colenso/Documents/client/node_modules/webpack/lib/NormalModuleFactory.js:817:13
    at eval (eval at create (/Users/colenso/Documents/client/node_modules/webpack/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:10:1)
    at /Users/colenso/Documents/client/node_modules/webpack/lib/NormalModuleFactory.js:275:22
    at eval (eval at create (/Users/colenso/Documents/client/node_modules/webpack/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:24:1)
    at /Users/colenso/Documents/client/node_modules/@embroider/webpack/src/webpack-resolver-plugin.js:50:29
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

[mansona]

Hey @colenso 👋 I just tried to recreate this issue in our test suite and I couldn't get it to fail 🤔 are you able to put together a reproduction for us?

[jknight12882]

We've encountered the same issue. We deleted a template/*.hbs file and the build server threw an error. Here's the relevant stack trace

Module build failed (from ../../thread-loader/dist/cjs.js):
Thread Loader (Worker 0)
ENOENT: no such file or directory, open '/redacted/node_modules/.embroider/rewritten-app/templates/settings/password-reset.hbs/password-reset.hbs'
    at PoolWorker.fromErrorObj (/redacted/node_modules/thread-loader/dist/WorkerPool.js:346:12)
    at /redacted/node_modules/thread-loader/dist/WorkerPool.js:219:29
    at mapSeries (/redacted/node_modules/neo-async/async.js:3625:14)
    at PoolWorker.onWorkerMessage (/redacted/node_modules/thread-loader/dist/WorkerPool.js:173:34)
    at /redacted/node_modules/thread-loader/dist/WorkerPool.js:146:14
    at Socket.onChunk (/redacted/node_modules/thread-loader/dist/readBuffer.js:40:9)
    at Socket.emit (node:events:513:28)
    at Socket.Readable.read (node:internal/streams/readable:527:10)
    at Socket.read (node:net:729:39)
    at flow (node:internal/streams/readable:1011:34)
    at emitReadable_ (node:internal/streams/readable:592:3)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)

This is the interesting line

ENOENT: no such file or directory, open '/redacted/node_modules/.embroider/rewritten-app/templates/settings/password-reset.hbs/password-reset.hbs'

It looks like the filename has been appended twice at the end of the path.

We're using

    "@ember/test-helpers": "^2.9.4",
    "@embroider/compat": "^3.4.0",
    "@embroider/core": "^3.4.2",
    "@embroider/webpack": "^3.2.1",

[swastik]

I've been running into this recently a fair bit — pretty much every time I switch a branch or rebase (usually because there's almost always something deleted / moved).

From my limited testing, it seemed like it only failed if the file that is deleted was imported somewhere. Template files being deleted didn't seem to break anything (which is a bit different from the original issue!)

[mattdonnelly]

I'm able to reproduce this very consistently by simply renaming a file from .js to .ts (and vice-versa). I created an app using Ember 5.12 and the latest version of Embroider (3.4.19) which can be used to reproduce it here: https://github.com/mattdonnelly/embroider-no-such-file-repro

Steps to reproduce it:

1. Clone the app, run npm install and then npm start to start the server.
2. Rename app/lib/test.js to app/lib/test.ts: mv app/lib/test.js app/lib/test.ts.
3. Observe the error

Build Error (PackagerRunner) in lib/test.js

Module build failed (from ../../thread-loader/dist/cjs.js):
Thread Loader (Worker 1)
ENOENT: no such file or directory, open '/Users/mattdonnelly/src/embroider-file-not-found-repro/node_modules/.embroider/rewritten-app/lib/test.js/test.js'
    at PoolWorker.fromErrorObj (/Users/mattdonnelly/src/embroider-file-not-found-repro/node_modules/thread-loader/dist/WorkerPool.js:346:12)
    at /Users/mattdonnelly/src/embroider-file-not-found-repro/node_modules/thread-loader/dist/WorkerPool.js:219:29
    at mapSeries (/Users/mattdonnelly/src/embroider-file-not-found-repro/node_modules/neo-async/async.js:3625:14)
    at PoolWorker.onWorkerMessage (/Users/mattdonnelly/src/embroider-file-not-found-repro/node_modules/thread-loader/dist/WorkerPool.js:173:34)
    at /Users/mattdonnelly/src/embroider-file-not-found-repro/node_modules/thread-loader/dist/WorkerPool.js:146:14
    at Socket.onChunk (/Users/mattdonnelly/src/embroider-file-not-found-repro/node_modules/thread-loader/dist/readBuffer.js:40:9)
    at Socket.emit (node:events:517:28)
    at Readable.read (node:internal/streams/readable:550:10)
    at Socket.read (node:net:781:39)
    at flow (node:internal/streams/readable:1032:34)
    at emitReadable_ (node:internal/streams/readable:615:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:81:21)

Stack Trace and Error Report: /var/folders/15/f1fw5cy10q70r3nw93q3gqtr0000gp/T/error.dump.ff5bd574b1a979402e4b773f001cae62.log

[mattdonnelly]

By starting the Ember app with JOBS=0 I think I was able to get a more useful error:

file deleted lib/test.js
⠹ building... [@embroider/webpack]assets by status 1.36 MiB [cached] 4 assets
asset assets/chunk.e6b12970ed036e9baa8e.js 1.01 MiB [emitted] [immutable] [big] (id hint: vendors)
Entrypoint assets/ember-quickstart5.js [big] 1.39 MiB = assets/chunk.e6b12970ed036e9baa8e.js 1.01 MiB assets/chunk.449be549ebd278b91a13.js 390 KiB assets/chunk.dc2bc2d497d287801140.js 7.58 KiB
Entrypoint assets/test.js [big] 2.36 MiB = assets/chunk.e6b12970ed036e9baa8e.js 1.01 MiB assets/chunk.48df15d92e80e956d1f6.js 681 KiB assets/chunk.449be549ebd278b91a13.js 390 KiB assets/chunk.59e83e298a6f683b863a.js 317 KiB
cached modules 1.31 MiB (javascript) 6.86 KiB (runtime) [cached] 328 modules
javascript modules 3.31 KiB
  modules by path ./lib/ 82 bytes
    ./lib/test.ts 43 bytes [built] [code generated]
    ./lib/test.js 39 bytes [built] [code generated] [1 error]
  ./assets/ember-quickstart5.js 3.05 KiB [built] [code generated]
  ./routes/test.js 184 bytes [code generated]

ERROR in ./lib/test.js
Module build failed (from ../../@embroider/babel-loader-9/index.js):
Error: ENOENT: no such file or directory, open '/Users/matthewdonnelly/src/embroider-no-such-file-repro/node_modules/.embroider/rewritten-app/lib/test.js'
 @ ./routes/test.js 2:0-35 5:16-20
 @ ./assets/ember-quickstart5.js 61:13-56

webpack 5.96.1 compiled with 1 error in 115 ms
Build Error (PackagerRunner) in lib/test.js

Module build failed (from ../../@embroider/babel-loader-9/index.js):
Error: ENOENT: no such file or directory, open '/Users/matthewdonnelly/src/embroider-no-such-file-repro/node_modules/.embroider/rewritten-app/lib/test.js/test.js'

Stack Trace and Error Report: /var/folders/pc/9my4zxps3fv55yh1ld85_b8c0000gp/T/error.dump.36318a69f94b4433c31760848f57b0a5.log

[mattdonnelly]

I think this could be a webpack + TS issue rather than an Embroider one. I just tried doing the same thing on a React project which uses Webpack and got a very similar error

[mattdonnelly]

I tried tweaking the webpack config for the project to see if that fixes it but didn't have any luck.

First I thought maybe this was a resolver caching in issue with webpack. By changing the file extension we change which loaders get applied and maybe that would cause this. So, I tried setting a custom resolve.unsafeCache which could check if the files exists before caching like in the docs: https://webpack.js.org/configuration/resolve/#resolveunsafecache This didn't work though so then I tried disabling caching entirely but that didn't work either.

Then I thought it could be an issue with the watch server not waiting long enough for all the changes to propagate. I tried increasing the watchOptions.aggregateTimeout but it didn't work: https://webpack.js.org/configuration/watch/#watchoptionsaggregatetimeout

[mattdonnelly]

Tried the reproduction steps on a fresh project with React, Webpack + Typescript (using babel/preset-typescript) and wasn't able to get the error to occur. This might be a red herring

[mattdonnelly]

One more thing I tried: I noticed that the logs in the Embroider app had this line file deleted lib/test.js so I thought maybe the issue was that the file was not being moved within .rewritten-app, the old one is getting deleted and a new one added. I tried doing the same thing in the fresh webpack app but surprisingly it handled this just fine. There would be an error initially when I removed test.js but once I added test.ts it would rebuild successfully

[patricklx]

Can you try to exclude the file by adding it to staticAppPaths option. if that helps then it might be related to the entry point, maybe its not rebuild correctly.

[mattdonnelly]

@patricklx Just tried this and got the same error unfortunately

[patricklx]

@mattdonnelly making webpack cache: false solved it for me in your repro.

packagerOptions: {
      webpackConfig: {
        cache: false,
      },
    },

[mattdonnelly]

@patricklx Ah it does! I think when I disabled the cache I only disabled the resolver cache (resolve.cache). Disabling the module cache prevents the exception but it doesn't really solve the problem because now rebuilds take about as long as cold builds.

[patricklx]

on the unsafeCache option. it should work. but it looks like @embroider/webpack is using lodash/merge on the options which converts the proxy to a plain object

[patricklx]

I was able to get unsafeCache option to be used, but no success.

 const realUnsafeCache = {};
  const unsafeCacheHandler = {
    get(cache, key) {
      const cachedValue = cache[key];

      if (key === Symbol.toStringTag) {
        // this will tell lodash to not merge this object
        return 'Proxy';
      }

      if (cachedValue && !existsSync(cachedValue.path)) {
        // and if it doesn't, evict that cache entry.
        delete cache[key];
        return undefined;
      }
      return undefined;
    },
  };
  const theProxiedCache = new Proxy(realUnsafeCache, unsafeCacheHandler);

[patricklx]

i think I found the real issue. it is because the app where webpack is running is inside node_modules. node_modules/.embroider. And webpack makes assumptions that all in node_modules does not need invalidating, including generated bundles.

if you create a symlink from ./node_modules/.embroider -> ./tmp/.embroider all works.

[mattdonnelly]

Adding the symlink works in the reproduction repo but isn't working in a larger real app. Not sure why right now but I'll try investigate some more when I have time