search

nrwl / nx

Smart Monorepos · Fast CI
https://nx.dev
MIT License
23.7k stars 2.37k forks source link

Io error. Look inside err_kind for more details #18111

Open kf6kjg opened 1 year ago

kf6kjg commented 1 year ago

Current Behavior

Executing 

rm -rf node_modules/.cache/nx
npx nx run-many --output-style=stream --target=build --parallel=8

works fine. A second execution of

npx nx run-many --output-style=stream --target=build --parallel=8

Results in

>  NX   Running target build for 18 projects:
...REDACTED...

 >  NX   Successfully ran target build for 18 projects

 >  NX   Io error. Look inside err_kind for more details.

   Pass --verbose to see the stacktrace.

The verbose flag only shows the following additional data:

Error: Io error. Look inside err_kind for more details.
    at /app/node_modules/nx/src/tasks-runner/cache.js:155:21
    at new Promise (<anonymous>)
    at Cache.<anonymous> (/app/node_modules/nx/src/tasks-runner/cache.js:153:20)
    at Generator.next (<anonymous>)
    at /app/node_modules/tslib/tslib.js:167:75
    at new Promise (<anonymous>)
    at Object.__awaiter (/app/node_modules/tslib/tslib.js:163:16)
    at Cache.remove (/app/node_modules/nx/src/tasks-runner/cache.js:151:24)
    at Cache.<anonymous> (/app/node_modules/nx/src/tasks-runner/cache.js:102:36)
    at Generator.next (<anonymous>)

I'm executing from inside of a customized Docker Dev Container based off of mcr.microsoft.com/vscode/devcontainers/javascript-node:0-18-bullseye using VSCode.

Expected Behavior

That NX should recognize that all entries are cached and not report any errors.

If there are indeed IO errors, then it should report the real error, not just the wrapper error posted by rust fs_extra. This would help me diagnose the underlying problem.

GitHub Repo

No response

Steps to Reproduce

I'm not sure I have a generic way to reproduce this at this time. The root cause may be unique to me - this request is more to see if we can get the error message handling provide better information. It looks like the change to using this library was recent? https://github.com/nrwl/nx/pull/17638

Nx Report

Node   : 18.16.0
   OS     : linux-arm64
   npm    : 8.19.4

   nx             : 16.5.2
   @nx/workspace  : 16.5.2
   @nx/devkit     : 16.5.2
   @nrwl/tao      : 16.5.2
   typescript     : 5.1.3
   ---------------------------------------
   Local workspace plugins:
         @REDACTED/config-demo
         @REDACTED/nx-config
         @REDACTED/types

Failure Logs

Error: Io error. Look inside err_kind for more details.
    at /app/node_modules/nx/src/tasks-runner/cache.js:155:21
    at new Promise (<anonymous>)
    at Cache.<anonymous> (/app/node_modules/nx/src/tasks-runner/cache.js:153:20)
    at Generator.next (<anonymous>)
    at /app/node_modules/tslib/tslib.js:167:75
    at new Promise (<anonymous>)
    at Object.__awaiter (/app/node_modules/tslib/tslib.js:163:16)
    at Cache.remove (/app/node_modules/nx/src/tasks-runner/cache.js:151:24)
    at Cache.<anonymous> (/app/node_modules/nx/src/tasks-runner/cache.js:102:36)
    at Generator.next (<anonymous>)

Operating System

Additional Information

No response

kf6kjg commented 1 year ago

So I found out more symptoms.

  1. This happens on Intel Macs as well.
  2. A co-worker discovered that if he downgraded to NX 16.4 he got the following much more useful error: 
    >  NX   EBUSY: resource busy or locked, rmdir '/app/packages/tsconfig-generator/dist'
  3. This is because NX has some form of default behavior ~for the build target~ that is conflicting with the fact that my devcontainer set up is mounting a tmpfs disk on the dist folder. [EDIT: it's not just the build target, it's any target even nonsense named targets.]

~Workaround: change the target name to something that NX doesn't have default behaviors for.~ If there's a way to turn this off I'd prefer that.

Suggestion to fix: stop removing targeted folders and recreating them. Instead empty them. You don't know who has mounted what where.

Cammisuli commented 1 year ago

This isn't a fix, but it'll help us solve this issue. I've done what you suggested, and exposed the IoError that gets swallowed up by fs_extra. Hopefully this will give us some more answers on what we can do for you

18656

kf6kjg commented 1 year ago

Thank you. We'd diagnosed the underlying cause by using an earlier version of nx: we got an error that the dist folder could not be deleted. We had mounted the dist folder with tmpfs. This makes the folder unable to be deleted, causing an error when nx tries to remove and recreate it.

github-actions[bot] commented 9 months ago

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs. If we missed this issue please reply to keep it active. Thanks for being a part of the Nx community! 🙏

kf6kjg commented 9 months ago

Do you need further input? With the new diagnostics it should be easy to reproduce.

Cammisuli commented 9 months ago

No, its something we still need to fix, as we're getting the same kind of errors in our e2es - randomly of course.

github-actions[bot] commented 3 months ago

This issue has been automatically marked as stale because it hasn't had any activity for 6 months. Many things may have changed within this time. The issue may have already been fixed or it may not be relevant anymore. If at this point, this is still an issue, please respond with updated information. It will be closed in 21 days if no further activity occurs. Thanks for being a part of the Nx community! 🙏

kf6kjg commented 3 months ago

Hopefully I'll be back to working on the monorepos in a couple months. Was working on converting a lot of repos, but other priorities took over.