search

yarnpkg / berry

📦🐈 Active development trunk for Yarn ⚒
https://yarnpkg.com
BSD 2-Clause "Simplified" License
7.38k stars 1.11k forks source link

[Bug?]: yarn install error ENOENT: no such file or directory #3214

Closed rdroog closed 2 years ago

rdroog commented 3 years ago

Self-service

Describe the bug

Duplicate of https://github.com/yarnpkg/yarn/issues/4563:

➤ YN0001: │ Error: ENOENT: no such file or directory, chmod '/my-user/my-repository/node_modules/a-package/node_modules/another-package'

To reproduce

I don't have a reproduction path, but there is more than enough information in https://github.com/yarnpkg/yarn/issues/4563, as this is a duplicate.

Environment

  System:
    OS: macOS 11.5.1
    CPU: (16) x64 Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
  Binaries:
    Node: 14.17.0 - /private/var/folders/1s/p6w4cpg94cj2_kbxlkbfrqqw0000gn/T/xfs-1d0a0a1f/node
    Yarn: 2.4.1 - /private/var/folders/1s/p6w4cpg94cj2_kbxlkbfrqqw0000gn/T/xfs-1d0a0a1f/yarn
    npm: 6.14.13 - ~/.volta/tools/image/node/14.17.0/bin/npm
  npmPackages:
    jest: 26.6.3 => 26.6.

Additional context

No solutions of the duplicate issue helped me, like using yarn install instead of yarn, or doing it multiple times (unless 10 times is not enough).

We have seen this occurring in our repository elsewhere too, but not everytime.

merceyz commented 3 years ago

It's not a duplicate of https://github.com/yarnpkg/yarn/issues/4563, that issue was about the cache and concurrency, this isn't that.

Can you see if it has been fixed already in 3.0?

yarn set version berry
yarnbot commented 3 years ago

Hi! 👋

This issue looks stale, and doesn't feature the reproducible label - which implies that you didn't provide a working reproduction using Sherlock. As a result, it'll be closed in a few days unless a maintainer explicitly vouches for it or you edit your first post to include a formal reproduction (you can use the playground for that).

Note that we require Sherlock reproductions for long-lived issues (rather than standalone git repositories or similar) because we're a small team. Sherlock gives us the ability to check which bugs are still affecting the master branch at any given point, and decreases the amount of code we need to run on our own machines (thus leading to faster bug resolutions). It helps us help you! 😃

If you absolutely cannot reproduce a bug on Sherlock (for example because it's a Windows-only issue), a maintainer will have to manually add the upholded label. Thanks for helping us triaging our repository! 🌟

rdroog commented 3 years ago

@merceyz: no, I can't, as I'm limited in what I can do on my work computer. And yarnpkg/yarn#4563 has exactly the same result.

seansfkelley commented 2 years ago

I'm seeing this too, but I have a large closed-source monorepo and I haven't been able to narrow down a smaller repro. I tried it on yarn set version from sources and it still happens.

The specific error I'm getting is

➤ YN0001: │ Error: ENOENT: no such file or directory, lstat '/path/to/monorepo/root/node_modules/@types/mini-css-extract-plugin/node_modules/acorn'

Two questions:

  System:                                                                                                     
    OS: macOS 10.15.7                                                                                         
    CPU: (16) x64 Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz                                                   
  Binaries:                                                                                                   
    Node: 14.15.1 - /private/var/folders/fn/ltn38fwn2x70zt_8w41h0f9w0000gn/T/xfs-fd7f5532/node                
    Yarn: 3.1.0-rc.8.git.20211006.hash-d919dd782 - /private/var/folders/fn/ltn38fwn2x70zt_8w41h0f9w0000gn/T/xfs-fd7f5532/yarn                                                                                               
    npm: 6.14.8 - ~/.volta/tools/image/node/14.15.1/bin/npm

Note to self: this seems to reliably repro when deleting all @types with find . -type d -name @types | xargs rm -rf. (I guess it puts node_modules into a weird state.)

jgillick commented 2 years ago

This is happening for me too. Also on a closed-source repo. For me, it seems to be a race condition. At least, each time I run into it, it gets stuck on a different package.

jgillick commented 2 years ago

Maybe this will help. I've created the overall structure and package dependencies of the monorepo I see this issue in: https://github.com/jgillick/yarn-install-bug

One notable point: the shared package is a dependency of nearly all other packages.

Unfortunately, this bug is not reliably reproducible. Usually, I'll be working with the code base for weeks without problems adding or upgrading packages. Then, if something larger happens to the dependency tree (a node_modeules directory is removed, upgrade-interactive is run and then reverted, etc), it gets into a state where trying to run yarn install ends in the ENOENT error. When that happens it becomes an hour or two of removing cache and running again until it works. I never know what exactly fixes and it usually feels like a race condition where it's trying to install sub-dependencies before the parent dependency has been installed.

jgillick commented 2 years ago

One way you might be able to reproduce it with this repo

From the root of the monorepo: 

yarn install
cd packages/app/
rm -rf ./node_modules/ 

# IMPORTANT: while still in the packages/app/ directory
yarn install

The output I get is:

$ yarn install                                                                                                                                                                                                                                                            18:33:01
➤ YN0000: ┌ Resolution step
➤ YN0002: │ @bug/api@workspace:packages/api doesn't provide @bugsnag/core (p630e7), requested by @bugsnag/plugin-express
➤ YN0002: │ @bug/app@workspace:packages/app doesn't provide prop-types (p96ec9), requested by @alessiocancian/react-native-actionsheet
➤ YN0002: │ @bug/app@workspace:packages/app doesn't provide prop-types (p44ad5), requested by react-native-render-html
➤ YN0002: │ @bug/app@workspace:packages/app doesn't provide prop-types (pb3fc3), requested by react-native-typing-animation
➤ YN0060: │ @bug/app@workspace:packages/app provides react (pb489a) with version 17.0.2, which doesn't satisfy what @harelpls/use-pusher requests
➤ YN0060: │ @bug/app@workspace:packages/app provides react (p49814) with version 17.0.2, which doesn't satisfy what @react-native-community/async-storage requests
➤ YN0060: │ @bug/app@workspace:packages/app provides react (pba4e4) with version 17.0.2, which doesn't satisfy what react-native-fast-image requests
➤ YN0060: │ @bug/app@workspace:packages/app provides react (pd13e2) with version 17.0.2, which doesn't satisfy what react-native-webview requests
➤ YN0060: │ @bug/app@workspace:packages/app provides react (p12dc4) with version 17.0.2, which doesn't satisfy what react-native and some of its descendants request
➤ YN0060: │ @bug/app@workspace:packages/app provides react-native (p94aa1) with version 0.66.3, which doesn't satisfy what react-native-webview requests
➤ YN0060: │ @bug/app@workspace:packages/app provides react-native (pc119c) with version 0.66.3, which doesn't satisfy what redux-flipper requests
➤ YN0060: │ @bug/app@workspace:packages/app provides react-native-flipper (p8bcf4) with version 0.117.0, which doesn't satisfy what redux-flipper requests
➤ YN0060: │ @bug/app@workspace:packages/app provides react-native-svg (p3cf63) with version 12.1.0, which doesn't satisfy what @ui-kitten/components requests
➤ YN0060: │ @bug/app@workspace:packages/app provides react-native-svg (pa8661) with version 12.1.0, which doesn't satisfy what @ui-kitten/eva-icons and some of its descendants request
➤ YN0002: │ @bug/app@workspace:packages/app doesn't provide react-native-web (pc6539), requested by react-native-maps
➤ YN0060: │ @bug/web@workspace:packages/web provides react (p3e5ff) with version 17.0.2, which doesn't satisfy what react-quill requests
➤ YN0002: │ @bug/web@workspace:packages/web doesn't provide react-is (pd62f7), requested by styled-components
➤ YN0002: │ @firebase/database@npm:0.6.8 doesn't provide @firebase/app-types (p495d9), requested by @firebase/auth-interop-types
➤ YN0002: │ @serverless/components@npm:3.12.0 doesn't provide inquirer (p9b5c5), requested by inquirer-autocomplete-prompt
➤ YN0002: │ @ui-kitten/eva-icons@npm:5.0.0 [82c4b] doesn't provide react (p90995), requested by react-native-eva-icons
➤ YN0002: │ appcenter-cli@npm:2.10.4 doesn't provide webpack (pd0b94), requested by webpack-cli
➤ YN0002: │ react-native-codegen@npm:0.0.7 doesn't provide @babel/preset-env (pe4ee6), requested by jscodeshift
➤ YN0002: │ scrubbr@npm:1.1.0 doesn't provide @types/node (pe3f64), requested by ts-node
➤ YN0002: │ scrubbr@npm:1.1.0 doesn't provide typescript (p1a240), requested by ts-node
➤ YN0000: │ Some peer dependencies are incorrectly met; run yarn explain peer-requirements <hash> for details, where <hash> is the six-letter p-prefixed code
➤ YN0000: └ Completed in 0s 388ms
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed in 1s 401ms
➤ YN0000: ┌ Link step
➤ YN0076: │ @next/swc-android-arm64@npm:12.0.4 The darwin-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ @next/swc-darwin-arm64@npm:12.0.4 The darwin-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ @next/swc-linux-arm-gnueabihf@npm:12.0.4 The darwin-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ @next/swc-linux-arm64-gnu@npm:12.0.4 The darwin-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ @next/swc-linux-arm64-musl@npm:12.0.4 The darwin-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ @next/swc-linux-x64-gnu@npm:12.0.4 The darwin-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ @next/swc-linux-x64-musl@npm:12.0.4 The darwin-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ @next/swc-win32-arm64-msvc@npm:12.0.4 The darwin-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ @next/swc-win32-ia32-msvc@npm:12.0.4 The darwin-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ @next/swc-win32-x64-msvc@npm:12.0.4 The darwin-x64 architecture is incompatible with this module, link skipped.
➤ YN0001: │ Error: While cloning /Users/jeremy-bloom/Documents/Bloom/yarn-install-bug/packages/app/node_modules/xcode/node_modules/uuid -> /Users/jeremy-bloom/Documents/Bloom/yarn-install-bug/packages/app/node_modules/appcenter-cli/node_modules/uuid ENOENT: no such file or directory, scandir '/Users/jeremy-bloom/Documents/Bloom/yarn-install-bug/packages/app/node_modules/xcode/node_modules/uuid'
➤ YN0000: └ Completed in 1s 302ms
➤ YN0000: Failed with errors in 3s 212ms
searchableguy commented 2 years ago

I can reproduce the error posted by @jgillick

I'm also getting random errors.

➤ YN0076: │ fsevents@patch:fsevents@npm%3A2.1.3#~builtin<compat/fsevents>::version=2.1.3&hash=18f3a7 The linux-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ fsevents@patch:fsevents@npm%3A2.3.2#~builtin<compat/fsevents>::version=2.3.2&hash=18f3a7 The linux-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ fsevents@patch:fsevents@npm%3A1.2.13#~builtin<compat/fsevents>::version=1.2.13&hash=18f3a7 The linux-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ @next/swc-android-arm64@npm:12.0.4 The linux-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ @next/swc-darwin-arm64@npm:12.0.4 The linux-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ @next/swc-darwin-x64@npm:12.0.4 The linux-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ @next/swc-linux-arm-gnueabihf@npm:12.0.4 The linux-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ @next/swc-linux-arm64-gnu@npm:12.0.4 The linux-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ @next/swc-linux-arm64-musl@npm:12.0.4 The linux-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ @next/swc-win32-arm64-msvc@npm:12.0.4 The linux-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ @next/swc-win32-ia32-msvc@npm:12.0.4 The linux-x64 architecture is incompatible with this module, link skipped.
➤ YN0076: │ @next/swc-win32-x64-msvc@npm:12.0.4 The linux-x64 architecture is incompatible with this module, link skipped.
➤ YN0001: │ Error: ENOENT: no such file or directory, lstat '/home/searchableguy/projects/yarn-install-bug/packages/app/node_modules/appcenter-cli/node_modules/temp/node_modules/mkdirp'
➤ YN0000: └ Completed in 2s 568ms
➤ YN0000: Failed with errors in 6s 107ms
searchableguy commented 2 years ago

@jgillick are you using wsl2 or inside a VM by any chance?

larixer commented 2 years ago

I think it should be fixed via this PR: #3467 You can try it via:

yarn set version from sources --branch 3467
jgillick commented 2 years ago

@larixer that fix is promising. So far it's working, I need to run through it a couple more times to be sure. Do you have an idea when it might go from draft to review?

larixer commented 2 years ago

@jgillick I need to write integration tests to change it to review status, hopefully I will find time to do this soon, perhaps this week

jgillick commented 2 years ago

I can confirm that PR https://github.com/yarnpkg/berry/pull/3467 seems to fix the problem.

jgillick commented 2 years ago

@jgillick I need to write integration tests to change it to review status, hopefully I will find time to do this soon, perhaps this week

That's good news, thank you. I also just threw your team some beer money on the sponsor page. Keep up the good work on Yarn!

larixer commented 2 years ago

That's good news, thank you. I also just threw your team some beer money on the sponsor page. Keep up the good work on Yarn!

Thank you Jeremy!

eugene-kim commented 2 years ago

@larixer @jgillick my apologies for the noob question but I'm running into the following error when attempting to install the branch 3467 changes via yarn set version from sources --branch 3467:

➤ YN0000: Fetching the latest commits

  $ git fetch origin pull/3467/head --force
  $ git reset --hard FETCH_HEAD
  $ git clean -dfx

➤ YN0000: Building a fresh bundle

Usage Error: Couldn't find the node_modules state file - running an install might help (findPackageLocation)

$ yarn run [--inspect] [--inspect-brk] [-T,--top-level] [-B,--binaries-only] <scriptName> ...
➤ YN0001: Error: Child "yarn" exited with exit code 1
    at ChildProcess.<anonymous> (/Users/eugenekim/Projects/shoptable/.yarn/releases/yarn-berry.cjs:317:64340)
    at ChildProcess.emit (events.js:315:20)
    at maybeClose (internal/child_process.js:1048:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)
➤ YN0000: Failed with errors in 3s 841ms

I'm currently on yarn 3.1.1. Any idea how I can resolve this? Running yarn install afterwards doesn't help.

merceyz commented 2 years ago

Make sure you haven't set nodeLinker: node-modules in ~/.yarnrc.yml and/or any parent directory of your temp folder

eugene-kim commented 2 years ago

@merceyz is the change only available if I'm using plug and play?

Unfortunately even with nodeLinker removed from my yarn config I'm running into the same issue.

penx commented 2 years ago

We ran in to this issue with 3.2.0 due to Netlify not restoring our cache correctly (we think) and then getting this issue during install.

Confirmed that yarn set version from sources --branch 3467 fixes the issue for us when using nodeLinker: node-modules and nmHoistingLimits: workspaces.

Would be great to get #3467 merged!

penx commented 2 years ago

Here's a reproduction for our issue - https://github.com/JakeSidSmith/yarn-link-step-issue