[Bug?]: Integration glitches with nested projects

mcandre commented 6 days ago

Self-service

[ ] I'd be willing to implement a fix

Describe the bug

Yarn repeatedly ignores the current working directory and insists on overwriting files in ancestor directories.

I often create nested projects in example/ directories within a large application. This happens when you end up writing lots of developer tools.

However, the Yarn tutorial, and yarn init -v2, and yarn install, keep messing up things by assuming that the outermost package.json file is always the most relevant one. But that's backwards. The closest file, such as ./package.json, is generally the most relevant.

After creating an empty yarn.lock file as described in the middle of a long error blurb from yarn install, then Yarn manages to corrupt git. It creates a bad .gitignore in the child folder that tries to check in the .yarn/install-state.gz archive into version control. Even though documentation and the gitignore.io patterns explain several times that this file, and others like .yarn/unplugged, should be excluded from version control.

Some of Yarn's assumptions are backwards: It should NOT blindly use yarn.lock's parent folder to assume the place to initialize git configurations. It should check for existing .git directories in the current working directory, then the parent, then the grandparent, trampolining up the file system until it runs out of ancestor directories to check, ancestor volumes to check (native Windows), or encounters a read access error. But it should not try to create a new .git directory in examples/ when examples/' parent already has one. Congratulations, you just broke git.

I tried deleting the bad git files and the .yarn directory, and then rerunning yarn install. Same issues.

To reproduce

Create a git repo with an outer Node.js application managed by NPM, and a dummy inner application managed by Yarn.

Environment

System:
    OS: macOS 15.0
    CPU: (10) arm64 Apple M1 Pro
  Binaries:
    Node: 20.17.0 - /private/var/folders/xc/s20l07yj76x8m3h20lmy5jlc0000gn/T/xfs-a742eb51/node
    Yarn: 4.5.0 - /private/var/folders/xc/s20l07yj76x8m3h20lmy5jlc0000gn/T/xfs-a742eb51/yarn
    npm: 10.8.2 - ~/.asdf/plugins/nodejs/shims/npm

Also, yarn init -2 or yarn install appear to have wiped out the vast majority of my inner package.json keys. My dependencies and devDependencies went missing. The Yarn dependency tree is empty. I can't tell whether the current version of Yarn did that, or perhaps the bastard Yarn v1 edition that Yarn 3 downgraded itself to.

Additional context

Never had these problems with NPM, pip, RubyGems, Cargo, or go mod.

arcanis commented 5 days ago

The closest file, such as ./package.json, is generally the most relevant.

Not really, no. The most typical case is to have a package.json inside a node_modules, in which case the closest file isn't the most relevant.

This heuristic is simple enough, predictable enough, easy to explain enough, and I don't think there would be enough value in researching for another.

But it should not try to create a new .git directory in examples/ when examples/' parent already has one. Congratulations, you just broke git.

Yes, that'd be a welcome improvement. We have a contribution guide to help you get started. The init command itself is fairly simple.

mcandre commented 5 days ago

Not really, no. The most typical case is to have a package.json inside a node_modules, in which case the closest file isn't the most relevant.

Sure, */node_modules/* should always be excluded from consideration when locating the package manager configuration.

yarnpkg / berry