[Bug?]: workspaces foreach --topological-dev run <cmd> ignores dependencies that don't have <cmd> specified

LaurensUP commented 2 years ago

Self-service

[ ] I'd be willing to implement a fix

Describe the bug

We have a dependency chain as follows: Package A depends on package B, which depends on package C. A -> B -> C

Normally, all packages have a build script, but package B does not.

When running yarn workspaces foreach -pv --topological-dev run build, it will skip over package B, because B does not have a build script. Therefor, it will miss the dependency chain of B -> C and yarn will not correctly build package C before A.

To reproduce

I've created a reproduce repo: https://github.com/LaurensUP/yarn-workspace-foreach-topological-repro

Dependency chain: A -> B -> C -> D E & F have no dependencies.

The build order should be D -> C -> B -> A However, because B doesn't have a build script (but it does have a devDependency on C), it is ignored.

The build order we see is A -> D -> C In our real-world application, Process started for A immediately crashes because it depends on C, which hasn't been built yet.

~/code/yarn-workspace-foreach-topological-repro$ yarn build
➤ YN0000: [package-a]: Process started
➤ YN0000: [package-d]: Process started
➤ YN0000: [package-e]: Process started
➤ YN0000: [package-f]: Process started
➤ YN0000: [package-f]: Package F
➤ YN0000: [package-f]: Process exited (exit code 0), completed in 0s 88ms
➤ YN0000: [package-d]: Package D
➤ YN0000: [package-d]: Process exited (exit code 0), completed in 0s 109ms
➤ YN0000: [package-e]: Package E
➤ YN0000: [package-e]: Process exited (exit code 0), completed in 0s 103ms
➤ YN0000: [package-a]: Package A depends on B
➤ YN0000: [package-a]: Process exited (exit code 0), completed in 0s 124ms
➤ YN0000: [package-c]: Process started
➤ YN0000: [package-c]: Package C depends on D
➤ YN0000: [package-c]: Process exited (exit code 0), completed in 0s 43ms

Environment

System:
    OS: Linux 5.11 Ubuntu 21.04 (Hirsute Hippo)
    CPU: (8) x64 Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
  Binaries:
    Node: 14.17.6 - /tmp/xfs-042a5b67/node
    Yarn: 3.1.1 - /tmp/xfs-042a5b67/yarn
    npm: 6.14.15 - ~/.nvm/versions/node/v14.17.6/bin/npm

Additional context

output with colors

yarnbot commented 2 years ago

Hi! 👋

This issue looks stale, and doesn't feature the reproducible label - which implies that you didn't provide a working reproduction using Sherlock. As a result, it'll be closed in a few days unless a maintainer explicitly vouches for it or you edit your first post to include a formal reproduction (you can use the playground for that).

Note that we require Sherlock reproductions for long-lived issues (rather than standalone git repositories or similar) because we're a small team. Sherlock gives us the ability to check which bugs are still affecting the master branch at any given point, and decreases the amount of code we need to run on our own machines (thus leading to faster bug resolutions). It helps us help you! 😃

If you absolutely cannot reproduce a bug on Sherlock (for example because it's a Windows-only issue), a maintainer will have to manually add the upholded label. Thanks for helping us triaging our repository! 🌟

LaurensUP commented 2 years ago

The tooling is with CLI, not in the code. I don't know how to reproduce that with Sherlock..

LaurensUP commented 2 years ago

This issue can't be reproduced with Sherlock. The linked repository is very lean to reproduce the issue.

eyalpost commented 2 years ago

Just encountered the same bug in our repo. We have a prepare script that only a few modules implement and the other (~30) modules don't need. Still I need this prepare to run in the correct order and unfortunately the only workaround is to add a dummy "prepare": ":" script to all other moudles.

LaurensUP commented 2 years ago

@eyalpost and to elaborate on this, it takes 100-200ms for yarn to "do nothing" in a dummy workspace.

arietrouw commented 1 year ago

This bug still seems to exist. Can can reproduce it at will when setting the --topological-dev flag and the --parallel flag at the same time. I believe the reason it seems intermittent is that if by luck, the order is correct it works, or if a clean is not done in between a success and this, the previous build of the required library is used. I can provide a sample repo if need to demonstrate.

yarnpkg / berry