vercel / turborepo

Build system optimized for JavaScript and TypeScript, written in Rust
https://turbo.build/repo/docs
MIT License
26.02k stars 1.79k forks source link

Persistent tasks in watch mode don't wait for first run of their dependencies to finish #8673

Open OliverJAsh opened 2 months ago

OliverJAsh commented 2 months ago

Verify canary release

Link to code that reproduces this issue

https://github.com/OliverJAsh/turbo-watch-deps-of-persistent

What package manager are you using / does the bug impact?

Yarn v2/v3/v4 (node_modules linker only)

What operating system are you using?

Mac

Which canary version will you have in your reproduction?

2.0.6-canary.0

Describe the Bug

Persistent tasks in watch mode don't wait for first run of their dependencies to finish.

In non-watch / run mode, build a correctly runs before build b:

$ TURBO_UI=false yarn run turbo run build --filter b
• Packages in scope: b
• Running build in 1 packages
• Remote caching disabled
a:build: cache bypass, force executing 7c09bd93554e1d1c
a:build: A
b:build: cache bypass, force executing 7f551e27d3b8aa66
b:build: B

 Tasks:    2 successful, 2 total
Cached:    0 cached, 2 total
  Time:    1.516s

However, in watch mode, Turbo runs both builds in parallel. Build b does not wait for build a to finish even though it's a dependency:

$ TURBO_UI=false yarn run turbo watch build --filter b
• Packages in scope: b
• Running build in 1 packages
• Remote caching disabled
• Packages in scope: b
• Running build in 1 packages
• Remote caching disabled
a:build: cache bypass, force executing 7c09bd93554e1d1c
b:build: cache bypass, force executing 25c5580a26314aac
b:build: B
a:build: A

Expected Behavior

Persistent tasks in watch mode should wait for the first run of their dependencies to finish, otherwise they may fail.

To Reproduce

  1. Clone https://github.com/OliverJAsh/turbo-watch-deps-of-persistent.
  2. Run yarn.
  3. Run TURBO_UI=false yarn run turbo watch build --filter b.

Additional context

I believe the idea is that persistent tasks should self-heal: https://github.com/vercel/turbo/issues/8164#issuecomment-2122878984

However, this is inefficient and can lead to confusing error messages.

In our case we're using webpack watch and rebuilding is CPU intensive, so it would be better if webpack waited for its dependencies to finish building before starting.

Furthermore, our webpack watch task can only self-heal after the build has started—it can't self-heal if the configuration itself is missing dependencies.

We only expect the order to be respected for the first run, not subsequent to that.

NicholasLYang commented 2 months ago

Hi, thanks for the issue. We currently don't have the capacity to work on this currently. If you want, you're welcome to open a PR for it and I can provide guidance.