vercel / turborepo

Build system optimized for JavaScript and TypeScript, written in Rust
https://turbo.build/repo
MIT License
26.47k stars 1.84k forks source link

No way to express relation with persistent task #9423

Open leo-petrucci opened 2 weeks ago

leo-petrucci commented 2 weeks ago

Verify canary release

Link to code that reproduces this issue

https://github.com/leo-petrucci/vite-turborepo-issue

What package manager are you using / does the bug impact?

pnpm

What operating system are you using?

Mac

Which canary version will you have in your reproduction?

2.2.4-canary.9

Describe the Bug

In our application we have many apps that don't all need to be started at the same time. For example app-1 might need:

Then app-2 might need:

So we ended up setting up different turbo commands for each of these parts of the app. It means that if a team is working on a specific part of our stack they don't need to start stacks they won't need.

Our turbo.json looks something like this:

{
  "$schema": "https://turbo.build/schema.json",
  "ui": "tui",
  "tasks": {
    "shop": {
      "cache": false,
      "persistent": true,
      "dependsOn": ["app-1#dev"]
    },
    "dashboard": {
      "cache": false,
      "persistent": true,
      "dependsOn": ["app-2#dev"]
    },
    "app-1#dev": {
      "persistent": true
    },
    "app-2#dev": {
      "persistent": true
    }
  }
}

If a team needs to work on the dashboard they'll run turbo dashboard if they have to work on the shop they'll run turbo shop.

This used to work great in v1, but it's now broken in v2.

In the current version 2.2.3 when I start a Vite app without the persistent option Vite shuts down as soon as it's finished booting up. This was first reported in #7279.

Many issues suggest adding persistent: true to your tasks, however that means we're unable to add those commands to dependsOn.

❯ turbo shop
turbo 2.2.4-canary.9

 WARNING  Unable to calculate transitive closures: Workspace 'apps/app-3' not found in lockfile
  × invalid task configuration

Error:   × "app-1#dev" is a persistent task, "app-2#shop" cannot depend on it
   ╭─[turbo.json:7:1]
 7 │       "persistent": true,
 8 │       "dependsOn": ["app-1#dev"]
   ·                     ─────┬─────
   ·                          ╰── persistent task
 9 │     },
   ╰────
Error:   × "app-1#dev" is a persistent task, "app-3#shop" cannot depend on it
   ╭─[turbo.json:7:1]
 7 │       "persistent": true,
 8 │       "dependsOn": ["app-1#dev"]
   ·                     ─────┬─────
   ·                          ╰── persistent task
 9 │     },
   ╰────
Error:   × "app-1#dev" is a persistent task, "app-1#shop" cannot depend on it
   ╭─[turbo.json:7:1]
 7 │       "persistent": true,
 8 │       "dependsOn": ["app-1#dev"]
   ·                     ─────┬─────
   ·                          ╰── persistent task
 9 │     },
   ╰────

Is there a solution for this beyond using the --filter option? Since that would require developers to remember which apps they want to start every time.

Expected Behavior

I should be able to start a task that depends on other persistent tasks.

To Reproduce

Additional context

No response

chris-olszewski commented 1 week ago

We're doing the correct thing here as dependsOn isn't the relation that we desire here, but more of a "this task should be run alongside". If we allowed tasks to depend on persistent tasks, then they would never run as they would wait for the persistent task to exit. In your example app-2#shop would never get to run as it would wait for app-1#dev to exit before starting.

The most immediate solution for you would be to create top level scripts that invoke the sets of tasks you want e.g. "app-1-shop": "turbo run app-1#shop app-3#dev" to avoid having developers remember that they need to start app-3#dev when working on the shop.

We could look at adding a new relation between tasks so tasks could add persistent tasks into the graph, but that would be a new feature.

This used to work great in v1, but it's now broken in v2.

This was an unintentional bug in v1 where we would keep stdin open regardless of task configuration. This lead to tasks getting stuck as they waiting for input that would never come.

In the current version 2.2.3 when I start a Vite app without the persistent option Vite shuts down as soon as it's finished booting up.

vite will shut down if stdin is closed and any task that isn't persistent or interactive will have stdin closed to prevent it from waiting on non-existent input. You should add persistent to your task.