dependsOn: run dependencies (^) first

[mpsanchis]

• [x] I'd be willing to implement this feature (contributing guide)

Hi team!

I would like to ask if there is a way of ensuring that the targets T_i that a target T depends on are run in a certain order.

Description

(TL;DR) As this example from the slack support channel mentions, I would like to know, given a target "dependsOn": ["^build", "pre-build"], if we can ensure that ^build runs before than pre-build. I know we can make pre-build depend on ^build, but this has limitations (see arguments below).

Motivations (M)

(Long description) In a big repo with different modules that use different tech stacks, we are using the nx.json to define implementations of basic CI stages (build, test, release, deploy, e2e). In the individual project.jsons, we only have to declare "empty" (nx:noop) stages that extend the interface defined in nx.json. In other words, we use the same name for all the CI stages (i.e., build builds a Typescript lib, but also a Java back-end), and the executors corresponding to these CI stages are a nx:noop that dependsOn the actual implementation (build depends on build-java in the project.json of a java project). This is the cleanest interface we have managed to get so far.

Some specific implementations require extra steps. For example, we can have a build-plugin implementation that depends on build-code, build-docs and ^build. Of course, we would like ^build to run first, but the current API doesn’t allow it as far as I am concerned. There’s two main arguments that support the addition of a feature that takes care of this:

M1: Cleaner APIs

In the previous example, we can always sacrifice parallelization and still get the tasks to run in order, by doing:

  “build-plugin”: {
    “dependsOn”: [“build-code”]
    …
  },
  “build-code”: {
    “dependsOn”: [“build-docs”]
    …
  },
  “build-docs”: {
    “dependsOn”: [“^build]
    …
  }

However, this doesn’t accurately represent what we want to achieve. Moreover, if at some point we get rid of the docs to do something else, we break the build target. This is the main argument for supporting the requested feature. Defining “auxiliary” dependencies in the configuration files only to make targets executed as expected is a bad practice. In our example, build-code does not depend on build-docs in any way, but we still have to write it if we want to avoid dependency problems.

M2: Parallelization

As mentioned in M1, in order to achieve the desired functionality we need to sacrifice parallelization. If build-code and build-docs were slow processes, running one after the other would be a big penalty.

Suggested Implementation

I would expect that the syntax of dependsOn allows us to define which dependencies must run before, and which can run in parallel.

An example of this would be to extend the current dependsOn definition to allow an extra parameter, called for instance priority. This would take a string value of first, last or parallel (default), and would indicate when to run it. The example we mentioned in the motivation would look like:

    "build-plugin”: {
      "dependsOn": [{
        "dependencies": true, // Run this target on all dependencies first
        "priority": "first", // first, last, parallel (default)
        "target": "build", // target name
        "params": "ignore" // "forward" or "ignore", defaults to "ignore"
      },
      {
        "dependencies": false,
        "target": "build-code",
        "params": "ignore" 
      },
        {
        "dependencies": false,
        "target": "build-docs",
        "params": "ignore"
      }
      ]
    }

This would make sure that ^build runs before than build-code and build-docs.

An additional thought: I guess most cases will require that targets from dependent projects (^some-target) run first. By default, I would give higher priority to dependent projects, making dependsOn: [^build, pre-build] run ^build first and pre-build second.

---

I have tried to keep the explanation as generic as possible, with examples that could come up, but kept it abstract on purpose.

Thank you!

[jbadeau]

In short,

run dep(^) targets first then project target. Maybe add a flag to keep current order or as suggested above, make it configurable from the object style dependsOn

[jbadeau]

On second though, a way to define order is prob needed, not just deps first

[jbadeau]

Any thoughts on this

[jokeyrhyme]

Howdie,

Yeah, I'm finding nx extremely useful, but I've encountered a few cases where it'd be much simpler for me if I could instruct it to perform certain targets in a strict sequence

I could use "nx:commands" with parallel = false ( https://nx.dev/packages/nx/executors/run-commands ) but then these would be separate invocations of nx and thus wouldn't integrate properly into the graph of the initial invocation, and thus don't benefit from deduplication, etc

e.g. commands =

• nx exec -- first --
• nx exec -- second --
• nx exec -- third --

[jbadeau]

Can we get this prioritised? I really think this issue is pretty important as we are not able to rely on ^ to build dependencies first which either means turn off parallel or move ^ to leaf tasks

[lppedd]

Any solution yet?

[jbadeau]

Any solution here? I don't understand why Nx does not run dependency task first.

[FrozenPandaz]

I believe the following sort of configuration will work for you:

"build": {
  "dependsOn": ["build-*"]
},
"build-plugin": {
  "dependsOn": ["^build"]
},
"build-code": {
  "dependsOn": ["^build"]
},
"build-docs": {
  "dependsOn": ["^build"]
}

With this configuration, no parallelization is lost.

If ProjectA depends on ProjectB -

1. ProjectB:build-docs, ProjectB:build-code, and ProjectB:build-plugin will all run in parallel.
2. ProjectB:build will run (noop)
3. ProjectA:build-docs, ProjectA:build-code, and ProjectA:build-plugin will all run in parallel.
4. ProjectA:build will run (noop)

This guarantees that everything from ProjectB will run before ProjectA. Does that work for you?