nrwl / nx

Smart Monorepos · Fast CI
https://nx.dev
MIT License
23.69k stars 2.36k forks source link

[Graph] `targetDefaults` non-named inputs broken as of Nx v18 #23271

Open warnellw opened 6 months ago

warnellw commented 6 months ago

Current Behavior

As of Nx v18 (also v19), inputs provided to targetDefaults that are not a namedInput group are not correctly used when calculating affected.

Note useInferencePlugins is set to false

Expected Behavior

Non-named/grouped inputs are used when calculating affected, just as they were in Nx v17.

GitHub Repo

https://github.com/warnellw/nx-bug

Steps to Reproduce

Clone the linked repo. Modify the .eslintrc.json file or the jest.preset.js file. Note no projects are affected.

Here is a basic example using the @nx/eslint:lint target:

"useInferencePlugins": false,
"targetDefaults": {
  "@nx/eslint:lint": {
      "inputs": [
        "default",
        "^production",
        "{workspaceRoot}/.eslintrc.json" // when modified, does not affect
      ],
      "cache": true
    },
}

As per the comment, modifying the .eslintrc.json file in this example does not affect any projects. There are a few caveats to this:

  1. If there are no named groups present affected is calculated correctly (i.e. only "{workspaceRoot}/.eslintrc.json" is present).
  2. Only named groups are used. The following example seems to work correctly:
"useInferencePlugins": false,
"targetDefaults": {
  "@nx/eslint:lint": {
      "inputs": [
        "rootESLint",
        "default",
        "^production"
      ],
      "cache": true
    },
},
"namedInputs": {
  "default": [...],
  "production": [...],
  "rootESLint": [
    "{workspaceRoot}/.eslintrc.json" 
  ]
}

Nx Report

Issue also present in Nx v19

Node   : 20.12.2
OS     : darwin-x64
npm    : 10.5.0

nx                 : 18.3.4
@nx/js             : 18.3.4
@nx/jest           : 18.3.4
@nx/linter         : 18.3.4
@nx/eslint         : 18.3.4
@nx/workspace      : 18.3.4
@nx/devkit         : 18.3.4
@nx/eslint-plugin  : 18.3.4
@nx/nest           : 18.3.4
@nx/node           : 18.3.4
@nrwl/tao          : 18.3.4
@nx/web            : 18.3.4
@nx/webpack        : 18.3.4
typescript         : 5.4.5

Failure Logs

No response

Package Manager Version

No response

Operating System

Additional Information

No response

warnellw commented 6 months ago

I updated the original report with a GitHub Repo. The repo was generated via npx create-nx-workspace@18.3.4, and the @nx/nest plugin added and initialized, and new app generated. Cloning the repo and modifying either the .eslintrc.json file or the jest.preset.js file does not affect any targets (as expected).

warnellw commented 6 months ago

In addition to workspace root files not working correctly, I also noticed that excluded inputs (those starting with a !) were broken. There is another related issue filed #22930

paulhobbel commented 6 months ago

We seem to experience similar issues in our repo. It started because Nx fails to pickup dependency changes from other languages then JS. So we've tried to add the Cargo.lock and Cargo.toml inputs and it seems to not pick up the changes

eliellis commented 5 months ago

This issue also appears to impact projects with targets that override inputs (at the target level) where the inputs array contains named inputs in combination with un-named inputs. For example, given the following project-level target configuration:

// project.json
"targets": {
  "something": {
    "inputs": ["production", "^production", "{workspaceRoot}/some-folder/**/*"]
    "executor": "@scope/package:my-executor"
  }
}

Changing any file within the project, or within its dependencies (where the "production" named input applies) and changing any file in the "some-folder" directory should cause the relevant project to be affected. This worked as expected prior to Nx 18.

With Nx 18 or greater, using the above target configuration it can be observed that:

With Nx 18 or greater, removing "production" and "^production" from the target's "inputs" array and touching a file in "some-folder" causes the project to be affected. So it appears like the combination of named and un-named inputs is the issue here, and is not just a problem isolated to "targetDefaults" usage in nx.json, but anywhere that un-named and named inputs are specified in combination with each other.

More exactly, this behavior does not function correctly after Nx 18.

apijay commented 5 months ago

I am too seeing this issue with our CI.

skyleguy commented 5 months ago

This issue also appears to impact projects with targets that override inputs (at the target level) where the inputs array contains named inputs in combination with un-named inputs. For example, given the following project-level target configuration:

// project.json
"targets": {
  "something": {
    "inputs": ["production", "^production", "{workspaceRoot}/some-folder/**/*"]
    "executor": "@scope/package:my-executor"
  }
}

Changing any file within the project, or within its dependencies (where the "production" named input applies) and changing any file in the "some-folder" directory should cause the relevant project to be affected. This worked as expected prior to Nx 18.

With Nx 18 or greater, using the above target configuration it can be observed that:

  • changing any file where "production" or "^production" applies, the corresponding project is affected ✅
  • changing any file where the non-named input ("{workspaceRoot}/some-folder/**/*") applies DOES NOT cause the corresponding project to be affected ❌

With Nx 18 or greater, removing "production" and "^production" from the target's "inputs" array and touching a file in "some-folder" causes the project to be affected. So it appears like the combination of named and un-named inputs is the issue here, and is not just a problem isolated to "targetDefaults" usage in nx.json, but anywhere that un-named and named inputs are specified in combination with each other.

More exactly, this behavior does not function correctly after Nx 18.

we also noticed this exact issue recently in our projects. definitely dont want to have to backtrack to nx 17

eliellis commented 5 months ago

This issue also appears to impact projects with targets that override inputs (at the target level) where the inputs array contains named inputs in combination with un-named inputs. For example, given the following project-level target configuration:

// project.json
"targets": {
  "something": {
    "inputs": ["production", "^production", "{workspaceRoot}/some-folder/**/*"]
    "executor": "@scope/package:my-executor"
  }
}

Changing any file within the project, or within its dependencies (where the "production" named input applies) and changing any file in the "some-folder" directory should cause the relevant project to be affected. This worked as expected prior to Nx 18.

With Nx 18 or greater, using the above target configuration it can be observed that:

  • changing any file where "production" or "^production" applies, the corresponding project is affected ✅
  • changing any file where the non-named input ("{workspaceRoot}/some-folder/**/*") applies DOES NOT cause the corresponding project to be affected ❌

With Nx 18 or greater, removing "production" and "^production" from the target's "inputs" array and touching a file in "some-folder" causes the project to be affected. So it appears like the combination of named and un-named inputs is the issue here, and is not just a problem isolated to "targetDefaults" usage in nx.json, but anywhere that un-named and named inputs are specified in combination with each other.

More exactly, this behavior does not function correctly after Nx 18.

This issue also appears to impact projects with targets that override inputs (at the target level) where the inputs array contains named inputs in combination with un-named inputs. For example, given the following project-level target configuration:

// project.json
"targets": {
  "something": {
    "inputs": ["production", "^production", "{workspaceRoot}/some-folder/**/*"]
    "executor": "@scope/package:my-executor"
  }
}

Changing any file within the project, or within its dependencies (where the "production" named input applies) and changing any file in the "some-folder" directory should cause the relevant project to be affected. This worked as expected prior to Nx 18.

With Nx 18 or greater, using the above target configuration it can be observed that:

  • changing any file where "production" or "^production" applies, the corresponding project is affected ✅
  • changing any file where the non-named input ("{workspaceRoot}/some-folder/**/*") applies DOES NOT cause the corresponding project to be affected ❌

With Nx 18 or greater, removing "production" and "^production" from the target's "inputs" array and touching a file in "some-folder" causes the project to be affected. So it appears like the combination of named and un-named inputs is the issue here, and is not just a problem isolated to "targetDefaults" usage in nx.json, but anywhere that un-named and named inputs are specified in combination with each other.

More exactly, this behavior does not function correctly after Nx 18.

Hunting around a bit, this problem is resolved by this change: https://github.com/nrwl/nx/commit/722b2d0fac608e1e3e3f10961fd9556e0669be3c#diff-af246d11eb7b076140b2ed1d7d01f57534c16decf430567f94263e96e53550b0

~But it does not appear to have been released yet. 😉~

Actually, the above commit appears in Nx >= 19.1.0, so upgrading to Nx 19.1.0 or higher should resolve the problem.

For anyone with this issue currently on Nx <= 19.1.0, here's the patch-package patch file:

diff --git a/node_modules/nx/src/project-graph/affected/locators/workspace-projects.js b/node_modules/nx/src/project-graph/affected/locators/workspace-projects.js
index 8df5cb8..2f18d5f 100644
--- a/node_modules/nx/src/project-graph/affected/locators/workspace-projects.js
+++ b/node_modules/nx/src/project-graph/affected/locators/workspace-projects.js
@@ -58,7 +58,9 @@ function extractFilesFromInputs(inputs, namedInputs) {
     const globalFiles = [];
     for (const input of inputs) {
         if (typeof input === 'string' && input in namedInputs) {
-            return extractFilesFromInputs(namedInputs[input], namedInputs);
+            globalFiles.push(
+              ...extractFilesFromInputs(namedInputs[input], namedInputs)
+            )
         }
         else if (typeof input === 'string' &&
             input.startsWith('{workspaceRoot}/')) {