Inputs including invalid files per wildcard

jakeleventhal commented 5 months ago

Verify canary release

[X] I verified that the issue exists in the latest Turborepo canary release.

Link to code that reproduces this issue

n/a

What package manager are you using / does the bug impact?

pnpm

What operating system are you using?

Mac

Which canary version will you have in your reproduction?

1.13.4-canary.1

Describe the Bug

Input globs are including inputs that ought to be ignored.


{
  "taskId": "eslint-config-riptech#tsc",
  "task": "tsc",
  "package": "eslint-config-riptech",
  "hash": "a23ac4d2526413e1",
  "inputs": {
    "dist/index.js": "b5362442ba73e509ca2f4db22418213542307211",
    "dist/index.js.map": "2c7e214b48957fa39ca16f872479395ea6eb7aba",
    "dist/index.test.js": "5021d3e66b01bd8095dd15675764a8329fd7ec25",
    "dist/index.test.js.map": "edc0241d78f9761571da8e61625ae66e856a0217",
    "dist/jest.config.js": "13abd846300e05b6f9294118563d4128f5b28980",
    "dist/jest.config.js.map": "0def8b1108468d921d6b57f113fa806c97ebad5f",
    "dist/package.json": "729a1af6c7cea032967d9908281695abae796f3a",
    "index.js": "58d92cba713db484ef8459d0c1f475724cf3f14e",
    "index.test.ts": "0ee517dc7ae63fc0b324e94353bc5e5b947d7bd0",
    "jest.config.ts": "1075088e1e5d5a6f8608f71dcd45a199357b4b31",
    "node_modules/.bin/eslint": "3bac316b57d759cba8e93ff7d29af61f6df4d303",
    "node_modules/.bin/eslint-config-prettier": "78702df3da5547a5980d7e16bec11b46d391d76c",
    "node_modules/.bin/jest": "46c29b48ef0068807b0c6bd2aa2f982654c7cf88",
    "node_modules/.bin/ts-jest": "3d6bc861c8966922a2f8730b44e27db6c6464708",
    "package.json": "e54a75366018afd223789bdc8aa2802e9819d527",
    "tsconfig.json": "9bd281cf01d1d9ddb4c3766e20c39832bb5cadcf"
  },
  "hashOfExternalDependencies": "4b94ff0e4880e173",
  "cache": {
    "local": true,
    "remote": false,
    "status": "HIT",
    "source": "LOCAL",
    "timeSaved": 10252
  },
  "command": "tsc",
  "cliArguments": [],
  "outputs": [
    "dist/**"
  ],
  "excludedOutputs": null,
  "logFile": "tools/eslint/.turbo/turbo-tsc.log",
  "directory": "tools/eslint",
  "dependencies": [
    "//#package-deps",
    "@riptech/tsconfig#tsc"
  ],
  "dependents": [
    "@apps-scripts/analytics-settings#tsc",
    "@apps-scripts/campaign-matrix#tsc",
    "@apps-scripts/input-tracking#tsc",
    "@apps-scripts/ppc-tools#tsc",
    "@apps-scripts/product-analysis#tsc",
    "@apps-scripts/restock-analytics#tsc",
    "@apps-scripts/returns#tsc",
    "@artelo/admin#tsc",
    "@artelo/api#tsc",
    "@artelo/bulk-product-create#tsc",
    "@artelo/client#tsc",
    "@artelo/database#tsc",
    "@artelo/integrations#tsc",
    "@artelo/landing#tsc",
    "@artelo/pricing-csv#tsc",
    "@artelo/process-image#tsc",
    "@artelo/product-catalog#tsc",
    "@artelo/shared-ui#tsc",
    "@artelo/sls-api#tsc",
    "@artelo/types#tsc",
    "@artelo/utils#tsc",
    "@ecominate/api#tsc",
    "@ecominate/client#tsc",
    "@ecominate/database#tsc",
    "@ecominate/integrations#tsc",
    "@ecominate/landing#tsc",
    "@ecominate/shared-ui#tsc",
    "@ecominate/types#tsc",
    "@ecominate/utils#tsc",
    "@ppc/client#tsc",
    "@ppc/database#tsc",
    "@ppc/integrations#tsc",
    "@ppc/types#tsc",
    "@riptech/apps-script-utils#tsc",
    "@riptech/aws#tsc",
    "@riptech/docker#tsc",
    "@riptech/emailer#tsc",
    "@riptech/google#tsc",
    "@riptech/jest-config#tsc",
    "@riptech/lcov-merger#tsc",
    "@riptech/logger#tsc",
    "@riptech/minimal-ui#tsc",
    "@riptech/seller-central-utilities#tsc",
    "@riptech/shared-ui#tsc",
    "@riptech/stripe#tsc",
    "@riptech/test-utils#tsc",
    "@riptech/utils#tsc",
    "@seller-central-utilities/database#tsc",
    "@seller-central-utilities/types#tsc",
    "@switch/artelo-splitter#tsc",
    "@switch/json-splitter#tsc",
    "@switch/print-load-balancer#tsc"
  ],
  "resolvedTaskDefinition": {
    "outputs": [
      "dist/**"
    ],
    "cache": true,
    "dependsOn": [
      "//#package-deps",
      "^tsc"
    ],
    "inputs": [
      "!.next/**",
      "!.vercel/**",
      "!coverage/**",
      "!dist/**",
      "!localDBData/**",
      "!next-env.d.ts",
      "!node_modules/**",
      "!out/**",
      "!uploadedFiles/**",
      "**/*.js",
      "**/*.jsx",
      "**/*.ts",
      "**/*.tsx",
      "tsconfig*.json"
    ],
    "outputMode": "new-only",
    "persistent": false,
    "env": [],
    "passThroughEnv": null,
    "dotEnv": null,
    "interactive": false
  },
  "expandedOutputs": [
    "tools/eslint/.turbo/turbo-tsc.log",
    "tools/eslint/dist",
    "tools/eslint/dist/index.js",
    "tools/eslint/dist/index.js.map",
    "tools/eslint/dist/index.test.js",
    "tools/eslint/dist/index.test.js.map",
    "tools/eslint/dist/jest.config.js",
    "tools/eslint/dist/jest.config.js.map",
    "tools/eslint/dist/package.json"
  ],
  "framework": "",
  "envMode": "loose",
  "environmentVariables": {
    "specified": {
      "env": [],
      "passThroughEnv": null
    },
    "configured": [],
    "inferred": [],
    "passthrough": null
  },
  "dotEnv": null,
  "execution": {
    "startTime": 1715178334020,
    "endTime": 1715178334056,
    "exitCode": 0
  }
}

This is one of my tasks (output from --summarize). As you can see, the inputs wildcard should be ignoring !dist/**; however, the following are showing up as task inputs:

    "dist/index.js": "b5362442ba73e509ca2f4db22418213542307211",
    "dist/index.js.map": "2c7e214b48957fa39ca16f872479395ea6eb7aba",
    "dist/index.test.js": "5021d3e66b01bd8095dd15675764a8329fd7ec25",
    "dist/index.test.js.map": "edc0241d78f9761571da8e61625ae66e856a0217",
    "dist/jest.config.js": "13abd846300e05b6f9294118563d4128f5b28980",
    "dist/jest.config.js.map": "0def8b1108468d921d6b57f113fa806c97ebad5f",
    "dist/package.json": "729a1af6c7cea032967d9908281695abae796f3a",

Same goes for my node_modules files

Expected Behavior

Ignore wildcards work

To Reproduce

See text

Additional context

No response

anthonyshew commented 5 months ago

Are you using the inputs key for the task? Note that, unless you add $TURBO_DEFAULT$ , .gitignore will not be taken into account as usual for your inputs.

https://turbo.build/repo/docs/reference/configuration#inputs

If that isn't the case, we'll need a reproduction to investigate.

jakeleventhal commented 5 months ago

This is how the tsc task is defined:

    "tsc": {
      "dependsOn": ["//#package-deps", "^tsc"],
      "inputs": [
        "!.next/**",
        "!.vercel/**",
        "!coverage/**",
        "!dist/**",
        "!out/**",
        "!next-env.d.ts",
        "!node_modules/**",
        "!localDBData/**",
        "!uploadedFiles/**",
        "**/*.js",
        "**/*.jsx",
        "**/*.ts",
        "**/*.tsx",
        "tsconfig*.json"
      ],
      "outputMode": "new-only",
      "outputs": ["dist/**"]
    }

Please let me know if this appears to be a failure in usage, or if I should create a minimal repro

jakeleventhal commented 5 months ago

@anthonyshew it seems this issue is resolved if I remove inputs altogether, and that may be the right approach for me. But it isn't clear to me why the "!" wildcard for "dist" was ignored

jakeleventhal commented 5 months ago

@anthonyshew using your approach with "$TURBO_DEFAULT$" seems to not work when adding excludes. it seems to just ignore the !.

"inputs": ["$TURBO_DEFAULT$", "!**/*.svg"]

I am seeing the inputs being used are now including all svg files, including from gitignored directories. i believe this may be its own bug

anthonyshew commented 5 months ago

I've just verified this bug with create-turbo on 1.13.4-canary.2. It looks like negative globs are not being accounted for.

If I use --no-daemon, the inputs appear to be respected correctly.

vercel / turborepo