[turborepo] Possibility to define ignore patterns - .turboignore and respective property in pipeline configuration

[b12k]

Which project is this feature idea for?

Turborepo

Describe the feature you'd like to request

It would be very handy to be able to ignore directories/files/*.ext globally and per workspace (using .turboignore), per task (using turbo.json).

That will allow more freedom in monorepo structure and more granular control over task execution.

Describe the solution you'd like

.turboignore

Similar to .gitignore in the repo root and per workspace.

Also like .eslintignore, .prettierignore and other common tools do.

turbo.json

{
  "$schema": "https://turborepo.org/schema.json",
  "pipeline": {
    "build": {
      "dependsOn": ["^build"],
      "outputs": ["dist"],
      "ignores": ["infra", "*.tf"]
    },
    "publish": {
      "dependsOn": ["build"]
    }
  }
}

Describe alternatives you've considered

Different monorepo structure.

[mehulkar]

Just to make sure I understand, what would this feature do?

E.g. you can ignore files that Turborepo watches for changes to bust the cache by using the inputs option. I think something this would work:

{
  "$schema": "https://turbo.build/schema.json",
  "pipeline": {
    "build": {
      // all files in src except .tf files are "watched". When they change,
      // `turbo run build` will use a new cache key
      "inputs": ["src/**/*", "!*.tf"]
    }
  }
}

[b12k]

@mehulkar

Regarding .turboignore

It would allow to ignore patterns globally/per-workspace with out a need to touch every pipeline in turbo.json. Folders with docs and architecture decision records are nice to keep alongside with code base they related to. But we do not want them to trigger pipelines - I would ignore them globally.

Regarding inputs

In test monorepo i created following instruction doesnt work:

"inputs": ["*", "!ignored/**/*"]

or any variation of it

"inputs": ["**/*", "!ignored/*"] etc...

IMHO

It would be very handy to be able to include everything (no inputs prop) and explicitly exclude specified files/directories (ignores prop).

[mehulkar]

It would allow to ignore patterns globally/per-workspace

are you wanting to ignore global files (e.g ./README.md at the root of your monorepo)? Or the same set of files in every workspace? e.g. app-a/somefile.txt, app-b/somefile.txt, app-c/somefile.txt).

It would allow to ignore patterns

I'm still not 100% I understand why you mean by "ignore". I may have taken us off course by introducing inputs, since your original question mentioned outputs. For outputs, I think you could mean:

• "I have a lot of apps that create similar outputs, and I don't want to tell turbo to ignore the same set of output files for each one"

  In this case, if all your apps are similar, then your task config should also be similar, so all apps should be describable by a single build config, for example

• "I have a lot of tasks that all produce the same outputs, and I want to ignore the same set of files from those outputs in all tasks"

  Can you describe your scenario more? E.g. I'd imagine that build and lint have different outputs, so I want to understand why different tasks should be able to configure the same patterns.

• "I am configuring lots of apps with app#build syntax in turbo.json and don't want to duplicate anything."

  Check out Workspace Configurations in 1.8

• { "outputs": ["!ignored/**"] } isn't working for me

  A reproduction repo would be great

Could you clarify your use case a little more (apologies if I'm being dense and don't understand 😅 )

---

i created following instruction doesnt work:

On a side note, in 1.9 we'll have some insight into how globs are being expanded: https://github.com/vercel/turbo/pull/3957, so hopefully that helps with this.

[b12k]

I'm not referencing outputs anywhere.

What I mean is to be able to ignore changes of the content of certain files or folders in a pipeline definition in turbo.json.

For example:

• I have a docs folder in every workspace of the monorepo.
• I change files in docs folder it manually (*.md)
• I have a build pipeline which builds an app
  • "outputs": ["dist"]
• I want build to have cache hit even when *.md files in docs folder were changed

What I mean is to be able to specify what should be excluded (not included, or ignored) in cache affecting mechanism.

{
  "$schema": "https://turbo.build/schema.json",
  "pipeline": {
    "build": {
      "outputs": ["dist"],
      "inputs": ["*", "!*.md"] // take everything except *.md files (this doesn't work)
    }
  }
}

OR

{
  "$schema": "https://turbo.build/schema.json",
  "pipeline": {
    "build": {
      "outputs": ["dist"],
      // no inputs prop - take everything
      "ignores": ["*.md"] // ignore *.md files
    }
  }
}

.turboignore and ignores prop - Specifies intentionally excluded files, directories from cache affecting list.

Word ignore is chosen because its common practice. .gitignore - what should not be included in git .eslintignore - what should be excluded from linting .prettierignore - what should not be code formatted

[mehulkar]

"inputs": ["", "!.md"] // take everything except *.md files (this doesn't work)

@b12k I think this is the bug that @gsoltis is fixing in https://github.com/vercel/turbo/pull/4066! I think once that is merged that should give you a way to do what you're looking for.

I'm not referencing outputs anywhere.

My mistake :). I got that impression from the title of this issue: "[turborepo] Possibility to define ignore patterns - .turboignore and similar to outputs configuration"

[b12k]

@mehulkar Oops, take my apologies. Changed the title to be more correct.

[jcollum-nutrien]

I'd like to see some ignore options as well. I have tests in spec.ts files. Most of them are unit tests. But some are e2e tests that won't run locally so they aren't relevant for turbo. I see no way to ignore that folder that contains e2e tests in the docs.

Tried:

    "test": {
      "inputs": ["!e2e/**"],

but it still tries to run e2e tests

[psychobolt]

This feature is needed if we wanted to cache lint task or running lint task only on root workspace e.g.

    "//#lint": {
      "inputs": ["**/*", "!packages/**", "!apps/**"]
    }

Root workspace scripts

  "scripts": {
    "lint": "eslint ."
  }

Running lint on root workspace, should not be a cache miss if for any workspace project change.

[b12k]

👀

[jkjustjoshing]

If the eventual solution for this is to use a negative glob in the inputs array, I'd like to see an explicit example in the documentation on how to write a glob that mirrors the "inputs": [] behavior (take all files) but excludes a file/glob. Most of the negative glob examples I've seen are something like "inputs": ["src/**/*.js", "!src/**/*.test.js"], which doesn't behave at all similarly to [].

Which of these would be correct?

• "inputs": ["*", ...]
• "inputs": ["**", ...]
• "inputs": ["**/*", ...]

I don't know, and since globbing syntax is different everywhere, it's not clear how I can verify without experimenting with Turbo. If the Turbo documentation explicitly made it clear how to do this it would increase my confidence in this feature.

Alternatively (or in addition to), if all input globs are negative, have it first include all files in the workspace like [] does, and then apply the negative globs.

[anthonyshew]

Hey, folks! We were just discussing this Issue amongst a few of us on the core team and we're thinking today's turbo has this concern almost fully taken care of. A few notes:

• Trying to parse some history around this Issue, it looks like there was some combination of bugginess and a fix and/or feature addtion in #4066 that allows for negative globbing. Looks handy for this use case, though not solving everything by itself in the original ask.
• In 1.12, we added support for a $TURBO_DEFAULTS microsyntax that allows you to use Turborepo's default inputs (namely, automatically following your .gitignore) along with your own customizations. I'd encourage trying it out. I've enjoyed using it on a few projects recently and it feels like a great sweet spot for "defaults with control". Documented here!
• @jkjustjoshing's callout was definitely correct that we didn't have a glob spec written and we should have. As fate would have it, I wrote a doc for our globbing syntax in #7076 recently. If it still has gaps, please do let me know with the docs issue template so I can get it tightened up.

Hopefully those three items get us a little closer to the needs in this Issue. Like I said, I think we're close.

Now, some thoughts on the design proposed in this Issue:

• Overcaching can be quite dangerous, especially compared to the tradeoffs of undercaching. Overcaching can mean your pipeline returns, for instance, a green test suite - when it's actually red. Undercaching just means things can be adjusted to go faster. Keep that in mind as I keep going here.
• We always ask "What if you tried to use this feature in a hyper-scaled monorepo? Imagine you had hundreds (thousands?!) of developers in the repository." when we are working through designs. Thinking in terms of "globally ignoring things," this feels like it would be more footgun than feature since we expect a developer building a Turborepo pipeline to start with the default globs (everything except what's .gitignored) in their mental model. I could, conceivably, not know about the .turboignore file in the root and start writing pipelines that overcache. (Danger! Overcaching is suddenly the default in the repository!) On the flipside, if my pipeline is undercaching, I'm likely a few ignored input globs away from making my pipelines faster while maintaining scalable safety for my teammates across the repo.
• However, the heart of the Issue description is correct: Expressing pipelines that are meant to have the same configurations should be easy, should be DRY. Repeating configurations is both annoying to work with and error prone. We see this with other turbo.json keys as well. Folks often find themselves repeating their env arrays over and over.

With those ideas in mind, I have a strawman I've been thinking about for awhile:

// turbo.json
{
  "templates": {
    "inputs": {
      "&&FOO&&": ["!*.md"],
      "&&BAR&&": ["!*.test.*"]
    }
  },
  "pipelines": {
    "build": {
      "inputs": ["$TURBO_DEFAULTS$", "&&FOO&&", "&&BAR&&", "!README.mdx"]
    }
  }
}

Hopefully I can let that code block speak for itself. I imagine I'll be writing an RFC for this in the future to gauge if other folks want this - but I've wanted it personally for quite some time! If anyone reading this doesn't want to wait for me to write it, feel free to beat me to it in a Discussion with the RFC label!

So, with that long-winded reply, I'm going to close this issue since turbo has changed in a few ways that materially impact this issue. If you feel strongly that we're way off here and that this feature request should still be taken into consideration, please do open a fresh RFC in Discussions with the features that have been released in the past year in mind!

[Grsz]

Hi!

I see the documentation has been updated, supporting ! in inputs, but "inputs": [..., "!**/*.test.ts"] still triggers a rebuild when test files are changed in watch mode. I guess it's not expected behaviour.

[anthonyshew]

@Grsz Please open an issue with a reproduction.