Please let me exclude some directories when running `bazel test //...`

[math4tots]

Description of the problem / feature request / question:

I need a command that will let me exclude certain directories when running bazel test //....

I'm working on a project that involves a ton of Go code that won't play nice with bazel and needs glide. The non-Go code in Scala uses bazel. Now our Go code depends on https://github.com/coreos/docs/tree/master/golang and it causes a BUILD file to get created under the vendor directory, that has nothing to do with bazel.

This messes up bazel test //..., and I can't use it anymore to test everything.

bazel test //... --deleted_packages=vendor doesn't work either, since bazel tries to parse all the BUILD files first, even the ones I'm trying to include.

The error message I get:

Kyumins-MacBook-Pro:source math4tots$ bazel test --deleted_packages=vendor //...
ERROR: /Users/math4tots/gopath/src/github.com/ascend-io/source/vendor/github.com/coreos/pkg/BUILD:3:4: syntax error at 'build': expected newline.
ERROR: /Users/math4tots/gopath/src/github.com/ascend-io/source/tools/build_rules/prelude_bazel:2:6: file '@io_bazel_rules_scala//scala:scala.bzl' was not correctly loaded. Make sure the 'load' statement appears in the global scope in your file.
ERROR: /Users/math4tots/gopath/src/github.com/ascend-io/source/tools/build_rules/prelude_bazel:3:6: file '@//tools/build_rules:grpc_proto.bzl' was not correctly loaded. Make sure the 'load' statement appears in the global scope in your file.
ERROR: /Users/math4tots/gopath/src/github.com/ascend-io/source/vendor/github.com/coreos/pkg/BUILD:3:1: name 'go' is not defined.
ERROR: package contains errors: vendor/github.com/coreos/pkg.
ERROR: /Users/math4tots/gopath/src/github.com/ascend-io/source/vendor/github.com/coreos/go-oidc/BUILD:6:1: invalid character: '$'.
ERROR: /Users/math4tots/gopath/src/github.com/ascend-io/source/vendor/github.com/coreos/go-oidc/BUILD:8:1: invalid character: '$'.
ERROR: /Users/math4tots/gopath/src/github.com/ascend-io/source/vendor/github.com/coreos/go-oidc/BUILD:5:6: syntax error at '"building bin/oidc-example-app..."': expected newline.
ERROR: /Users/math4tots/gopath/src/github.com/ascend-io/source/vendor/github.com/coreos/go-oidc/BUILD:6:10: syntax error at '}': expected :
ERROR: /Users/math4tots/gopath/src/github.com/ascend-io/source/vendor/github.com/coreos/go-oidc/BUILD:9:6: syntax error at '"done"': expected newline.
ERROR: /Users/math4tots/gopath/src/github.com/ascend-io/source/tools/build_rules/prelude_bazel:2:6: file '@io_bazel_rules_scala//scala:scala.bzl' was not correctly loaded. Make sure the 'load' statement appears in the global scope in your file.
ERROR: /Users/math4tots/gopath/src/github.com/ascend-io/source/tools/build_rules/prelude_bazel:3:6: file '@//tools/build_rules:grpc_proto.bzl' was not correctly loaded. Make sure the 'load' statement appears in the global scope in your file.
ERROR: /Users/math4tots/gopath/src/github.com/ascend-io/source/vendor/github.com/coreos/go-oidc/BUILD:5:1: name 'echo' is not defined.
ERROR: /Users/math4tots/gopath/src/github.com/ascend-io/source/vendor/github.com/coreos/go-oidc/BUILD:6:2: contains syntax error(s).
ERROR: /Users/math4tots/gopath/src/github.com/ascend-io/source/vendor/github.com/coreos/go-oidc/BUILD:9:1: name 'echo' is not defined.
ERROR: package contains errors: vendor/github.com/coreos/go-oidc.
ERROR: error loading package 'vendor/github.com/coreos/pkg': Package 'vendor/github.com/coreos/pkg' contains errors.
INFO: Elapsed time: 0.114s
ERROR: Couldn't start the build. Unable to run tests.

If possible, provide a minimal example to reproduce the problem:

Start with a working bazel workspace. Create a directory named vendor/foo. Create a file named vendor/foo/BUILD, and add rubbish inside it.

Now try running bazel test //....

Environment info

• Operating System: OS X

• Bazel version (output of bazel info release): release 0.3.2-homebrew

Have you found anything relevant by searching the web? (e.g. GitHub issues, email threads in the bazel-discuss@googlegroups.com archive)

Someone asked something very similar before: https://github.com/bazelbuild/bazel/issues/829

But the solution proposed there (deleted_packages) doesn't work, as demonstrated above.

[meteorcloudy]

Hi @math4tots , can you try bazel test //... --deleted_packages=vendor/github.com/coreos/pkg?

[math4tots]

@meteorcloudy ah thanks, that seems to work

I don't want to push too hard on something where we don't immediately need it...

But this is going to be a bit annoying going forward having to specify this for every new directory that might have this issue right?

Is there a way to ignore all subdirectories of a given directory?

[meteorcloudy]

Yeah, I agree. We should support not only a package, but also all packages under a certain directory. @damienmg WDYT?

[david-german-tri]

(see also #1083)

This would also be useful to us on RobotLocomotion/drake. Our legacy CMake build checks out externals as git submodules. The submodules are underneath the WORKSPACE directory, so if they contain BUILD files, bazel build ... spuriously tries to build them.

[damienmg]

We should definitely do something along.

[haxorz]

@damienmg

hi, i randomly came across this FR just now. we actually already have the meat of this functionality implemented (the original commit was https://github.com/bazelbuild/bazel/commit/7ef96d778a0baf8c4e3a06bf50a4651e3e8bf1fe), it's currently unused in bazel. it's used internally at google to blacklist certain directories we don't want to index.

BlacklistedPackagePrefixesValue [1] contains a set of directories which and whose subdirectories (i) be traversed at when doing 'build foo/...' (ii) considered to have packages. that is, the label '//foo/bar:baz' will not be considered a valid target if 'foo' is blacklisted.

BlacklistedPackagePrefixesValue is produced from the contents of a file in the workspace whose path is hardcoded into bazel. you can see in [2] that it's hardcoded to be nothing, but you could imagine changing since into a constant agreed-upon path in the workspace (perhaps under 'tools/'?) or into a value set by a user flag.

let me know if you have any questions.

[1] https://github.com/bazelbuild/bazel/blob/65a15eda1b077827e3f70df01f5cfe2ddc96ea59/src/main/java/com/google/devtools/build/lib/skyframe/BlacklistedPackagePrefixesValue.java [2] https://github.com/bazelbuild/bazel/blob/65a15eda1b077827e3f70df01f5cfe2ddc96ea59/src/main/java/com/google/devtools/build/lib/skyframe/SequencedSkyframeExecutor.java#L172

[pcj]

I have trouble specifically with nested WORKSPACE files.

For example, if I have a top-level WORKSPACE with a nested foo/WORKSPACE whose workspace name is tastic, the command bazel build //... from the top-level will traverse into foo/bazel-tastic/external/bazel_tools/tools/build_defs/pkg and fail with various error about missing BUILD files...

Seems like any folder with a WORKSPACE file should be blacklisted (or did I generalize this too far?)

[damienmg]

nested WORKSPACE should be ignored if they are included in the parent workpsace by way of local_repository. /cc @katre

[katre]

@pcj To re-iterate what @dmg just said, there are two pieces to the logic at play with WORKSPACE files in your project:

1. During directory traversal (ie, during any kind of //...-like expansion), directories that are local repositories are skipped (you need to always refer to them as @repo_name//...)
2. local repositories are detected by checking first, for the presence of a WORKSPACE file, and secondly, for a local_repository rule pointing to the directory. If either of these isn't true the directory isn't considered as a local directory.

There is a bug (#2841) currently where a local_directory without a WORKSPACE is considered valid. I'm in the process of submitting https://bazel-review.googlesource.com/c/10271/ to make this an error.

[kgreenek]

I have a similar issue to @david-german-tri. Our project has some sub-directories that use cmake to pull external projects (one of which is grpc). When doing "bazel test //...", it tries to build the files in that sub-directory. I would really like a way to ignore a whole directory, and all of its sub-directories.

We also have a sub-directory at the top of the repo called "experimental", where developers are allowed to submit whatever they want (working or not). This whole directory should be ignored when running "bazel test //...", and I'm not sure how to achieve that currently

[MarkusTeufelberger]

Happy birthday issue #2460!

I also ran into this issue with golang's vendor folders that I want to ignore completely when running bazel test //...

It would be great to be able to ignore a folder AND all its subfolders, because maintaining a list of ignored folders is a bit much...

That being said --deleted_packages is also not the most intuitive name for this functionality.

[jwnimmer-tri]

As katre notes above, a work-around is to:

1. Commit an empty thing/WORKSPACE file.
2. Add to your WORKSPACE file a rule:

   local_repository(
   name = "ignore_thing",
   path = "./thing",
   )

   Then, commands on //... will skip thing and all of its sub-packages.

[MarkusTeufelberger]

local_repository(
  name = "ignore_vendor",
  path = "./thing/vendor",
)

bazel build //thing/... now has lots of lines like this:

ERROR: /home/XXX/thing/BUILD.bazel:3:1: no such package 'thing/vendor/github.com/pkg/errors': Invalid package reference thing/vendor/github.com/pkg/errors crosses into repository @ignore_vendor: did you mean to use @ignore_vendor//github.com/pkg/errors instead? and referenced by '//thing:go_default_library'

The gazelle tool unfortunately doesn't automatically sort these things out...

I do want to access/build files in there if they are necessary, but sometimes they have dependencies in files I don't care about (e.g. I use only a small part of library A, but if I were using its XML parsing capabilities too, it would also depend on library B for that). Normally this is not an issue if I explicitly build/test targets directly, but if I use ... bazel also tries not just to build/test all files that I directly control, it also wants to build/test all BUILD files in the vendor folder.

I want to run bazel test //thing/... --ignore_folder=vendor (to make sure all my tests are being run, but files from the vendor folder should be only used if explicitly depended on by BUILD files that are NOT in the vendor folder). Otherwise I'd need to vendor all transitive dependencies recursively just to make sure some parts of libraries that are not even used by my code(!) also can build...

Maybe in the future go dep will get better and better pruning to just clean out the vendor folder more and more to only have relevant files in there in the first place, for now this is an annoying issue that has a few workarounds but no good solution so far as I understand it.

[mpuncel]

+1 for this, I have a problem where our CI system pushes branches called BUILD whenever a successful build completes. You can probably guess how that interferes with bazel query 'kind(go_test, rdeps("//...", //client:client.go)). --deleted_packages doesn't seem to stop rdeps from trying to analyze the BUILD files in the .git directory

[phedoreanu]

~ » bazel help target-syntax
                                                 [bazel release 0.14.1-homebrew]
Target pattern syntax
=====================

The BUILD file label syntax is used to specify a single target. Target
patterns generalize this syntax to sets of targets, and also support
working-directory-relative forms, recursion, subtraction and filtering.
Examples:

Specifying a single target:

  //foo/bar:wiz     The single target '//foo/bar:wiz'.
  foo/bar/wiz       Equivalent to:
                      '//foo/bar/wiz:wiz' if foo/bar/wiz is a package,
                      '//foo/bar:wiz' if foo/bar is a package,
                      '//foo:bar/wiz' otherwise.
  //foo/bar         Equivalent to '//foo/bar:bar'.

Specifying all rules in a package:

  //foo/bar:all       Matches all rules in package 'foo/bar'.

Specifying all rules recursively beneath a package:

  //foo/...:all     Matches all rules in all packages beneath directory 'foo'.
  //foo/...           (ditto)

  By default, directory symlinks are followed when performing this recursive traversal, except
  those that point to under the output base (for example, the convenience symlinks that are created
  in the root directory of the workspace) But we understand that your workspace may intentionally
  contain directories with weird symlink structures that you don't want consumed. As such, if a
  directory has a file named
  'DONT_FOLLOW_SYMLINKS_WHEN_TRAVERSING_THIS_DIRECTORY_VIA_A_RECURSIVE_TARGET_PATTERN'
  then symlinks in that directory won't be followed when evaluating recursive
  target patterns.

Working-directory relative forms:  (assume cwd = 'workspace/foo')

  Target patterns which do not begin with '//' are taken relative to
  the working directory.  Patterns which begin with '//' are always
  absolute.

  ...:all           Equivalent to  '//foo/...:all'.
  ...                 (ditto)

  bar/...:all       Equivalent to  '//foo/bar/...:all'.
  bar/...             (ditto)

  bar:wiz           Equivalent to '//foo/bar:wiz'.
  :foo              Equivalent to '//foo:foo'.

  bar               Equivalent to '//foo/bar:bar'.
  foo/bar           Equivalent to '//foo/foo/bar:bar'.

  bar:all           Equivalent to '//foo/bar:all'.
  :all              Equivalent to '//foo:all'.

Summary of target wildcards:

  :all,             Match all rules in the specified packages.
  :*, :all-targets  Match all targets (rules and files) in the specified
                      packages, including .par and _deploy.jar files.

Subtractive patterns:

  Target patterns may be preceded by '-', meaning they should be
  subtracted from the set of targets accumulated by preceding
  patterns. (Note that this means order matters.) For example:

    % bazel build -- foo/... -foo/contrib/...

  builds everything in 'foo', except 'contrib'.  In case a target not
  under 'contrib' depends on something under 'contrib' though, in order to
  build the former bazel has to build the latter too. As usual, the '--' is
  required to prevent '-f' from being interpreted as an option.

  When running the test command, test suite expansion is applied to each target
  pattern in sequence as the set of targets is evaluated. This means that
  individual tests from a test suite can be excluded by a later target pattern.
  It also means that an exclusion target pattern which matches a test suite will
  exclude all tests which that test suite references. (Targets that would be
  matched by the list of target patterns without any test suite expansion are
  also built unless --build_tests_only is set.)

(Use 'help --long' for full details or --short to just enumerate options.)

Example:

$ bazel test -- //... -foo/service:acceptance_tests

Also useful: https://stackoverflow.com/questions/49119400/can-i-ignore-some-folder-containing-bazel-configuration-while-building-the-pro

[mpassell]

Neither of the workarounds mentioned by @jwnimmer-tri or @phedoreanu seem to help in my repo - bazel still scans and scans node_modules until it runs out of memory. Any other suggestions?

[ashi009]

We created a read-only FUSE mount of symlinks with all the undesired directory and files excluded to make it possible to use //.... The approach comes with some performance cost, but still manageable.

Given the VFS implementation in bazel, adding a filter should not be that hard. However, I don't think bazel team will implement that anytime soon, as they have much bigger fishes to fry all the time.

[benjaminp]

I wonder if the recent work on #4888 is good enough to close this issue?

[evanj]

I tested 0.17.1rc1 which adds support for .bazelignore files and it definitely solves this problem for me. However, this is blocked by #5974 which causes .bazelignore to break running any specific targets, which makes it unusable for now. However, once that is resolved, I will personally consider this issue closed.

[aehlig]

@evanj, #5974 was fixed by 6f8e36be464829d9cd36e7692653e4cd99d344e9 which is part of 0.18.0rc3. Can you please test this rc and report back, preferrably also on #5963.

[evanj]

@aehlig Confirmed; 0.18.0rc4 using .bazelignore allowed me to ignore our node_modules folder, instead of using our large and growing --deleted_packages flag in tools/bazel.rc. Thanks!

[aehlig]

@aehlig Confirmed

Great, so I think we can finally consider this feature request closed.

[Profpatsch]

1. During directory traversal (ie, during any kind of //...-like expansion), directories that are local repositories are skipped (you need to always refer to them as @repo_name//...)
2. local repositories are detected by checking first, for the presence of a WORKSPACE file, and secondly, for a local_repository rule pointing to the directory. If either of these isn't true the directory isn't considered as a local directory.

Is that explained in the docs somewhere? I have to resort to random DuckDuckGo searches, because the docs are just not usable for even basic use-cases like having nested workspaces. :(

[katre]

@Profpatsch: I'm not even sure where this comment should live, but you are correct that it's not well documented. Please feel free to file additional issues as feature requests to add more documentation.