`glob` does not see sources in a `.bazelignore` directory

[UebelAndre]

Description of the problem / feature request:

When using glob on a directory that is specified in .bazelignore, I would expect glob to still give me a list of all sources in this directory.

Feature requests: what underlying problem are you trying to solve with this feature?

My expectations of .bazelignore is that Bazel will ignore the directories specified as potential packages such that BUILD/BUILD.bazel files can live in them and Bazel will ignore them.

Example

I've created an example of this: repro.zip

├── BUILD.bazel
├── WORKSPACE.bazel
├── failures
│   └── BUILD.bazel
└── lib
    ├── BUILD.bazel
    ├── WORKSPACE.bazel
    ├── bar
    ├── baz
    └── foo

2 directories, 8 files

Current State

With the following .bazelignore file, I'm able to build my target :parent but am not seeing the results I'd expect

failures
lib

Below is the an example of the unexpected results

bazel run :parent

[]

:parent is a [genrule]() target with the following implementation

my_glob = glob(["lib/**/*"])

genrule(
    name = "parent",
    outs = ["parent.sh"],
    cmd = """\
cat > "$@" << EOF 
#!/bin/bash
echo "{}"
EOF
""".format(my_glob),
    executable = True,
)

Expected Behavior

With both .bazelignore and BUILD.bazel files in the ignored directory, I still expect to see the source files in said directory:

bazel run :parent

[lib/BUILD.bazel, lib/WORKSPACE.bazel, lib/bar, lib/baz, lib/foo]

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Not sure if this is a bug or not but a repro has been given in the section above

What operating system are you running Bazel on?

MacOS/Linux

What's the output of bazel info release?

release 3.4.1

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

NA

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

NA

Have you found anything relevant by searching the web?

10428 seems relevant

Any other information, logs, or outputs that you want to share?

NA

[laurentlb]

glob doesn't match files in subpackages (subdirectories that contain a build file)

See https://docs.bazel.build/versions/master/be/functions.html#glob:

Labels are not allowed to cross the package boundary and glob does not match files in subpackages

[UebelAndre]

Yeah, I'm suggesting that directories should not be considered subpackages if they're specified in .bazelignore.

[UebelAndre]

I see this got relabeled. Anyone have any idea if/when this will be fixed?

[alexeagle]

I think this is a feature, not a bug. I would want glob to not descend into nested packages nor into ignored directories.

If you just want to mark that a given BUILD file should not be treated as a package indicator you can use --deleted_packages to turn it into a regular file (we do this in rules_nodejs so that our nested workspaces can appear as regular test inputs)

[pauldraper]

The current behavior is...odd to say the least.

.bazelignore

dir

BUILD.bazel

filegroup(
  name = "example1",
  srcs = ["dir/example.txt"]
)

filegroup(
  name = "example2",
  srcs = glob(["dir/example.txt"])
)

How many files do these filegroups have?

The first filegroup has 1 file. The second filegroup has 0 files.

---

Glob ought be the same as explicit listing the files.

(Personally, I'd prefer the deleted_packages-style interpretation, since you can always just add an exclude to glob. The problem with just using --deleted_packages is that it's not ergonomic to add the option for many packages.)

[kylecordes]

I think the behavior where srcs = ["dir/example.txt"] sees the file even if dir is .bazelignored is likely a bug. At least it doesn't match the general experience of .bazelignore stopping Bazel from doing anything else with specified directories.

[pauldraper]

As described in Java doc, this is meant to only avoid interpreting BUILD files, and nothing else.

  /**
   * The file .bazelignore can be used to specify directories to be ignored by bazel
   *
   * <p>This is intended for directories containing non-bazel sources (either generated,
   * or versioned sources built by other tools) that happen to contain a file called BUILD.</p>
   *
   * <p>For the time being, this ignore functionality is limited by the fact that it is
   * applied only after pattern expansion. So if a pattern expansion fails (e.g., due to
   * symlink-cycles) and therefore fails the build, this ignore functionality currently
   * has no chance to kick in.</p>
   */

For example, if I want to treat a directory as plain files for Bazel integration testing.

Affecting the behavior of glob() appears to be undocumented and unintentional.

[mvgijssel]

+1 on this feature. This would be very handy for Bazel integration tests, where the integration test could glob all the files in an ignored directory and use that to spin up a bazel invocation inside the test sandbox using those files.

[alexeagle]

@mvgijssel for that case, you can just set --deleted_packages to mark particular BUILD files as not denoting a package boundary. for example: https://github.com/aspect-build/aspect-cli/blob/a9aaa61868ec60b08b0ad47c3cc2a5275c19a2dc/.bazelrc#L12-L15