No way to run tests in bzlmod dependency if they depend on dev_dependency

davidben commented 4 months ago

Description of the bug:

Based on https://github.com/bazelbuild/bazel-central-registry/pull/1833#issuecomment-2083516837, I gather best practice for bzlmod is to mark your tests' dependencies as a dev_dependency. We've thus tentatively set googletest as a dev_dependency in BoringSSL.

Now, suppose someone consuming BoringSSL wants to run the tests anyway. For example, they may have some issue and we might ask them to run the tests and confirm it's not a toolchain problem. This then fails because the dependency is missing:

Starting local Bazel server and connecting to it...
ERROR: no such package '@@[unknown repo 'googletest' requested from @@boringssl~]//': The repository '@@[unknown repo 'googletest' requested from @@boringssl~]' could not be resolved: No repository visible as '@googletest' from repository '@@boringssl~'
ERROR: /usr/local/google/home/davidben/.cache/bazel/_bazel_davidben/11e4cdfc071b85e46241e062520598c0/external/boringssl~/BUILD.bazel:93:13: no such package '@@[unknown repo 'googletest' requested from @@boringssl~]//': The repository '@@[unknown repo 'googletest' requested from @@boringssl~]' could not be resolved: No repository visible as '@googletest' from repository '@@boringssl~' and referenced by '@@boringssl~//:crypto_test'
ERROR: Analysis of target '@@boringssl~//:crypto_test' failed; build aborted: Analysis failed
INFO: Elapsed time: 4.755s, Critical Path: 0.03s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully
ERROR: No test targets were found, yet testing was requested
FAILED: 
    Fetching repository @@bazel_tools~cc_configure_extension~local_config_cc; starting

The same error occurs if I explicitly pull in googletest from the downstream module. As far as I can tell, there is no way to go back and tell boringssl that I do indeed have a googletest for you.

Which category does this issue belong to?

External Dependency

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

git clone https://boringssl.googlesource.com/boringssl
cd boringssl
git checkout 70d05d5a34f6366116e2b0a530ea8d0186bb2a8e
cd util/bazel-example
bazelisk test @boringssl//:crypto_test
echo 'bazel_dep(name = "googletest", version = "1.14.0.bcr.1")' >> MODULE.bazel
bazelisk test @boringssl//:crypto_test

Which operating system are you running Bazel on?

Linux

What is the output of `bazel info release`?

release 7.1.1

If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

No response

What's the output of `git remote get-url origin; git rev-parse HEAD` ?

https://boringssl.googlesource.com/boringssl
70d05d5a34f6366116e2b0a530ea8d0186bb2a8e

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

Wyverald commented 4 months ago

This is WAI; downstream users aren't expected to be able to run your tests.

We don't want to let boringssl see a googletest pulled in by downstream modules, because that would be akin to circumventing strict deps checks -- behavior could silently and unexpectedly change depending on whether downstream has pulled in googletest.

davidben commented 4 months ago

I see. Running the tests in BCR is not as critical, but being able to run our tests with the downstream toolchain is quite important for triaging issues. Without this, we'll probably need to establish a policy of not accepting bug reports from bzlmod-based projects until they can reproduce it standalone.

davidben commented 4 months ago

I suppose the other option is to make googletest a full dependency, and not dev_dependency. That's probably the best option given the lack of this mechanism.

Note I am not specifically requesting that boringssl pick up the outer googletest. An option like include_dev_dependencies to bazel_dep would also suffice.

Wyverald commented 4 months ago

Note I am not specifically requesting that boringssl pick up the outer googletest. An option like include_dev_dependencies to bazel_dep would also suffice.

This seems like a reasonable enough FR. Or maybe a flag --include_transitive_dev_dependencies that is the inverse of --ignore_dev_dependency, which would allow the end user to run tests of all transitive deps.

bazelbuild / bazel