[CLI] `Gathering metadata for bazel version...` breaks with some `tools/bazel` wrappers

[brentleyjones]

Describe the bug

The Gathering metadata for bazel version... step calls bazel --ignore_all_rc_files help version via tools/bazel, which can fail depending on what tools/bazel does.

To Reproduce

Have a tools/bazel wrapper that adds --config flags to commands. Then run bb in such a way that it needs to collect metadata, e.g. by running a new command like bb version. The tools/bazel wrapper will end up calling something like this:

/Users/brentley.jones/Library/Caches/bazelisk/downloads/sha256/dae351f491ead382bfc7c14d8957b9c8d735300c566c2161e34035eab994c1f2/bin/bazel --ignore_all_rc_files help --config=buildbuddy_cache version

Which will fail because of --ignore_all_rc_files.

Expected behavior

This step probably shouldn't call into the wrapper.

[sluongng]

Currently, I think the order of execution is like this

bb
 -> bazelisk
  -> tools/bazel

In bb CLI layer, we manually parse all the provided user flags and rc files, then translate them into bazelisk call. This lets us fiddle with the canonical flag of the invocation and inject our flags into it, as well as passing the canonical flags to our Plugin systems.

If you are injecting --config= in tools/bazel, this will be un-noticed by bb CLI, which could lead to unexpected behaviors.

With that said, I wonder if we could add a config flag somewhere that would let you opt-in to disabling --ignore_all_rc_files 🤔 . cc: @bduffany

[sluongng]

@brentleyjones Is it possible for you to implement your feature as a pre_bazel bb plugin? https://www.buildbuddy.io/docs/cli-plugins#pre_bazelsh

Then inside tools/bazel, you could detect whether BB CLI is being used to optionally add/skip the --config= flag.

[brentleyjones]

Maybe. But I'm classifying this as a bug because it prevents the CLI from being out of the box with tools/bazel that already exist and work with bazelisk.

[bduffany]

@brentleyjones Are you planning to use the .bazelversion trick for bb distribution or will you run bb directly? I think that might also factor into how we solve this.

Tentatively, I have a proposed fix for this here: https://github.com/buildbuddy-io/buildbuddy/compare/master...cli-toolsbazel

The patch makes sure that we call tools/bazel before canonicalizing / expanding flags, rather than after. However, there are some pros/cons to weigh:

Pros:

• It fixes this bug, and allows passing --config flags in tools/bazel, resolving the incompatibility with vanilla bazelisk.

Cons:

• The proposed fix sets $BAZEL_REAL to point to bb which is a bit weird because bb is not a "real bazel" binary. It also changes the execution model so that the CLI immediately invokes tools/bazel rather than canonicalizing args first. It's not obvious to me what the "right" order of execution should be here, and I'm not sure if these changes will break people or will cause issues in the future.
• Passing --config flags in tools/bazel might be an anti-pattern to begin with - it seems the bazelisk documentation for tools/bazel mentions that it can be useful for setting env vars but doesn't say anything about flags. I would think it would be tricky for a tools/bazel script to modify bazel flags, because they'd have to be aware of the schema for each bazel command.

[brentleyjones]

Are you planning to use the .bazelversion trick for bb distribution or will you run bb directly

We would use .bazelversion. Also, I think that is the main way people will use the CLI in the future, as it doesn't require additional infrastructure to get going.

• Passing --config flags in tools/bazel might be an anti-pattern to begin with - it seems the bazelisk documentation for tools/bazel mentions that it can be useful for setting env vars but doesn't say anything about flags. I would think it would be tricky for a tools/bazel script to modify bazel flags, because they'd have to be aware of the schema for each bazel command.

Sadly lots of wrappers do this, so the ship has sailed there.