Migration needed for automatic exec groups

[rickeylev]

What version of protobuf and what language are you using?

Version: 27.0-rc2 Language: Python

What operating system (Linux, Windows, ...) and version? Linux/Mac/Windows

What runtime / compiler are you using (e.g., python version or gcc version) n/a What did you do?

Build with bazel with --incompatible_auto_exec_groups set. https://github.com/bazelbuild/bazel/issues/24373 has some examples. In rules_python:

git clone rules_python
cd examples/bzlmod
bazel test ... --incompatible_auto_exec_groups

What did you expect to see

Successful build.

What did you see instead?

    File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f0406cbf12177f6cbe046ff7eea9e4d2/external/protobuf~/python/internal.bzl", line 57, column 24, in _internal_copy_files_impl
        ctx.actions.run(
Error in run: Couldn't identify if tools are from implicit dependencies or a toolchain. Please set the toolchain parameter. If you're not using a toolchain, set it to 'None'.

Anything else we should know about your project / environment

This is for Bazel 9 support, see https://github.com/bazelbuild/bazel/issues/24373

[haberman]

Is it possible to reproduce this issue using a command against the protobuf repo directly?

btw are you really testing against 27.0-rc2? If so, could you try 29.0-rc3 instead?

[rickeylev]

I also tried rc3; same error.

It looks like this might have been fixed in 80a5a511e47a33a860b004e3336ba89ab45058f0

That commit says "first step", so not sure if that's all that's needed or not. That commit doesn't say it's part of any released tags, so unless it was cherry picked somewhere, I'm assuming its unreleased.

repro on protobuf itself

here's a copy/paste of what BCR ran to see a failure:

bazel build --show_progress_rate_limit=5 --curses=yes --color=yes --terminal_columns=143 --show_timestamps --verbose_failures --jobs=30 --announce_rc --experimental_repository_cache_hardlinks --disk_cache= --sandbox_tmpfs_path=/tmp --remote_cache=remotebuildexecution.googleapis.com --remote_instance_name=projects/bazel-untrusted/instances/default_instance --google_default_credentials --bes_backend=buildeventservice.googleapis.com --bes_timeout=360s --project_id=bazel-untrusted --remote_timeout=60 --remote_max_connections=200 --remote_default_platform_properties=properties:{name:"cache-silo-key" value:"6eeeb3cb49532a444ab5ce1f5ab0888b174438c1fa027fb539c5a4703b208eed"} --remote_download_toplevel --host_cxxopt=-std=c++14 --cxxopt=-std=c++14 --test_env=HOME --test_env=BAZELISK_USER_AGENT --test_env=USE_BAZEL_VERSION --incompatible_auto_exec_groups -- //...

Basically just looks like test ... with --incompatible_auto_exec_groups

This is from this run: https://buildkite.com/bazel/bcr-bazel-compatibility-test/builds/45#01932622-abd4-4a94-8bb7-32bba363f13e

(the error is a bit hidden, expand the Running Bazel with --incompatible_auto_exec_groups section)

[kotlaja]

@rickeylev yes, https://github.com/protocolbuffers/protobuf/commit/80a5a511e47a33a860b004e3336ba89ab45058f0 fixes this error inside python/internal. I noted that this is the first part of the migration since the next error is:

ERROR: /usr/local/google/home/kotlaja/.cache/bazel/_bazel_kotlaja/3b1d6911411754876f642abf5e6636f0/external/rules_kotlin/kotlin/compiler/BUILD.bazel:22:22: in kt_js_import rule @@rules_kotlin//kotlin/compiler:kotlin-stdlib-js: 
Traceback (most recent call last):
    File "/usr/local/google/home/kotlaja/.cache/bazel/_bazel_kotlaja/3b1d6911411754876f642abf5e6636f0/external/rules_kotlin/kotlin/internal/js/impl.bzl", line 137, column 20, in kt_js_import_impl
        ctx.actions.run(
Error in run: Couldn't identify if tools are from implicit dependencies or a toolchain. Please set the toolchain parameter. If you're not using a toolchain, set it to 'None'.

and this error is coming from rules_kotlin. Interesting info here is that the file causing this error doesn't exist anymore in rules_kotlin 2.0.0. Furthermore, 2.0.0 version of rules_kotlin doesn't fail with --incompatible_auto_exec_groups (see https://buildkite.com/bazel/bcr-bazel-compatibility-test/builds/45#_). Which is why I suggest first trying to upgrade to this version of rules_kotlin and checking if "bazel build ... --incompatible_auto_exec_groups" passes.

Note that rules_kotlin 2.0.0 is a pre-release and it could cause unrelated failures (?) to AEGs.