Broken C++ Coverage with hermetic clang toolchain - No override for COVERAGE_GCOV_PATH and LLVM_COV possible with --test_env

[omar-droubi]

Description of the bug:

Hello Everyone,

Starting with Bazel 7.2.1 it's no longer possible to generate c++ code coverage using a hermetic Clang toolchain. I am using pw_toolchain_bazel to generate the toolchains definitions. Since our system doesn't have GCOV installed, and would like to use LLVM for the coverage, we override COVERAGE_GCOV_PATH and LLVM_COV with --test_env:

bazelrc content:

test --test_env=COVERAGE_GCOV_PATH=external/_main~_repo_rules~llvm_toolchain/bin/llvm-profdata
test --test_env=LLVM_COV=external/_main~_repo_rules~llvm_toolchain/bin/llvm-cov

By default Bazel extracts these two env variables from the tool_path for GCOV and LLVM_COV, but not from the action_configs for coverage actions.

cc_helper.bzl@1090

def _get_coverage_environment(ctx, cc_config, cc_toolchain):
    if not ctx.configuration.coverage_enabled:
        return {}

    env = {
        "COVERAGE_GCOV_PATH": cc_toolchain.tool_path(tool = "GCOV"),
        "LLVM_COV": cc_toolchain.tool_path(tool = "LLVM_COV"),
        "LLVM_PROFDATA": cc_toolchain.tool_path(tool = "LLVM_PROFDATA"),
        "GENERATE_LLVM_LCOV": "1" if cc_config.generate_llvm_lcov() else "0",
    }

Given that the current recommend way is to use action_configs and not tool_paths to define Action Tools, we do not set the tool path for GCOV and LLVM_COV, and their value is set to \"\".

This was fine since we could override their values using --test_env. However after the change 87b0a1f202992ac98f73bc230551b6166c313a06 we can no longer override them.

This is happening because of the change in the order of env variables population in src/main/java/com/google/devtools/build/lib/exec/TestPolicy.java.

// Overwrite with the environment common to all tests, see --test_env.
// Omar: Here COVERAGE_GCOV_PATH and LLVM_COV take the correct values from --test_env
testAction.getConfiguration().getTestActionEnvironment().resolve(env, clientEnv);

// Rule-specified test env.
// Omar: getExtraTestEnv() has already COVERAGE_GCOV_PATH and LLVM_COV values defined 
// by cc_helper and they are set to "". These values overwrite what was supplied using --test_env
testAction.getExtraTestEnv().resolve(env, clientEnv);

Proposed Solutions:

1. _get_coverage_environment in cc_helper.bzl should use get_tool_for_action instead of tool_path.
2. Remove setting to default if tools are not found in _get_coverage_environment.
3. Explicit filtering for COVERAGE_GCOV_PATH, LLVM_COV, LLVM_PROFDATA,GENERATE_LLVM_LCOV in resolve function.

Which category does this issue belong to?

No response

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

No response

Which operating system are you running Bazel on?

WSL Ubuntu 20.04

What is the output of bazel info release?

release 7.2.1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

87b0a1f202992ac98f73bc230551b6166c313a06

Have you found anything relevant by searching the web?

No

Any other information, logs, or outputs that you want to share?

No response

[fmeum]

@bazel-io flag

[fmeum]

@c-mita

[iancha1992]

@bazel-io fork 7.4.0