Bazel test runs locally without giving errors when remote executor connection fails

Sowmya-Iyer commented 1 week ago

Description of the bug:

Trying to run Bazel test using a grpc remote executor to test custom bazel-buildfarm server and worker deployed. But, bazel starts locally and runs locally without any explicit error message. First tried it with the original dns and port and the server logs showed no execution. Then tried chenaging the ip to dummy values and the bazel test still runs

siyer@siyer-ThinkPad-T15g-Gen-2i:~/workspace/bazel_training$ USE_BAZEL_VERSION=7.1.2 bazel test --verbose_failures --experimental_circuit_breaker_strategy="failure" --remote_executor=grpc://00.000.00.00:8980 //...
INFO: Invocation ID: 909464f3-f944-41ea-b4d8-4866b56ed84e
INFO: Analyzed 9 targets (0 packages loaded, 0 targets configured).
INFO: Found 8 targets and 1 test target...
INFO: Elapsed time: 0.049s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
PASSED: //src:base_test (see /home/siyer/.cache/bazel/_bazel_siyer/7d82a6405652796670e01aca9a02ed52/execroot/_main/bazel-out/k8-fastbuild/testlogs/src/base_test/test.log)
INFO: From Testing //src:base_test
==================== Test output for //src:base_test:
[==========] Running 3 tests from 3 test suites.
[----------] Global test environment set-up.
[----------] 1 test from SampleClass
[ RUN      ] SampleClass.Initialize
[       OK ] SampleClass.Initialize (0 ms)
[----------] 1 test from SampleClass (0 ms total)

[----------] 1 test from SampleSharedClass
[ RUN      ] SampleSharedClass.Initialize
[       OK ] SampleSharedClass.Initialize (0 ms)
[----------] 1 test from SampleSharedClass (0 ms total)

[----------] 1 test from Lambda
[ RUN      ] Lambda.Class
[       OK ] Lambda.Class (0 ms)
[----------] 1 test from Lambda (0 ms total)

[----------] Global test environment tear-down
[==========] 3 tests from 3 test suites ran. (1 ms total)
[  PASSED  ] 3 tests.
================================================================================
//src:base_test                                                 (cached) PASSED in 0.1s

Executed 0 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.

Would like to know if there is any config or a way to make sure it terminates when the --remote_executor connection cannot be established

Which category does this issue belong to?

No response

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

USE_BAZEL_VERSION=7.1.2 bazel test --verbose_failures --remote_executor=grpc://12.141.11.110:8980 //... (random ip)

Which operating system are you running Bazel on?

Linux

What is the output of `bazel info release`?

release 7.1.2

If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

No response

What's the output of `git remote get-url origin; git rev-parse HEAD` ?

No response

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

Sowmya-Iyer commented 1 week ago

Sometimes running the same test while using my buildfarm worker as IP, I get:

INFO: Found 1 test target...
Target //src:base_test up-to-date:
  bazel-bin/src/base_test
INFO: Elapsed time: 0.857s, Critical Path: 0.81s
INFO: 40 processes: 14 internal, 26 remote.

What is that 14 internal and 26 remote supposed to mean?

iancha1992 commented 1 week ago

@Sowmya-Iyer Could you please provide sample code to reproduce this issue?

bazelbuild / bazel