(Why) do we need test-setup.sh?

[EdSchouten]

Description of the feature request:

All tests are currently launched through a shell script named test-setup.sh. Even though that works, I've observed that it is somewhat of an impediment if you're dealing with slightly non-standard use cases.

For example, test-setup.sh has a #!/usr/bin/env bash at the top, meaning that you can only run tests on platforms that support/ship with Bash.

Furthermore, when replaying a previously failed test, it's a bit annoying to get tests to run under tracers/debuggers, as they all need to 'jump over' the shell script, so to speak.

My question is, why do we need test-setup.sh in the first place? Are there certain things that it does that could also just be solved by calling into SpawnRunner/the remote execution system differently? For example, this code is only setting an environment variable, which we can also do trivially through other means:

# Declare that the executable is running in a `bazel test` environment
# This allows test frameworks to enable output to the unprefixed environment variable
# For example, if `BAZEL_TEST` and `XML_OUTPUT_FILE` are defined, write JUnit output
export BAZEL_TEST=1

Or are there genuine features that are missing? If so, could we maybe work with the remote execution working group to get those features added to the protocol?

One example I can think of is this:

# Bazel sets some environment vars to relative paths to improve caching and
# support remote execution, where the absolute path may not be known to Bazel.
# Convert them to absolute paths here before running the actual test.
is_absolute "$TEST_PREMATURE_EXIT_FILE" ||
  TEST_PREMATURE_EXIT_FILE="$PWD/$TEST_PREMATURE_EXIT_FILE"

Maybe we could extend REv2's Command message to have a dedicated field/feature for this?

What underlying problem are you trying to solve with this feature?

Increase portability across different remote execution platforms. Make it easier to being able to debug tests.

Which operating system are you running Bazel on?

n/a

What is the output of bazel info release?

n/a

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

n/a

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

n/a

Have you found anything relevant by searching the web?

I see that someone filed an issue a couple of hours ago to add automatic injection of timestamps to test logs: #18973 But this information is also provided as part of REv2's ExecutedActionMetadata. Bazel could just display that information if needed.

Any other information, logs, or outputs that you want to share?

No response

[Jonpez2]

Getting rid of it would have the nice side effect of solving this too :)

https://github.com/bazelbuild/bazel/issues/18859

[github-actions[bot]]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 90 days unless any other activity occurs. If you think this issue is still relevant and should stay open, please post any comment here and the issue will no longer be marked as stale.

[EdSchouten]

any comment here