bazel-contrib / rules_go

Go rules for Bazel
Apache License 2.0
1.39k stars 664 forks source link

unclear how to consistently get all of the log output from tests #4158

Open jmhodges opened 3 weeks ago

jmhodges commented 3 weeks ago

What version of rules_go are you using?

0.50.1

What version of gazelle are you using?

0.39.1

What version of Bazel are you using?

7.3.1

Does this issue reproduce with the latest releases of all the above?

Yes.

What operating system and processor architecture are you using?

macOS arm

What did you do?

Tried to run all of the tests in a package without caching with verbosity turned up to see all of the t.Logf lines in it.

I tried both with -test.v, and test.count and without:

$ bazel test --cache_test_results=no --test_output=all //foobar/baz:baz_test
$ bazel test --cache_test_results=no --test_output=all --test_arg="-test.v" //foobar/baz:baz_test 
$ bazel test --cache_test_results=no --test_output=all --test_arg="-test.count=1" //foobar/baz:baz_test 
$ bazel test --cache_test_results=no --test_output=all --test_arg="-test.count=1" --test_arg="-test.v" //foobar/baz:baz_test 

What did you expect to see?

All of the tests enumerated and all the log lines run by the tests like the usual go test -v command.

Output like:

$ go test -v .
=== RUN   TestGoldenPath
    baz_test.go:23: I'm here

What did you see instead?

The versions with -test.v or -test.count always just print (with --cache_test_results=no --test_output=all included, of course) will only get you the

//foobar/baz:baz_test                      PASSED in 1.0s

Dropping those and only run bazel test --cache_test_results=no --test_output=all will get you only:

==================== Test output for //foobar/baz:baz_test:
PASS
================================================================================

with the usual

//foobar/baz:baz_test                      PASSED in 1.0s

after it

jmhodges commented 3 weeks ago

(Updated the summary for clarity and with gazelle 0.39.1 testing)

fmeum commented 3 weeks ago

What's the content of the test log file under bazel-testlogs in these cases?

jmhodges commented 3 weeks ago

For bazel test --cache_test_results=no --test_output=all, it's the same single line of PASS. Specifically:

exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //foobar/baz:baz_test
-----------------------------------------------------------------------------
PASS
bazel-testlogs/foobar/baz/baz_test/test.log (END)

However, the bazel test --cache_test_results=no --test_output=all --test_arg="-test.v" does output the log lines to the test log. I'm surprised by that. I would have expected test_output=all (esp with -test.v) to put the output to stdout & stderr. Perhaps I'm wrong to be surprised by that, but as day-to-day Go hacker, I was.

jmhodges commented 1 week ago

Oh, interesting, if you use a fmt.Printf, and drop the --cache_test_results=no but leave the --test_output=all, you will get output. Still doesn't change the t.Logf results though. Hunh! That is:

$ bazel test --test_output=all //foobar/baz/...

You can get the t.Logf by adding the --test_arg="-test.v", but, again, leaving out the --cache_test_results=no