bazel test command sometimes not able to render icons (and colour)

[Globegitter]

Description of the problem / feature request:

When running tests via bazel test it seems, in a quite non-deterministic manner, that icons can not be rendered. And when running it again they render fine. Also it is not possible to see any colour in the output (not even when passing --color=yes), but I guess that was implemented on purpose that way?

Feature requests: what underlying problem are you trying to solve with this feature?

Fix the rendering of icons and ideally also allow a way to display the output in is original colour.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Use any testing framework that renders icons or colour in its output. My use-case is with jest and nodejs_test from rules_nodejs utilising https://jestjs.io/ - I could create a github repository for reproduction if necessary.

What operating system are you running Bazel on?

Ubuntu 16.04

What's the output of bazel info release?

release 0.17.0rc1

Have you found anything relevant by searching the web?

No

Any other information, logs, or outputs that you want to share?

This is how the icons look rendered incorrectly:

This is after running bazel test on the same target again.

[philwo]

@Globegitter A repro would be nice, thanks!

[Globegitter]

Interestingly enough I have not been able to reproduce the icon rendering issue recently (it has been very inconsistent in general), the colour issue is consistent though and can be seen in this repo: https://github.com/Globegitter/colour-icon-repro

When running bazel test //:jest_test one can see that the output looks different to running bazel run //:jest_bin, the colour is missing. Imo colours for test output should either be on by default or respect the --color flag.

I will see if I can still reproduce the icon issue though.

[Globegitter]

Aahh interesting, I have managed to get colour output and also no icon issue. Turns out rules_nodejs wraps every test in a sh_test rules, see: https://github.com/bazelbuild/rules_nodejs/blob/master/internal/node/node.bzl#L383 - if I exchange that with

native.sh.binary(
  ...
  testonly=1
)

then everything seems to work, but then bazel test itself is not working anymore.

I have also noticed that the sh_test rules is causing some issues if I want my test to keep on listening to stdin for them to rerun when they receive a message - again fixed with switching to a sh_binary. How come there are these differences and limitation of sh_test? Can these be removed?

[jmmv]

I haven't checked, but the way most programs decide to output color or not is by calling isatty on any of stdin, stdout, or stderr.

When you a run a test from Bazel, stdout and stderr are redirected to files and thus isatty is going to return false for them. I'm not sure what happens with --test_output=streamed, but I think Bazel is also buffering the output, so the same behavior should appear. stdin should be closed so isatty would fail, but I don't know if that's the case, and the behavior here could differ between builds and tests...

Now, about the difference between a first and a second build: remember the Bazel server. It's possible that the first time it starts, the streams are connected to the console differently than when we run a second session against the running server, thus causing different behavior.

So, many things can be happening here, and many things have changed since this was filed.

Is this still a problem?

[jin]

This is still a problem with --test_output=streamed not surfacing gtest log colors.

cc @enh

[lberki]

Uh, this is difficult. We could in theory change Bazel such that tests run under --test_output=streamed have a pseudoterminal instead of a regular file as an output, but then that would introduce a discrepancy between the environment a test gets in streamed and non-streamed mode, which isn't great.

If changing the test to disregard isatty() is not an option, one could write a little tool that wraps the test in a pty (in fact, someone has already implemented that tool: https://github.com/magicant/pseudo-terminal-wrapper). I could imagine (eventually) making that part of Bazel, but I'm not that enthusiastic given that it would be Yet Another Knob.

[heethesh]

This would be an useful feature to have! Helps glance at lengthy test failure logs and pay attention to logs based on color.