search

gotestyourself / gotestsum

'go test' runner with output optimized for humans, JUnit XML for CI integration, and a summary of the test results.
Apache License 2.0
1.99k stars 118 forks source link

read "go test" output from stdin #414

Open pohly opened 1 month ago

pohly commented 1 month ago

In Kubernetes, we run fairly complex commands to execute our unit tests. gotestsum is then used for post-processing with --raw-command cat <filename>.

It might be possible to turn those complex commands into a script (I'm trying...) but the shell quoting is nasty.

Would you accept a patch that makes gotestsum read input from stdin?

pohly commented 1 month ago

Work in progress, feedback on command line flags welcome: https://github.com/gotestyourself/gotestsum/pull/415

dnephin commented 1 month ago

Hello! I believe it is possible to run from stdin. The readme has this example:

Example: accept intput from stdin

cat out.json | gotestsum --raw-command -- cat

I've generally suggested against running using stdin for a couple reasons:

  1. you lose the output that is sent to stderr (or force the tool to have to demux both streams from stdout)
  2. if you are not very careful with shell options you lose the status code from the go test process. There have been enough bugs with test2json parsing that relying on the test output for status code is too risky.

That said, I think build failures were added to go test stdout output in a relatively recent release, so maybe capturing stderr isn't as critical anymore. And It also seems like your PR has a new flag to consume that stream, which is great!

The status code I think is still a risk, so I wouldn't want to suggest this approach in general, but for a sufficient complex setup (as I imagine is the case for kubernetes) that's probably a fine tradeoff.

I've been very busy with life outside of github lately, but I will find some time to review your PR. Thank you!

pohly commented 1 month ago

Indeed, these are the two drawbacks. The documentation in https://github.com/gotestyourself/gotestsum/pull/415 addresses both.

For reference, this is the shell code which I found too hard to convert into a generated script:

https://github.com/kubernetes/kubernetes/blob/ef9965ebc66dafda37800bb04f5e284535bbba10/hack/make-rules/test.sh#L254-L266

A bash script which currently contains a bash script would have to be turned into a bash script which generates a bash script which contains a bash script...

pohly commented 1 month ago

I'm checking whether we want to go the --raw-command <script> route after all (https://github.com/kubernetes/kubernetes/pull/125534/files#r1642283167).