search

gotestyourself / gotestsum

'go test' runner with output optimized for humans, JUnit XML for CI integration, and a summary of the test results.
Apache License 2.0
2.03k stars 119 forks source link

testname format still prints failure output with --hide-summary=output #328

Open Tadimsky opened 1 year ago

Tadimsky commented 1 year ago

I would like to use the testname format for our tests as we like being able to see each test that passed. However, when a test fails, it prints out all of the output of that test which in our case can be many megabytes of log output.

I've tried suppressing this with --hide-summary=output but it looks like it's not from the summary and is rather part of the testname output.

I'm running these tests in GitLab CI and so whenever there is a test failure GitLab is unable to capture all of the output and so we can't see what actually happened. I'm sending the standard-verbose output to a file so I can look there to see exactly what happened.

Is there some config that I'm missing that will let me do this? Or is this something that testname does not support?

dnephin commented 1 year ago

That you for raising this issue! Currently it's not possible to hide the test failure output with testname.

In #326 we talked about adding a --format-with-fails flag. This flag would be a way to include the test failure output, but I think we could use --format-with-fails=false to hide the output from testname.

I'm sending the standard-verbose output to a file so I can look there to see exactly what happened.

As a workaround until we add the new flag, another way to accomplish this is to use --jsonfile=<filename>. That should let you use the testname format, while still getting the full output in a file. You can turn the jsonfile into the standard-verbose format using:

gotestsum --raw-command -f standard-verbose -- cat jsonfile.log

You could do that before saving the artifact if you want to save the human readable output instead of the jsonfile.

Tadimsky commented 1 year ago

Ok, great! Yeah #326 seems like that would be perfect for us!

Yeah this is kinda what I'm doing right now:

go run gotest.tools/gotestsum --format testname --no-color=false --junitfile unit-tests.xml --jsonfile tests.json -- -vet=off -v -tags=test_unit ./... | grep -v EMPTY
go run gotest.tools/gotestsum --format standard-verbose --raw-command -- cat tests.json > test_unit.log

It's been working great.