Closed mdwhatcott closed 4 years ago
@dmitshur possible release-blocker?
Can you take a look at https://github.com/golang/go/issues/24929#issuecomment-558768304. Is this issue different from that or the same?
/cc @jadekler @ianlancetaylor
Not trying to be antagonistic, but why is this unexpected behavior? These are tests running in parallel, and their output is arriving as it happens rather than buffered and presented in-order at the end. That seems to be WAI.
(I may be missing something - typing from the car :) )
This is due to https://golang.org/cl/127120, for issue #24929.
This is documented in the release notes: search for "Testing" under https://tip.golang.org/doc/go1.14#tools.
The right thing happens with go test -json
, which is the intended mechanism for programs that parse test output.
So this change is intentional. The question is: should we roll it back? What are the bad effects of this change on your system? Can they be ameliorated by using go test -json
instead?
CC @rsc @mpvl
Thanks everyone for chiming in.
@dmitshur - I've reviewed #24929 and can confirm that my issue is a direct result of that issue having been implemented. I understand what that issue is trying to accomplish and see the need for it.
@ianlancetaylor - Thank you for pointing out the explanation in the release notes. I missed that the first time through. Be aware that the "testing" package documentation for the T
interface contradicts what's on the release notes:
T is a type passed to Test functions to manage test state and support formatted test logs. Logs are accumulated during execution and dumped to standard output when done.
I agree in principle with the idea that -v
output is for humans and -json
output is for tools like IDEs. Currently, my IDE of choice (Jetbrains Goland) makes use of the -json
flag in some scenarios, but probably just needs some work to make sure each JSON object is correctly organized and displayed with its proper test in the final report. I'll probably have to take this up as a separate issue on the Jetbrains issue tracker (@zolotov ).
Crazy idea: could we emit log output immediately to stdout AND buffer it for a more tidy presentation at the end (like we've always had)?
Change https://golang.org/cl/219540 mentions this issue: testing: remove obsolete comment in testing.(*T) docs
@ianlancetaylor
The right thing happens with go test -json, which is the intended mechanism for programs that parse test output.
The real reason I filed this issue was because the GoLand test runner wasn't able to handle the new style of output (#24929). I've since learned that it's the output of go test -json
and go tool test2json
are no longer correct in all cases in Go 1.14 and I've filed a subsequent issue: https://github.com/golang/go/issues/38050
@mdwhatcott thank you for filing this issue, thank you Ian, Liam, Jean and Dmitri for the responses! @mdwhatcott I believe that your last statements were acknowledgements of the change that we intentionally made and you filed a subsequent follow-up for the new output being incorrect, on behalf of what you saw in GoLand. Can we close this issue then? Thank you.
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
Consider the following sample program, which runs 10 subtests using
t.Parallel()
:What did you expect to see?
I expected to see the output aligned with the final report for each subtest, which works in go1.13.8 (just released earlier today).
What did you see instead?
Notice that in go1.14rc1 the test log output appears in unexpected places in the overall verbose output. This change in behavior makes the output more difficult to decipher for humans and causes test log output to be lost when scanned by test runners in editors and IDEs (such as Intellij GoLand).
This feels very much like a newly-introduced bug.