bitfield / gotestdox

A tool for formatting Go test results as readable documentation
https://bitfieldconsulting.com/golang/test-names
MIT License
114 stars 4 forks source link

Added a verbose flag which makes gotestdox print the output of tests. #13

Closed Jumziey closed 1 year ago

Jumziey commented 1 year ago

Just recently found this little gem, really makes it way more fun to code and easier to to make focused test cases.

One issue I had though is that I usually only develop with tests and the outputs from the code and tests weren't displaying when running the tests. Which is essential when you hit a snag and want to debug. One alternative is to just go back to the good ol' go test tool without gotestdox, but that isn't as easy and pretty to parse at a glance and hides the test name in that TestCamelCase.

So I added a -v flag that behaves similarly to go test -v.

I like feedback in any form so go ham if you'd like, lived on the internet for too long to get offended, even if you don't like the concept to begin with.

bitfield commented 1 year ago

Thanks, @Jumziey! It's maybe worth reviewing this discussion on issue #4:

gotestdox is explicitly not a tool for formatting the output of your tests. It presumes that your tests are passing, and tells you what the names of your tests assert about the behaviour of your program. Its only concession to the idea that some tests might fail is to exit with a non-zero status, to stop CI pipelines.

That said, I've noticed that a lot of people seem to like using gotestdox as their principal test runner, and it would be useful to report the failure messages for failing tests, so that's something I'm considering. For example:

foo:
  ✔ Bar (0.00s)
  x Baz (0.00s)
    --- FAIL: TestBaz (0.00s)
        shift_test.go:116: want [84 104 105 115 32 109 101 115 115 97 103 101 32 105 115 32 101 112 97 99 116 108 121 32 51 50 32 98 121 116 101 115], got [84 104 105 115 32 109 101 115 115 97 103 101 32 105 115 32 101 120 97 99 116 108 121 32 51 50 32 98 121 116 101 115]
Jumziey commented 1 year ago

Thanks for the quick response! Yeah I understand if you have a different purpose for the tool and this PR really does not seem to fit into it.

It's a neat idea to let the tests print the output in the way you describe. I don't see how you could only get the failure message for the failing test without wrapping it in some tooling specific way, but just by default printing the output when the test is failing is a really good idea!

On the subject of purpose there's some real social/organizational benifits of having the same tools you develop with being the tool that reports successes in CI pipelines and the like, so hope you might reconsider this in the future :) (not to mention I personally like it, which personally I find most important ;p)

Really like what you've done here! :muscle: I'll close this PR and continue on my own fork in the mean time, but feel free to reach out whenever :)

bitfield commented 1 year ago

Maybe a good compromise would be to add support for the gotestdox output format to https://github.com/gotestyourself/gotestsum, which is a general-purpose Go test runner. If you're interested in taking that on, great!

Jumziey commented 1 year ago

I hadn't even considered it, thank you for the input! I'll look into that.

bitfield commented 1 year ago

@Jumziey notwithstanding adding gotestdox-ness to gotestsum, which I still think is a good idea (the maintainers may take a different view), I've now added output from failing tests in v0.2.0. Let me know if you find it useful!

Jumziey commented 1 year ago

Yeah I've actually added dox format on my own fork and are trying it out a bit before doing a PR, I'm still a bit unsure in what way the gotestdox-ness fits best into gotestsum. I like the approach quite a lot more then just a fork of gotestdox(but as you elude to, the maintainers might not like it).

I looked into the change and I liked it :) I still think it scans better having the filename/number on a different line then the output, especially if its run in a small terminal/window etc. But this does not seem to map to the philosophy of gotestdox and go test, besides... simply just adding the output is technically quite clean.

Jumziey commented 1 year ago

@bitfield I was just about to set out looking into this again and to my suprise you've done an amazing work getting the testdox format into gotestsum already (https://github.com/gotestyourself/gotestsum/pull/359). Thanks! Really appreciate it ^^