deadfoxygrandpa / elm-test

A unit testing framework for Elm
MIT License
204 stars 21 forks source link

Add test runner that outputs TAP13 format #41

Open fapian opened 8 years ago

fapian commented 8 years ago

Hey,

this PR adds a console runner that outputs TAP13 formatted test results (https://testanything.org/tap-version-13-specification.html).

Might help with #34

Here's a sample sample_tap_output

If there is anything that I can do to get that merged I will be happy to help.

klaftertief commented 8 years ago

This is cool.

I started implementing a TAP runner as well, but wanted to support indented output for nested suites. Some TAP-to-xunit-transformer can preserve the hierarchy. But it I git stuck in I guess some numbering issues. And the nesting is not in the actual TAP specification.

That being said, did you consider writing a nested output? In the current output you cannot easily see to which suite a test belongs.

What I'm not liking that much is that formatting is bound to the console output and that the public API has to change. I understand that this needs to be this way at the moment because the stringRunner, which could be used a starting point for a different output type (as mentioned in the docs), does not output the test results in success cases.

So my main question is if adding new formatter/runner like in this PR is the way to go or if elm-test should add better support (opening up its internal API) for custom formatting? I think it would be cool if such customisations (I mean adding the TAP format) could live in user land.

Sorry for hijacking this PR.

fapian commented 8 years ago

To be honest, I thought the same as I was working on it. I think to do that, the ElmTest.Run module should be made ​​accessible.

That being said, did you consider writing a nested output?

I'll try it.

fapian commented 8 years ago

Here's the new output

screen shot 2016-03-07 at 19 17 21