search

projectmallard / mallard-ducktype

Parser for the lightweight Ducktype syntax for Mallard
MIT License
10 stars 4 forks source link

Output TAP from tests #17

Closed smcv closed 6 years ago

smcv commented 6 years ago

As far as I can see, this project's current test "API" is:

I've been looking at packaging this project for Debian and I'd like to replace that test "API" with something more suitable for automated build and test environments. TAP seems like a good human/machine-readable text format for output, and I think it would also be helpful for the test script to exit with status != 0 on failure. Does that seem appropriate to you? (Or I can guard this behind a --tap command line option like GLib tests do, if you would prefer.)

This branch converts runtests to produce TAP and an exit status. On failure, it also diffs the expected vs. actual output so that failures can be diagnosed, which is useful in automated build/test environments where everything except the log is going to be discarded after the build/test finishes.

shaunix commented 6 years ago

Non-zero exit status on failure is an obvious improvement. As for the TAP output, that seems fine. If it ends up annoying me, I'll just add a -s option to shut it up. I like that the TAP output shows the diff when a test fails.