search

Raku / tap-harness6

A TAP harness for Raku
Artistic License 2.0
8 stars 15 forks source link

It would be nicer if prove6 hid output of todo failures #17

Closed szabgab closed 7 years ago

szabgab commented 7 years ago

We have a test marked todo in Bailador. prove6 displays the failure as follows:

$ prove6 -l t/30-examples-app.t
t/30-examples-app.t ..1/15
    # Failed test 'route GET /hello/Foo.html'
    # at t/30-examples-app.t line 68
    # expected: $[200, [:Content-Type("text/html")], "Hello Foo.html!"]
    #      got: $[404, [:Content-Type("text/html;charset=UTF-8")], "<html>\n    <head>\n        <title>Custom 404 page</title>\n        <meta charset=\"UTF-8\">\n    </head>\n    <body>\n        <h1>Hello, this is 404 for you.</h1>\n    </body>\n</html>\n"]
    # Looks like you failed 1 test of 2
t/30-examples-app.t .. ok
All tests successful.
Files=1, Tests=15,  4 wallclock secs
Result: PASS

I think this makes people, especially people new to perl* uneasy as the failure takes up a lot of the reporting and the final success is less obvious. I just checked and prove of Perl 5 does the same, but I wonder if it wouldn't be more user-friendly if the default was to hide the error from todo tests and only show them in verbose mode.

In addition IMHO the final report should also say "1 todo"

Our repo is https://github.com/Bailador/Bailador

Leont commented 7 years ago

That output has nothing to do with todos, and everything with diagnostics. By default all output to stderr is untouched by prove6, and this is the most sensible thing it can do. The real issue is in Test.pm6, which should redirect diagnostics to stdout instead of stderr when inside of a subtest (this is what Test::Builder does on perl5).

szabgab commented 7 years ago

Oh thanks. So if I understand correctly I need to send this request to rakudobug.