ugexe / zef

Raku Module Management
Artistic License 2.0
207 stars 44 forks source link

Include more information on failure #215

Open JJ opened 6 years ago

JJ commented 6 years ago

This is related to #214 below. I can't figure out what's the failure, only there's a failure.

Context

zef installs itself, it just informs of version and identity when it's failed

Expected Behavior

Include reason for failure in the test message

Actual Behavior

A terse Aborting due to test failure: zef:ver<0.1.32>:auth (use --force-test to override) informing of the failure.

Steps to Reproduce

See @214. It fails in that environment.

Your Environment

Perl6 freshly installed with rakudobrew build moar and rakudobrew build zef

ugexe commented 6 years ago

For the record (from #216):

The failure reason cannot reasonably be obtained without a structured test parse, so upon failure the only way to (currently) include any hints at the failure is to show all the output. If being run with --verbose or --debug the test output is already shown, and hence the hints would already exist. Without --verbose it feels messy to dump the entire test suite output.

The problem with structured test parses (so zef can say "test X and Y failed") will be worked out eventually, but consider that zef will test 1 of 3 different ways depending on what is available: using perl6, using prove -e 'perl6', and using TAP::Harness. While TAP::Harness gives us a lot of power to do cool things, the majority of systems will still be using the first 2 options which dictates a pretty basic interface (basically :out, :err, :exitcode). This happens to be enough to reliably know -if- something failed and send the expected report to cpantesters (the bare minimum requirement).

I -think- the solution for now that would work is instead of dumping output to screen, dump it to a file and report the name of the file: