Open JJ opened 6 years ago
For the record (from #216):
The failure reason cannot reasonably be obtained without a structured test parse, so upon failure the only way to (currently) include any hints at the failure is to show all the output. If being run with --verbose or --debug the test output is already shown, and hence the hints would already exist. Without --verbose it feels messy to dump the entire test suite output.
The problem with structured test parses (so zef can say "test X and Y failed") will be worked out eventually, but consider that zef will test 1 of 3 different ways depending on what is available: using perl6, using prove -e 'perl6', and using TAP::Harness. While TAP::Harness gives us a lot of power to do cool things, the majority of systems will still be using the first 2 options which dictates a pretty basic interface (basically :out, :err, :exitcode). This happens to be enough to reliably know -if- something failed and send the expected report to cpantesters (the bare minimum requirement).
I -think- the solution for now that would work is instead of dumping output to screen, dump it to a file and report the name of the file:
- Handle the reporting (notifying the user) in Zef::CLI (so users of Zef::Client can handle things their own way) - this is fairly obvious for zef test . (similar to your PR, just inside the CLI module instead) but zef install . may make the control flow a tad harder to follow along. This https://github.com/ugexe/zef/blob/222c626efd764fc64e97ac3db1860216520ab391/lib/Zef/CLI.pm6#L742-L763, notably the reporter stuff, is the correct way/spot to handle both zef test and zef install.
- Use https://github.com/ugexe/zef/blob/master/lib/Zef/Service/P6CReporter.pm6 as a template for a Zef::Service::LocalReporter - use this to do the actual saving to file from Zef::CLI.
- Don't worry too much about making it a plugin (runtime loading, config file stuff... can be handled later). Just use the module in Zef::CLI
This is related to #214 below. I can't figure out what's the failure, only there's a failure.
Context
zef installs itself, it just informs of version and identity when it's failed
Expected Behavior
Include reason for failure in the test message
Actual Behavior
A terse Aborting due to test failure: zef:ver<0.1.32>:auth (use --force-test to override) informing of the failure.
Steps to Reproduce
See @214. It fails in that environment.
Your Environment
Perl6 freshly installed with rakudobrew build moar and rakudobrew build zef