Open zcorpan opened 5 years ago
Much of this functionality is provided by the mozlog utility. It's actually very tightly integrated with the WPT CLI itself, and this makes documentation a bit more challenging. Not only do changes need to go through that project's review process and release pipeline, they must also be appropriate for non-WPT users of the utility.
About a year ago, feeling the same confusion that @zcorpan reported above, I tried to improve things in the mozlog utility. The full text now includes instructions for where to find more information:
Output Logging:
Each option represents a possible logging format and takes a filename to
write that format to, or '-' to write to stdout. Some options are provided
by the mozlog utility; see https://firefox-source-
docs.mozilla.org/mozbase/mozlog.html for extended documentation.
--log-unittest LOG_UNITTEST
Unittest style output (provided by mozlog)
--log-wptreport LOG_WPTREPORT
wptreport format
--log-raw LOG_RAW Raw structured log messages (provided by mozlog)
(etc.)
The fact that Simon filed this issue indicates that this note isn't sufficient. We're in better shape today than when Simon requested an improvement because as of gh-16851, the documentation site includes the description of all the CLI arguments. That's probably most useful to folks using the site's built-in "search" feature.
Still, I think the main problem is the vagueness of the statement "provided by mozlog". I've submitted a patch which (among other things) changes that to "refer to the mozlog project for further documentation": https://bugzilla.mozilla.org/show_bug.cgi?id=1574253
https://web-platform-tests.org/running-tests/from-local-system.html says how to run tests but doesn't say how to see the results. Currently when running tests the output in the console is:
I want to know how which tests passed/failed. The docs doesn't say how to get this information. It does say this at the end
From there I can learn about output logging options. I wouldn't know which of the logging formats would be reasonably human-readable, suitable to write to stdout, though.
This could be better documented, and maybe the summary by default could say something more than how many tests ran and were skipped.
cc @jugglinmike