search

ackama / lighthouse-matchers

Provides RSpec matchers for executing and evaluating Google Chrome Lighthouse audit scores
MIT License
14 stars 1 forks source link

Save HTML report when an audit test fails #16

Open joshmcarthur opened 3 years ago

joshmcarthur commented 3 years ago

As a developer

When an audit test fails

I want to be able to view the lighthouse report for the failure

So that I can see the exact reason for the failure

Lighthouse supports multiple output formats - this would be --output=[JSON,HTML] JSON outputs to STDOUT by default - would need to check what it does with HTML output

Ideally we would export the HTML report to a file, while capturing JSON to STDOUT. If we can't do this, we can export both HTML and JSON to a file, and read the file to grab the results. If the audit passes, we can clean up the test artifacts. If the audit fails, we can let the user know where to find the HTML report.

joshmcarthur commented 3 years ago

Confirming that --output json --output html --output-path='lighthouse_results.html' outputs lighthouse_results.report.json and lighthouse_results.report.html - so no STDOUT for JSON anymore, but it would allow us to read the JSON result and discard the files if the audit passes.

RSpec examples have metadata we can use to get information about the test. We should look at how Capybara's save_screenshot gets the filename, since that would be a good one to use.

joshmcarthur commented 3 years ago

Maybe I spoke to soon, Capybara's implementation is pretty basic: https://github.com/teamcapybara/capybara/blob/8c4bffda867c9edc4c5a7b8d37738065775f9945/lib/capybara/session.rb#L876

G-Rath commented 2 weeks ago

@robotdana did something like this in https://github.com/ackama/lighthouse-matchers/pull/32, which I've since cherry-picked out into another branch as it was useful for debugging - it outputs JSON not HTML, but I think it's probably a good place to start