search

WordPress / theme-review-action

Other
31 stars 10 forks source link

Try: Add a scorecard #59

Open StevenDufresne opened 3 years ago

StevenDufresne commented 3 years ago

DO NOT MERGE.

We are currently printing test results in natural language to the respective action logs. In the future, it will make sense to share the theme results in a format that is easier to consume.

The Idea:

If we could create a scorecard, similar to lighthouse, we will be able to use that information to help authors and the theme directory.

This PR adds logic that prints error tests to a log called theme-scorecard.json in json format.

Example : 

{
"title_check":0,
"page_should_return_200_status":0
}

It isn't very sophisticated at this point and only prints errors as a 0 score. It's a bit tricky because we currently only capture errors & warnings, and therefore the absence of a test result would suggest that it has passed. We could capture passes in the ui-check but I don't think that will work with the theme-check results.

carolinan commented 3 years ago

I think that before solving the problem with how to display positive results (scorecards/flags), there needs to be a discussion and decision about which checks to show results for.