treosh / lighthouse-ci-action

Audit URLs using Lighthouse and test performance with Lighthouse CI.
MIT License
1.15k stars 82 forks source link

Make LH Scores available as output parameters #41

Open staabm opened 4 years ago

staabm commented 4 years ago

Would be great to have the main LH Scores available as output parameters in the github action.

That way it would be highly flexible so you could e.g.

alekseykulikov commented 4 years ago

It's a great idea 👍

How do you see a possible output format? It needs to refer to multiple URLs, and we can set outputs dynamically. My idea:

  perf: ${{ steps.lighthouse.outputs.results-0-performance }}
  bp: ${{ steps.lighthouse.outputs.results-0-best-practices }}

Don't you mind providing a PR for this idea, that we could start exploring the best options?

staabm commented 4 years ago

thx for your fast feedback.

your suggestions reads plausible

  perf: ${{ steps.lighthouse.outputs.results-0-performance }}
  bp: ${{ steps.lighthouse.outputs.results-0-best-practices }}

I could also think of something along the lines (not sure this is valid syntax)

   perf: ${{ steps.lighthouse.outputs.results[0].performance }}
   bp: ${{ steps.lighthouse.outputs.results[0].best-practices }}

Don't you mind providing a PR for this idea, that we could start exploring the best options?

sorry, I dont have any experience with github actions

Berkmann18 commented 4 years ago

That would be awesome to have, especially since the artefacts aren't that practical to easily compare the master scores (+ average using, say lh-avg) and the ones from a PR.

paulirish commented 4 years ago

@magus also proposed this over here: https://github.com/treosh/lighthouse-ci-action/issues/2#issuecomment-622027310
In a subsequent comment he gave an example: https://github.com/magus/lighthouse-ci-output/blob/master/.github/workflows/ci.yml ... and here's the CI logs

I like this since it just allows people to do whatever with the data. I haven't yet seen output be used on other actions I'm consuming but it feels composable and good.


Thinking out loud, the most Lighthouse-y way would be to expose the JSON data that's sitting in the .lighthouseci directory: image

But that sorta doesn't work, due to how the LHRs are named by timestamp. @patrickhulce what do you think makes sense here?

My first thought is starting with

output: {
  assertionResults,
  lhrs: [...],  // have to iterate and inspect requestedUrl to find your target
  links,
  flags,
}

One downside here is that the "median run" pick isn't part of this data.

patrickhulce commented 4 years ago

I think what you're looking for here is something similar to the "manifest" output requested in target=filesystem feature.

https://github.com/GoogleChrome/lighthouse-ci/issues/142#issuecomment-579830878

the output would look something like this then?

output: {
  reports: [
    {
      "url": "http://localhost:8080/path/to/page",
      "representative": true,
      "extension": "html",
      "filePath": "./localhost_8000-path_to_page-2020_01_30_15_12_12.report.html",
      "summary": {"performance": 0.52, "accessibility": 0.79, "seo": 1, "best-practices": 0.23}
    },
  ]
}
staabm commented 4 years ago

I like this since it just allows people to do whatever with the data. I haven't yet seen output be used on other actions I'm consuming but it feels composable and good.

Its the native GithubAction way to provide a composable output. Its the way to go.

https://help.github.com/en/actions/building-actions/metadata-syntax-for-github-actions#outputs