search

treosh / lighthouse-ci-action

Audit URLs using Lighthouse and test performance with Lighthouse CI.
MIT License
1.15k stars 81 forks source link

Assertion is not done on category #93

Open mkathu opened 2 years ago

mkathu commented 2 years ago

I am using lighthouse ci github action of the latest version. My goal is to assert the following:

  1. Metrics
  2. Category (Performance)

I have configured the budget.json(desktop) and mob-budget.json(mobile) for validation of metrics and also I have separate config file for dekstop and mobile.

Below is budget.json image

Below is the desktop config file: image

Below is mob-budget.json image

Below is the mobile config file image

Below is the workflow using above mentioned files: image

Expectation: The lighthouse scan should be done for all pages mentioned in urls and it should asserted for metrics and categories for specified values.

Observed: Only metrics is getting asserted and not category.

Link of one of the execution: https://github.com/mkathu/ul-witelabel-visual-test/runs/4042090535?check_suite_focus=true

Sharing the report of one of the page which does not match the configured performance value of 0.9 for desktop, but not getting asserted. image

Any pointers would be greatly appreciated in making this work.

mkathu commented 2 years ago

any update?

mkathu commented 2 years ago

https://github.com/GoogleChrome/lighthouse-ci/issues/731#issue-1068199904

Any updated on this. I have also asked the same question in lighthouse ci GitHub.

earlman commented 2 years ago

also having this issue