search

GoogleChrome / lighthouse-ci

Automate running Lighthouse for every commit, viewing the changes, and preventing regressions
Apache License 2.0
6.41k stars 644 forks source link

LHCI and PageSpeed Insights results doesn't match #924

Closed Gmantiqueira closed 1 year ago

Gmantiqueira commented 1 year ago

Hello! 👋 My team uses LHCI for auditing ecommerce performance, so we run LHCI autorun in every PR commit checking Home, Product List and Product Page performance. We run 3 times and use the median result.

The problem is that our stakeholders, customers or even ourselves use PageSpeed Insights for comparison and the result doesn't match with LHCI, with ~20 points of difference in performance.

Is there an specific config or parameter that we can use to simulate PageSpeed Insights with LHCI without using the PSI API? I checked that we use the same throttling and configuration and I already tried to disable our CDN, but without success.

Examples:

LHCI: https://googlechrome.github.io/lighthouse/viewer/?jsonurl=https://webops-lighthouse-reports.s3.amazonaws.com/vtex-sites--starter.store_0eba108__.json

Screenshot 2023-07-10 at 10 32 36 AM

PageSpeed Insights: https://pagespeed.web.dev/analysis/https-starter-vtex-app/lrdy1ad1ht?form_factor=mobile

Screenshot 2023-07-10 at 10 35 37 AM

Note: The URLs are equivalents, we use the first one for PR/commits previews and we points to production when merged (PSI URL).

adamraine commented 1 year ago

Is there an specific config or parameter that we can use to simulate PageSpeed Insights with LHCI without using the PSI API?

Reducing performance variance in PageSpeed is an active area of work but unfortunately there isn't a way to simulate the same PageSpeed environment. https://github.com/GoogleChrome/lighthouse-ci/blob/main/docs/troubleshooting.md#the-results-im-seeing-dont-match-the-results-i-see-in-another-environment is probably your best resource for this problem.