GoogleChrome / web-vitals

Essential metrics for a healthy site.
https://web.dev/vitals
Apache License 2.0
7.49k stars 410 forks source link

The threshold values for each indicator are set the same for both desktop and mobile devices. Is this reasonable? #399

Closed zengzuo613 closed 10 months ago

zengzuo613 commented 10 months ago

According to statistics from the HTTP Archive or CrUX, we can see that the figures for mobile devices are significantly larger than those for desktop devices. So, is it reasonable to set the threshold values for each indicator the same for both desktop and mobile devices in Web Vitals?

httparchive-dcl

image

The indicator values for mobile devices are almost twice as large.

tunetheweb commented 10 months ago

Hi, you can read more about how the thresholds were defined here: https://web.dev/articles/defining-core-web-vitals-thresholds

In short the thresholds were set based on mobile devices and achievability. There is an argument that we could have stricter thresholds for desktop, and for Lighthouse the limits are set per device category where the device capabilities are more strictly defined through the network and CPU emulation used. For simplicity sake we went with the same thresholds for the field Core Web Vitals where the lines are a little more blurry (many mobile devices are faster than desktop and are used on WiFi networks that may be equal to desktop).

This defines a minimum, achievable threshold that we believe represents “good” experience regardless of device category.

zengzuo613 commented 10 months ago

Thank you for your detailed response, but such settings are too lenient for the desktop. All of our desktop sites, including many sites that were developed many years ago and have not yet been optimized, have an LCP P90 value of 1827.441ms and P75 value of 1072.683ms, which are far greater than the 'good' level. In fact, we believe there’s still a lot of room for optimization on many pages.

Therefore, we are not using the built-in thresholds of Web Vitals for our PC performance evaluation at the moment, but are making judgments on the server side, referencing the 90th percentile line of Lighthouse.

image

Should we consider unifying the standard with Lighthouse, and establish two sets of thresholds that reference CRUX data? This would be more consistent with objective facts.

We use both Lighthouse and Web Vitals , but the differing standards can cause confusion for ordinary developers.

tunetheweb commented 10 months ago

This library does expose the rating value in the metric (good, needs-improvement, poor) but this is a relatively recent change (added last year), and this rating can be ignored and you can set your own thresholds for whatever you want based on the metric value if you want more strict values for some segment of your values.

Changing the wider web-vitals program thresholds is more a question for the web-vitals feedback group rather than just for this library.

zengzuo613 commented 10 months ago

ok tks!