Some web vitals threshold are wrong or possibly out of date

andrea-berling commented 6 months ago

Core or SDK?

Core Sentry product

Which part? Which one?

Web Vitals

Description

Threshold for 2 of the Web Vitals are different from the ones reported on the official web.dev pages that the documentation links to (as of 2024-04-29):

Time To First Byte is reported to have threshold values of 100ms, 200ms, and 600ms for the transition from good to meh and meh to poor. Web.dev reports 800ms and 1800ms for the same thresholds instead: https://web.dev/articles/ttfb#good-ttfb-score
First Contentful Paint is reported to have threshold values of 1 and 3 seconds for the transition from good to meh and meh to poor. Web.dev reports 1.8 and 3 seconds for the same thresholds instead: https://web.dev/articles/fcp#what-is-a-good-fcp-score

Suggested Solution

I suggest you either re-align values to the ones reported on Web.dev, or provide clarification as to why they seem to agree for some vitals and disagree for others in the official docs.

getsantry[bot] commented 6 months ago

Assigning to @getsentry/support for routing ⏲️

vivianyentran commented 4 months ago

@gggritso @edwardgou-sentry Can either of you take a look at this?

edwardgou-sentry commented 4 months ago

Hi @andrea-berling sorry for the confusion. The thresholds table you may have seen is for an older web vitals product that doesn't apply to the current web vitals performance scoring. We will be deprecating this table in the future.

Please see the table here for the current thresholds on our web vital scoring: https://docs.sentry.io/product/insights/web-vitals/#performance-score

Our current thresholds are set to match Google's Lighthouse as close as possible, but we don't make any guarantees to exactly match Google. This is because Lighthouse operates on lab data while Sentry operates on real user data which will have inherent differences in reported data.

Also, the threshold screenshots you linked from Google are for mobile devices which have different values compared to desktop devices. We're still working on supporting mobile devices, so for now we only list our desktop thresholds. You can check this Lighthouse calculator and toggle between Mobile and Desktop to see the differences: https://googlechrome.github.io/lighthouse/scorecalc/#FCP=3000&SI=5800&FMP=4000&TTI=7300&FCI=6500&LCP=4000&TBT=600&CLS=0.25&device=desktop&version=10

Let us know if there's anything we can help clarify!

andrea-berling commented 4 months ago

@edwardgou-sentry thank you for the thorough reply, that clarified things for me 🙂

getsentry / sentry-docs