danielroe / page-speed.dev

https://page-speed.dev
MIT License
320 stars 13 forks source link

Website that tests exactly at CWV #21

Closed dylanmhowell closed 6 months ago

dylanmhowell commented 6 months ago

Description

I've noticed a discrepancy in how the tool evaluates Google's Core Web Vitals scores, specifically in cases where a site's performance is exactly on the threshold between passing and failing. In these borderline cases, the tool marks the site as failing, whereas Google's own evaluation considers it passing. This difference could lead to misunderstandings or misinterpretations of a site's performance based on Core Web Vitals.

Alternatively, it looks like page-speed is already taking INP into consideration while Google isn't until March.

Steps to Reproduce

  1. Run the tool on a site with Core Web Vitals scores exactly on the threshold of passing/failing (for example, a LCP score of 2.5 seconds, FID of 100 milliseconds, or CLS of 0.1).
  2. Observe the evaluation provided by the tool.
  3. Compare the result with Google's Core Web Vitals assessment for the same site.

Expected Behavior

For consistency with Google's standards, the tool should mark sites as passing when their Core Web Vitals scores are exactly on the threshold. This alignment ensures that users receive an accurate assessment of a site's performance in line with Google's evaluation criteria.

Actual Behavior

The tool currently marks sites as failing if their scores are exactly on the passing/failing threshold.

Possible Solution

Adjust the evaluation logic in the tool to treat scores exactly on the threshold as passing, mirroring Google's assessment approach. This could involve reviewing and modifying the conditional checks for Core Web Vitals scores within the tool's codebase.

Screenshots

Google-page-speed kadence-google

tunetheweb commented 6 months ago

The tool is using INP instead of FID. Admittedly this is a little ahead of Google which won’t do this until March, but it’s so close that I for one think it’s better to just use INP straight away for any new tool launched now - including this one.

You can see in the PageSpeed Insights screenshot that is Passes but with a yellow warning triangle indicating it will fail as soon as INP replaces FID. So even the Google tools have started to indicate this as a problem.

(Full disclosure: I work at Google on many of these tools, and have been following this one with interest!).