Question about rankings methodology

[jaredgorski]

Looking at these two ranking details side-by-side leaves me a bit curious as to the real methodology behind these rankings.

Ranked #2	Ranked #71

As you can see, the statistics for the website ranked #71 match or beat the rankings for the #2 site in all displayed categories. Intuitively, I would think that #71 would be considered a higher performing site than #2.

However, I'm aware that the #71 site uses Zeit/Vercel rather than Netlify, so I'm hypothesizing that there may be some internal artificial preference for sites hosted on Netlify?

This isn't an urgent question/issue, but it's got me curious. I'd love to get any insight.

[aanomm]

hello beautiful humans 👋🏾

• i have some curiosities about speedlify too
• i've noticed when testing very unperformant sites (namely eccom sites; shopify, woocomm, etc) that my speedlify instance shows shocking performance result numbers (often <10)
• but if i test the same sites via lighthouse directly the performance scores are drastically higher (while seo, best practices, and seo all score the same as my speedlify)
• if anyone has some insight i'd love to know why

[zachleat]

The other runs on speedlify.dev may exhibit some Netlify bias as those are running on Netlify servers however the Eleventy Leaderboards specifically have too many sites to do that in the 15m build limit so I run them manually from my laptop. The algorithm seems to be operating as designed and is documented here https://www.zachleat.com/web/eleventy-leaderboard-speedlify/

Note that the third tiebreaker is speed index to page weight ratio. So all other things equal a 2KB site and a 500KB site with the same speed index will prefer the 500KB site.

[jaredgorski]

Interesting. Thanks for the response. I'll check the algorithm out.

You mention that the speed to page weight ratio is a tiebreaker, which makes sense intuitively, but I'm assuming it's a bit more than a tiebreaker based on the two-site comparison I made in this issue. I'd venture a guess that this ratio is actually considered in the ranking metrics, notwithstanding a tie that needs breaking. In other words, performance per byte is an influential metric. Is this correct?

[zachleat]

A question of perspective, I think. It depends entirely on how competitive the board is. The Eleventy Leaderboards specifically are very competitive. But the other leaderboards I’m running (on speedlify.dev) are not.

[zachleat]

Hope that answers your questions!

[AleksandrHovhannisyan]

I think Axe is what's throwing things off. I too have noticed that my site has been ranking progressively lower and lower on the 11ty Leaderboards, despite using less data over time (now only ~70 kB), not having any Lighthouse issues, and often loading in just ~0.5s. To put this into perspective, my site uses about 10 times less data than most of the sites that rank in the top three, and I have similar if not better metrics in other areas.

I ran a manual Axe test locally, and it flagged issues that are not really issues 😕 For example, my home page links to my /art/ page, but it also links to /tags/art/ for individual posts that are about art. Axe flags this under the premise that Links with the same name must have a similar purpose, which imo is silly because it's clear from the context what those two links are about. Moreover, it seems to take issue with the anchor-pseudo-element design pattern from Inclusive Components: https://inclusive-components.design/cards/. It flags this as a contrast issue because Element's background color could not be determined because it partially overlaps other elements.

I suppose it's more appropriate to report these to Axe devtools, but I'm also unsure how reliable the leaderboards are if one of the tools has false positives.

[AleksandrHovhannisyan]

Follow-up to this since I just realized that Speedlify has links to the Lighthouse calculators as well:

My site's scores for Aug 10

No. 1 site's scores for Aug 10

No. 2 site's scores for Aug 10

I still suspect one or more tools (likely Axe per comment above) are giving false positives that are swaying the rankings.

Here's a more detailed breakdown between my site and the no. 1 site:

No. 1

LCP | Max Potential FID | TBT | CLS | FCP | SI | TTI -- | -- | -- | -- | -- | -- | -- 1.65s | 18ms  | 0ms | 0.00 | 0.99s | 0.99s | 0.99s Total | HTML | CSS | JS | Images | Fonts | Third Party -- | -- | -- | -- | -- | -- | -- 603 KiB | 3 KiB | 4 KiB ×1 | 4 KiB ×1 | 541 KiB ×10 | 50 KiB ×3 | 5 KiB ×3

My site

LCP | Max Potential FID | TBT | CLS | FCP | SI | TTI -- | -- | -- | -- | -- | -- | -- 1.20s |  16ms |   0ms   | 0.00   | 828ms   | 828ms   | 828ms Total | HTML | CSS | JS | Images | Fonts | Third Party -- | -- | -- | -- | -- | -- | -- 78 KiB | 6 KiB | 6 KiB ×1 | 2 KiB ×2 | 7 KiB ×1 | 58 KiB ×2 | 0

I don't get it 😕

[zachleat]

Hey @AleksandrHovhannisyan unfortunately your site got smaller which negatively affected your ranking 😭 https://www.11ty.dev/speedlify/aleksandrhovhannisyan-com/

Note that the third tiebreaker is speed index to page weight ratio. So all other things equal a 2KB site and a 500KB site with the same speed index will prefer the 500KB site.

https://github.com/zachleat/speedlify/issues/34#issuecomment-689294185

[AleksandrHovhannisyan]

Oh, my bad, I should've probably read the thread more carefully.

Brb, gonna add Google Analytics to bump up my score 😛