Suggestions for new report features

[benmccann]

Hi, I maintain a somewhat popular web framework that's included in the reports. I'd like to use the data here to understand how we can have faster sites built with our framework and if there are features we need to add, evangelize, etc. And I'd also like to understand why some frameworks have higher CWVs and whether there's anything we can learn from what they're doing.

Some ideas that would be helpful:

• It would be incredibly helpful to have a list of origins or at least a random sampling of a few dozen each time I run the report. If I can see some actual sites it will give me a lot more insight as I can manually check a number of the things below for a small sample of sites.
• I'd love to see aggregate Lighthouse suggestions. E.g. what percentage of origins are given the feedback "serve images in next-gen formats", "defer offscreen images", "properly size images", "serve static assets with an efficient cache policy", etc.
• Things like number of images, number of DOM elements, number of cookies, website category might help understand things about the types of sites being built with a framework.

[rviscomi]

Thanks @benmccann, those are great suggestions. Would it be helpful if we got you that info in something like a custom report first? That way you can get answers to what you're looking for sooner and we can apply what worked best for you to the dashboard.

[benmccann]

Sure! That would be amazing

[rviscomi]

Cool. Want to set up a time to chat and work out the details? You can email me at rviscomi@google.com.

[rviscomi]

Here's the sample of 100 SvelteKit and Astro sites, including their origin-level CWV performance: https://docs.google.com/spreadsheets/d/1YjIEI-52dFkxczhKyih5-SI8CKssQwHUoGXTeFkfm5Y/edit?usp=sharing

Query (2.37 GB)

```sql WITH crux AS ( SELECT CONCAT(origin, '/') AS root_page, fast_lcp / (fast_lcp + avg_lcp + slow_lcp) AS good_lcp, fast_inp / (fast_inp + avg_inp + slow_inp) AS good_inp, small_cls / (small_cls + medium_cls + large_cls) AS good_cls FROM `chrome-ux-report.materialized.device_summary` WHERE date = '2023-10-01' AND device = 'phone' ), frameworks AS ( SELECT technology, root_page, rank FROM `httparchive.scratchspace.frameworks` WHERE client = 'mobile' AND is_root_page AND technology IN ('SvelteKit', 'Astro') ), sites AS ( SELECT technology, root_page, rank, crux.* EXCEPT (root_page) FROM frameworks JOIN crux USING (root_page) ), sveltekit AS ( SELECT * FROM sites WHERE technology = 'SvelteKit' ORDER BY RAND() LIMIT 100 ), astro AS ( SELECT * FROM sites WHERE technology = 'Astro' ORDER BY RAND() LIMIT 100 ) SELECT * FROM sveltekit UNION ALL SELECT * FROM astro ORDER BY rank, NET.REG_DOMAIN(root_page) ```

[benmccann]

Thanks Rick!! I think I've found the main cause of SvelteKit slowness by looking at the slow sites from the spreadsheet, so no need to spend any more time on this. I'll investigate some more on my side and confirm.

[rviscomi]

Nice! Let us know if there's anything else you need.

FYI @sarahfossheim seems like sample URLs would be useful for the v2 dashboard.

Stashing this custom/LH audit query here for later...

```sql /* https://github.com/HTTPArchive/cwv-tech-report/blob/main/opportunities/README.md */ CREATE TEMP FUNCTION GET_AUDITS(page STRING, lhr STRING, custom_metrics STRING) RETURNS ARRAY> AS ( [ STRUCT( 'LCP' AS metric, 'server-response-time' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."server-response-time".score') AS FLOAT64) IS NULL AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."server-response-time".metricSavings.LCP') AS FLOAT64) AS savings ), STRUCT( 'LCP' AS metric, 'render-blocking-resources' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."render-blocking-resources".score') AS FLOAT64) >= 0.9 AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."render-blocking-resources".metricSavings.LCP') AS FLOAT64) AS savings ), STRUCT( 'LCP' AS metric, 'redirects' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."redirects".score') AS FLOAT64) >= 0.9 AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."redirects".metricSavings.LCP') AS FLOAT64) AS savings ), STRUCT( 'LCP' AS metric, 'uses-text-compression' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."uses-text-compression".score') AS FLOAT64) >= 0.9 AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."uses-text-compression".metricSavings.LCP') AS FLOAT64) AS savings ), STRUCT( 'LCP' AS metric, 'uses-rel-preconnect' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."uses-rel-preconnect".score') AS FLOAT64) >= 0.9 AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."uses-rel-preconnect".metricSavings.LCP') AS FLOAT64) AS savings ), STRUCT( 'LCP' AS metric, 'uses-rel-preload' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."uses-rel-preload".score') AS FLOAT64) IS NULL AS passing, NULL AS savings ), STRUCT( 'LCP' AS metric, 'font-display' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."font-display".score') AS FLOAT64) >= 0.9 AS passing, NULL AS savings ), STRUCT( 'LCP' AS metric, 'unminified-javascript' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."unminified-javascript".score') AS FLOAT64) >= 0.9 AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."unminified-javascript".metricSavings.LCP') AS FLOAT64) AS savings ), STRUCT( 'LCP' AS metric, 'unminified-css' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."unminified-css".score') AS FLOAT64) >= 0.9 AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."unminified-css".metricSavings.LCP') AS FLOAT64) AS savings ), STRUCT( 'LCP' AS metric, 'unused-css-rules' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."unused-css-rules".score') AS FLOAT64) >= 0.9 AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."unused-css-rules".metricSavings.LCP') AS FLOAT64) AS savings ), STRUCT( 'LCP' AS metric, 'largest-contentful-paint-element' AS audit, NULL AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."largest-contentful-paint-element".metricSavings.LCP') AS FLOAT64) AS savings ), STRUCT( 'LCP' AS metric, 'prioritize-lcp-image' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."prioritize-lcp-image".score') AS FLOAT64) >= 0.9 AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."prioritize-lcp-image".metricSavings.LCP') AS FLOAT64) AS savings ), STRUCT( 'LCP' AS metric, 'unused-javascript' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."unused-javascript".score') AS FLOAT64) >= 0.9 AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."unused-javascript".metricSavings.LCP') AS FLOAT64) AS savings ), STRUCT( 'LCP' AS metric, 'efficient-animated-content' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."efficient-animated-content".score') AS FLOAT64) >= 0.9 AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."efficient-animated-content".metricSavings.LCP') AS FLOAT64) AS savings ), STRUCT( 'LCP' AS metric, 'lcp-lazy-loaded' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."lcp-lazy-loaded".score') AS FLOAT64) >= 0.9 AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."lcp-lazy-loaded".metricSavings.LCP') AS FLOAT64) AS savings ), STRUCT( 'INP' AS metric, 'third-party-facades' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."third-party-facades".score') AS FLOAT64) IS NULL AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."third-party-facades".metricSavings.TBT') AS FLOAT64) AS savings ), STRUCT( 'INP' AS metric, 'bootup-time' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."bootup-time".score') AS FLOAT64) IS NULL AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."bootup-time".metricSavings.TBT') AS FLOAT64) AS savings ), STRUCT( 'INP' AS metric, 'mainthread-work-breakdown' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."mainthread-work-breakdown".score') AS FLOAT64) IS NULL AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."mainthread-work-breakdown".metricSavings.TBT') AS FLOAT64) AS savings ), STRUCT( 'INP' AS metric, 'duplicated-javascript' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."duplicated-javascript".score') AS FLOAT64) >= 0.9 AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."duplicated-javascript".metricSavings.TBT') AS FLOAT64) AS savings ), STRUCT( 'INP' AS metric, 'legacy-javascript' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."legacy-javascript".score') AS FLOAT64) >= 0.9 AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."legacy-javascript".metricSavings.TBT') AS FLOAT64) AS savings ), STRUCT( 'INP' AS metric, 'viewport' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."viewport".score') AS FLOAT64) >= 0.9 AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."viewport".metricSavings.INP') AS FLOAT64) AS savings ), STRUCT( 'CLS' AS metric, 'layout-shift-elements' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."layout-shift-elements".score') AS FLOAT64) IS NULL AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."layout-shift-elements".metricSavings.CLS') AS FLOAT64) AS savings ), STRUCT( 'CLS' AS metric, 'non-composited-animations' AS audit, JSON_VALUE(lhr, '$.audits."non-composited-animations".scoreDisplayMode') = 'notApplicable' AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."non-composited-animations".metricSavings.CLS') AS FLOAT64) AS savings ), STRUCT( 'CLS' AS metric, 'unsized-images' AS audit, SAFE_CAST(JSON_VALUE(lhr, '$.audits."unsized-images".score') AS FLOAT64) >= 0.9 AS passing, SAFE_CAST(JSON_VALUE(lhr, '$.audits."unsized-images".metricSavings.CLS') AS FLOAT64) AS savings ), /* Custom audits */ STRUCT( 'LCP' AS metric, 'custom-lazy-load' AS audit, ARRAY_LENGTH(`httparchive.fn.GET_LCP_LAZY`(JSON_QUERY(custom_metrics, '$.performance')).custom) = 0 AS passing, NULL AS savings ), STRUCT( 'LCP' AS metric, 'custom-high-priority' AS audit, `httparchive.fn.GET_LCP_PRIORITY`(JSON_QUERY(custom_metrics, '$.performance')) = 'high' AS passing, NULL AS savings ), STRUCT( 'LCP' AS metric, 'custom-statically-discoverable' AS audit, JSON_VALUE(custom_metrics, '$.performance.is_lcp_statically_discoverable') = 'true' AS passing, NULL AS savings ), STRUCT( 'LCP' AS metric, 'custom-is-image' AS audit, JSON_VALUE(custom_metrics, '$.performance.lcp_elem_stats.url') != '' AS passing, NULL AS savings ), STRUCT( 'LCP' AS metric, 'custom-same-host' AS audit, NET.HOST(JSON_VALUE(custom_metrics, '$.performance.lcp_elem_stats.url')) = NET.HOST(page) AS passing, NULL AS savings ) ] ); WITH audits AS ( SELECT technology, audit FROM `httparchive.scratchspace.frameworks`, UNNEST(GET_AUDITS(page, lighthouse, custom_metrics)) AS audit WHERE client = 'mobile' AND is_root_page AND /* Only look at pages failing the corresponding CWV metric. */ CASE WHEN audit.audit = 'LCP' THEN SAFE_CAST(JSON_VALUE(payload, '$._CrUX.metrics.largest_contentful_paint.percentiles.p75') AS NUMERIC) > 2500 WHEN audit.audit = 'CLS' THEN SAFE_CAST(JSON_VALUE(payload, '$._CrUX.metrics.cumulative_layout_shift.percentiles.p75') AS NUMERIC) > 0.1 WHEN audit.audit = 'INP' THEN SAFE_CAST(JSON_VALUE(payload, '$._CrUX.metrics.interaction_to_next_paint.percentiles.p75') AS NUMERIC) > 200 END ) SELECT technology, COUNT(0) AS sites, audit.metric, audit.audit, COUNTIF(audit.passing) / COUNT(0) AS pct_passing, APPROX_QUANTILES(IF(audit.passing, NULL, audit.savings), 1000 IGNORE NULLS)[OFFSET(500)] AS median_savings FROM audits GROUP BY technology, metric, audit ORDER BY sites DESC, metric, pct_passing ```

[benmccann]

To share my findings, the performance characteristics of SPA vs SSR apps is very different. While many frameworks only support one or the other, SvelteKit can run in different modes and supports both. It quite hard to compare SvelteKit to an SSR-only framework or SPA-only framework because our numbers are grouped together despite these being totally different deployment modes.

It would also be interesting to split SvelteKit on https://cwvtech.report into SvelteKit SSR and SvelteKit SPA. We can differentiate because the single page apps generally have a very small number of DOM elements before client-side rendering kicks in as pictured below. I'm not sure if that's enough to report on. I could also look into generating a meta tag or something to show what mode it's operating in. I think it would also allow us to highlight to our users the performance impacts of picking one vs the other.

Getting a full list of all SvelteKit URLs that appear on https://cwvtech.report would also be helpful to do some further investigations.

[benmccann]

Just checking in on this, do you think it'd be possible to split SvelteKit in the report into SvelteKit SSR and SvelteKit SPA?

[rviscomi]

Yeah that sounds doable from our side. A meta tag would definitely be a more idiomatic way to differentiate between modes, if possible. When available, you could let us know here or open a PR against the Wappalyzer detections directly to get it implemented.

[benmccann]

Thanks. Will do! It'll probably have to wait a couple months so that we can make the change as part of a major version. For now, I've filed an issue against the 3.0 milestone so that we don't lose track of it: https://github.com/sveltejs/kit/issues/11724