🐞 Significant performance issue when loading Qwik pages using 4g Slow on basic interactions

JaxCavalera commented 2 months ago

Which component is affected?

Qwik Runtime

Describe the bug

I have noticed that there's a problem with the way Qwik currently tries to lazy load scripts. It seems to be bundling way more into a streamed (on demand) script than it should. When you run this simulating 4G Slow (which is most of Australian mobile internet at best) it's clear that the delays on very basic user interactions are simply unacceptable, and there's no visual indicator anything is happening.

Reproduction

https://qwik.dev/

Steps to reproduce

Go to the reproduction URL
In dev tools - change network speed to 4G Slow
Clear Cache and Hard Reload - Keep the network tab in dev tools open
Wait for the page to fully load what it needs
Click on the Search panel at the top header bar
Count how long it is taking for the actual search modal to appear
Notice there's no indication anything is happening in between these 2 events

System Info

System:
    OS: Windows 10 10.0.19045
    CPU: (32) x64 Intel(R) Core(TM) i9-14900K
    Memory: 46.24 GB / 63.75 GB
  Binaries:
    Node: 20.14.0 - C:\Program Files\nodejs\node.EXE
    npm: 10.7.0 - C:\Program Files\nodejs\npm.CMD
    pnpm: 7.18.1 - ~\AppData\Local\pnpm\pnpm.EXE
    bun: 1.1.10 - ~\.bun\bin\bun.EXE
  Browsers:
    Edge: Chromium (127.0.2651.74)
    Internet Explorer: 11.0.19041.4355

Am actually using Chrome latest to test this
'Version 128.0.6613.85 (Official Build) (64-bit)'

Additional Information

No response

maiieul commented 2 months ago

Hi @JaxCavalera this is a misleading behavior of "empty cache and hard reload" on your browser, not the reality your users will experience: https://qwik.dev/docs/advanced/speculative-module-fetching/#misleading-empty-cache-and-hard-reload.

To see what happens on 3G / slow 4G, you need to

open an incognito navigation
open the dev tools
go to network tab
set network throttling to 3G or slow 4G
wait for everything to load properly
click -> it should be instantaneous.

JaxCavalera commented 2 months ago

@maiieul Actually it's exactly what users will experience the first time they reach the page (first impressions matter a lot), and it's also what Google measures when it tests for CWV. A service worker will have nothing cached for a first time user so that linked guide is incorrect.

maiieul commented 2 months ago

Actually it's exactly what users will experience the first time they reach the page (first impressions matter a lot) ... A service worker will have nothing cached for a first time user so that linked guide is incorrect.

On first page load, the service worker will request all the chunks used in the page and add them to the cache using Cache API. Have you tried the steps I mentioned above?

it's also what Google measures when it tests for CWV.

qwik.dev scores a 100 on pagespeed insights: https://pagespeed.web.dev/analysis/https-qwik-dev/gzbu9khzos?form_factor=mobile

JaxCavalera commented 2 months ago

On first page load, the service worker will request all the chunks used in the page and add them to the cache using Cache API. Have you tried the steps I mentioned above?

Yep doing those steps of course gives a much better result because everything remains in the service worker cache, however the problem is that none of this is in there originally, you can't expect an end user to wait the full 1+ seconds before they start wanting to interact with your web applications. (In my testing it was more like 3 seconds however I can see pagespeed has it at an average of 1s.

As per the link to Pagespeed it shows that TTFB is not passing, which is exactly my point

The overall score is solid, however there's clearly a problem here on first load. If a user tries to interact on first load before every single asset has been loaded into the service worker cache you will get a poor user experience as per the original post.

To be fair this is probably a better case scenario the qwik home page is pretty clean, not a lot of elements in there etc.

maiieul commented 2 months ago

In my testing it was more like 3 seconds however I can see pagespeed has it at an average of 1s

This is a different problem than the first one you pointed out. This is true and not related to a "empty cache and hard reload" gotcha.

Note that slow 3G in your network tab is simulated throttling, so it might not show how a real 3G connection would behave on qwik.dev. Also qwik.dev users probably have good connections even on mobile, hence why 1s instead of 3s.

But in any case I agree that 1s/3s delay on first page load first user interaction is not ideal. However, it's far better than the usual 20s on hydration based frameworks, and I don't think any other interactive framework/approach can get it any better. If you have multiple javascript-based interactive components on a page, the browser will have to download this javascript whatever the tool you use, and this takes time. For the case of qwik.dev, it probably takes about 3s to load the framework's js (~20kb) when there's a visible task or something, + the component's js (~5kb) + SW + fonts, etc, on slow 3G.

Also we haven't spent much time improving those scores, so maybe it can be improved to about 1s if it's something like fonts slowing down page load + interactions.

As a side note, Qwik has "qwik insights" for big apps which need the chunks of the most likely to be interacted components on the page to be prefetched in priority. Also, in case a chunk hasn't been loaded yet, I think Qwik can prioritize fetching it as soon as the user interacts with it.

Last nit pick on my end, you're right that our TTFB isn't exceptional, see the scores for qwik.dev in its entirety (with quite a lot of interactive examples on many pages) 👇

but does it matter? 😛 : https://web.dev/articles/ttfb#good-ttfb-score

Key point: Because TTFB isn't a Core Web Vitals metric, it's not absolutely necessary that sites meet the "good" TTFB threshold, provided that it doesn't impede their ability to score well on the metrics that matter.

Websites vary in how they deliver content. A low TTFB is crucial for getting markup out to the client as soon as possible. However, if a website delivers the initial markup quickly, but that markup then requires JavaScript to populate it with meaningful content—as is the the case with Single Page Applications (SPAs)—then achieving the lowest possible TTFB is especially important so that the client-rendering of markup can occur sooner.

Conversely, a server-rendered site that does not require as much client-side work could have a higher TTFB, but better FCP and LCP values than an entirely client-rendered experience. This is why the TTFB thresholds are a "rough guide", and will need to be weighed against how your site delivers its core content.

TLDR: performance is more than just network tab slow 3G or core web vitals, in the end what matters is the final user experience and the cost to get there. I believe Qwik delivers better than any other solution on that. Sure you can find benchmarks that will show better scores than Qwik but it will likely be very specific and uncomparable in terms of DX.

maiieul commented 2 months ago

P.S. for a good comparison with other frameworks, you can go on qwikui.com on slow 3G with your network tab in a new incognito navigation. First interactions right after page load will indeed take ~3s, but if you go to any other hydration-based framework's component libraries, you'll find it rather takes ~20s. You can search for the shadcn equivalents for example.

QwikDev / qwik