stevesouders
commented
7 years ago Thanks for sharing this. Fun!
=== 1. I think the goal is good:
There are an increasing number of cases where pages render reasonably quickly (yay Speed Index) but the page is unusable because the javascript framework is being hooked up and blocks the main thread for multiple seconds.
I get it. So you want a new metric that merges visual completion AND main thread availability. But regarding this quote: How did you determine that the page is unusable? Is it anecdotal?
=== 2. Interactive is hard to define:
interactive being defined as a time window of > 500ms where no task blocks the main thread for more than 50ms
Is "task" the main categories of scripting, painting, layout, loading? If so, that makes sense. How many total "tasks" are there? I assume it's less than 10 so even if all of them are right at 50ms that still leaves at least a 50ms non-blocking window in the 500ms.
=== 3. You say:
Speed Index is the floor
Seems like SI is the BEST score possible, and the score can only get worse when you factor in the main thread. So I guess it's true that SI is the "floor" in the sense that the score will be greater than or equal to SI, but "floor" usually means "worse" whereas here it's "best". I might say "Speed Index is the best score possible. The new metric's value will only get worse as lack of interactivity is factored in."
=== 4. The first example, of WPT itself, doesn't seem logical. Below I show the main thread with SI and ISI indicated. It "feels" like if SI happened at 2450ms, and then there was a lot of blocking, ISI should be around 3200ms (after SI when there's a significant free up of the main thread). What I'm suffering from is my desire to have a "point in time" like "Time to Interactive". But what you're calculating is sometime similar to SI which captures the visual & interactive behavior across the ENTIRE page load. So the metric is larger than 3200ms because of the big non-interactives at 3800ms and 5700ms. I just don't know if I feel the ISI should be penalized by these later non-interactive times. Would the user have really noticed them? When do you stop?
=== 5. Is it really best to merge these two metrics - visual progress and interactivity? If you combine too many metrics, the number stops having any relevance to the real world, hides important information, and is hard for humans to interpret. What if SI and "main thread blocking" were separate metrics? For CNN, SI is 12 seconds and Interactive is 55 seconds. That tells me that my problem isn't rendering - it's JS. So I actually know more quickly what I should focus on.
=== 6. I think a BIGGER problem is that pages do NOT render quickly because they're blocked by 1) critical blocking resources that download slowly or 2) JS execution that blocks the main thread. To further complicate the situation, the JS execution may come from a synchronous script or an async/defer script. Developers need help figuring out:
- their rendering is being blocked
- whether it's from slow downloads or JS execution
- if JS execution, was that a sync script or async/defer
There's a yin-yang here of wanting metrics that indicate there's a problem, and wanting other metrics that help developers pinpoint the problem. Maybe SI and some new "interactive" metric help indicate if there's a problem that the USER notices - lack of visual progress or inability interacting with the page. Then we might want some other new metrics like:
- "critical downloads complete" - the time at which the last critical blocking resource finished downloading
- "render blocking JS" - the number of milliseconds that the browser main thread is blocked by JS parsing & execution before the page is visually complete(?). This could be split into sync js and async/defer js.
zeman
pmeenan
nilskuhn
addyosmani
sburnicki
jaroslawrosiek
github-actions[bot]
There are an increasing number of cases where pages render reasonably quickly (yay Speed Index) but the page is unusable because the javascript framework is being hooked up and blocks the main thread for multiple seconds.
Lighthouse has a Time to Interactive audit that grades for it and WebPageTest needs some way to also account for it.
My current thinking is to use the visual progress and Speed Index as a base and modify it based on how interactive a page is.
Specifically:
Benefits:
Downsides:
Visual Examples
WebPageTest
This test is of the WebPageTest main page (mostly static) on a Moto G.
From the filmstrip we can see the visual progress used for the Speed Index calculation:
Speed Index is calculated as the area above the visual progress, resulting in a Speed Index of 2458:
If we look at the main thread browser activity (below the waterfall) and identify the periods where the main thread can be considered "interactive" (currently defined as a 500ms window with no tasks that block the main thread for more than 50ms) we get something that looks like this:
Then, we take the visual progress and only count areas in the graph where the main thread is also interactive and the other areas count as 0% progress, we re-calculate the area over the graph and get something closer to 4450:
CNN
A much more extreme case where the content is ready pretty fast visually but there is a TON of script blocking the page activity is CNN.
In this case most of the visual content loads around the 12 second mark and produces a Speed Index of 12235:
If We look at the main thread activity though, there are ~ 6 seconds of interactivity spread out after the initial render until you get to the 50 second mark where the tasks don't block the main thread to block interactivity:
In this case, the modified area above the graph would be closer to 44000 and better represent how poor the experience is.