Image rendering stages: finer granularity

WICG / element-timing

A proposal for an Element Timing specification.

https://wicg.github.io/element-timing/

Other

57 stars 14 forks source link

Image rendering stages: finer granularity #54

Open jonsneyers opened 3 years ago

jonsneyers commented 3 years ago

For image loading, I think the following stages are relevant:

Intrinsic dimensions are known (relevant for layout jank)
Placeholder paint (e.g. solid color, gradient, blurhash)
Preview paint (e.g. first full DC scan, embedded preview shown, top 70% of a sequential image)
"Good enough" paint (e.g. far enough in the progressive scan script to be very close to the final image)
Final image

It would be good if these could be distinguished, both for the purpose of metrics and for the purpose of html events.

jyrkialakuijala commented 3 years ago

One possibility is to add say 4 different timing meters. Yes another possibility is to add infinitely many and integrate over them by using a weighting function to get a number that represents overall image rendering utility. It may be easier to optimize image transport systems when there is a single meaningful number that approximate system behaviour.

jonsneyers commented 3 years ago

For a metric, you can compute an integral. For an event, something discrete is needed.

One example is a Javascript-based image carrousel/gallery that needs to know when the next image is available for it to allow starting a transition effect to the next image – if the onload event is used for that, progressive encoding or not does not matter, and it will only trigger when the final image is available, even if the user experience could benefit from being able to see images that are still loading. By exposing various intermediate stages, a smoother ux could be obtained here (while still being able to prevent transitioning to an image that is still completely blank/unavailable).

valiantvu commented 3 years ago

Finer granularity for image rendering stages would be useful at Pinterest - if possible we measure when all the images rendered within the viewport have reached at least 65% quality.

A callback that provides the overall image rendering progress at significant stages would be great. If the stages were defined by industry standards for what is considered "good-enough" rendering, we could potentially align with that if the threshold was high enough (we have seen through experimentation that lowering image quality can significantly impact user engagement, for example, using 474px in place of 736px for our hero images resulted in metrics losses). Many of our images have text in them, which may make standards for image quality higher.