Closed JoshTumath closed 4 years ago
Hmm, this was previously discussed at https://www.w3.org/Bugs/Public/show_bug.cgi?id=17842, but GitHub is more friendly for people. Let me merge that thread into here, but please please please read all the contents of the discussion there, as this is very well-trod ground and we don't want to have to reiterate the same discussions over again.
I've just spent an hour reading the thread on the original bug report (which @JoshTumath actually reported). There was initially confusion between two features: (1) Being able to tag images as "not important" so that the browser can give priority to other resources. (2) Being able to opt in to loading specific images only at the point where they are in the viewport or just about to enter it. This issue is specifically for (2). I will refer to this as "lazy loading".
The thread goes around in circles and doesn't really have a clear outcome, although the implementations discussed still seem valid and relevant today (Jake's summary in comment 49 is a good point to start at if you don't want to read the entire thread). I'm going to try not to repeat too much from that thread, but it has been 5 years now and as far as I can see lazy loading images is still a relatively common pattern. On top of that, the profile of the average internet-connected device has changed drastically (under-powered Android devices on very expensive cellular connections) and in my opinion the argument for lazy loading images is stronger now than it was 5 years ago.
I'm going to provide some insight into a use case that I'm very familiar with: the BBC News front page. I'll do this in the hopes that it provides some real life context around why I think lazy loading images is important, and why doing it in JS is not good for users.
Loading the page in Firefox in a 360 x 640 viewport from the UK (important because the UK does not get ads, which skews the results), the browser makes the following requests:
We use lazysizes to lazy load all but the very first article image. Lazysizes makes up about half of our JS bundle size. I know it's overkill for our use case but it's a popular and well-tested library. We load our JS with a <script async>
tag, so it can take some time before the JS is executed and the images are inserted into the document. The experience of seeing so many image placeholders for several seconds can be quite awkward. We actually used defer
for a while but the delay was deemed too long on slower devices.
From our point of view the benefits of the UA providing lazy loading are:
Despite Ilya's arguments against lazy loading in general, we've been doing it for 5 years and we're going to continue doing it until cellular data is much cheaper. If we got rid of our lazy loading, two thirds of our mobile users would download 170kB of data that they never use. Keeping the next billion in mind, that's about 3 minutes of minimum wage work. At our scale (up to 50M unique mobile visitors to the site each week) 170kB per page load starts to feel uncomfortably expensive for our users.
So what do the WHATWG folk think? Is it worth having this conversation again? Is there still vendor interest? Mozilla expressed interest 5 years ago but it seems like nothing really happened.
We literally halve the amount of JS in our bundle
Intersection observers means the JS for triggering loading on element visibility is tiny.
The UA can load images earlier, probably as early as DOMContentLoaded.
That's also possible with a small amount of JS.
The UA can decide whether to lazy load at all (e.g. only lazy load on cellular connections).
Yeah I think browser heuristics (along with no JS dependency) are the remaining selling points of something like lazyload
. But is it enough to justify it?
Intersection observers means the JS for triggering loading on element visibility is tiny.
Yeah, fair call. If we drop our big ol' lazy loading JS for a lazyload
attribute we may as well drop it for 10 lines of intersection observer wiring.
I guess the thing that appeals to me most about a lazyload
attribute is that it's pretty much the minimum amount of friction you could have for implementing lazy loading, and it leaves all of the nuance up to the UA. In my experience developers don't really know about or care about the nuance of whether their JS is blocking or deferred; runs at DOMCL or load. If there was a big slider that controlled who did the most work UA o-----------|--o Devs
I would shift it all the way to UA
because devs often don't have the time to do things in a way that provides the best experience for users. I realise this kind of thinking goes against the Extensible Web Manifesto, though. 🙊
I can see two more arguments in favour of an attribute. The first is that lazy loading mechanisms which depend on scripts have a significant impact for user agents where scripts don't execute. To prevent images from loading early, the images are only inserted into the DOM later on, leaving non-scripting environments without images at all. Few sites seem to think about the <noscript>
element these days.
The second is that providing it through an attribute means that the user can configure the behaviour as they prefer to experience the web. Someone on a slow connection might want to make images start loading earlier than when the image enters the viewport in order to finish loading in time, while someone else with a lot of bandwidth who dislikes lazy loading can disable it entirely.
(In general, I believe it is important that common website practices are standardised in order to give some control of the experience back to the user, or we may eventually find ourselves with a web that is more of a closed runtime than a document platform which is open to changes by extensions, user scripts and userstyles.)
@Zirro those arguments are the "browser heuristics" and "no JS dependency" benefits I already mentioned, no?
@jakearchibald I suppose I understood the "no JS dependency" benefit as referring only to having to load less JavaScript rather than the content being available to non-scripting agents, and missed the meaning of "browser heuristics" in your sentence. Still, I hope that detailing the arguments and why they are important can help convince those who are not yet sure about why this would be useful.
In general non-scripting agents are not a very compelling argument to get browsers to support a proposal, given that they all support scripting :). (And I believe these days you can't turn off scripting in any of them without extensions.)
@domenic I would hope that they see the value in having a Web that is accessible to all kinds of agents beyond their own implementations, much like a person without a disability can see the value of designing a website with accessibility in mind.
In general non-scripting agents are not a very compelling argument to get browsers to support a proposal, given that they all support scripting :).
@domenic The issue is more whether these scripts fail to download, which does lead to an odd experience. It's becoming harder and harder these days to progressively enhance websites as we seem to depend on scripting more and more for the core functionality of our websites.
Yeah I think browser heuristics (along with no JS dependency) are the remaining selling points of something like lazyload. But is it enough to justify it?
I think both of these are big selling points for the reasons above. As I say, this is not something that's possible to progressively enhance. There is not any way to provide a fallback for those for whom the JS fails for whatever reason.
A few years ago, GDS calculated how many visits do not receive 'JavaScript enhancements', which was a staggering 1.1%. Like GDS, at the BBC, we have to cater to a very wide audience and not all of them will have stable internet connections. I have a good connection at home and even for me the lazyloading script can fail to kick in sometimes.
Additionally, I feel as though we haven't covered one of the main issues with this that I mentioned in my original comment:
We don't know in advance the size of the visitor's viewport, so we have to arbitrarily determine which images to load in lazily.
Because we're using a script, we've had to use placeholder div
s for most images. While this is great for mobile devices whose viewports are too small to see many images at once, this is really unhelpful on large viewports. It creates an odd experience and means we can't benefit from having the browser start downloading the images as normal before DOMContentLoaded
is triggered. Only a browser solution can know in advance the viewport size and determine which images to download immediately and which ones to only download once scrolled into view.
@domenic The issue is more whether these scripts fail to download, which does lead to an odd experience. It's becoming harder and harder these days to progressively enhance websites as we seem to depend on scripting more and more for the core functionality of our websites.
I completely agree with this. At Wikipedia/Wikimedia, we have seen that interrupted JS downloads in low quality bandwidth situations are one of the most common causes of various problems. And that's also exactly the user situation where you'd want lazy loaded images. I'd guess with service workers you could do lazy loaded images as well, and then at least you're likely to have them on your second successful navigation, but yeah:
It's becoming harder and harder these days to progressively enhance websites as we seem to depend on scripting more and more for the core functionality of our websites.
Only a browser solution can know in advance the viewport size and determine which images to download immediately and which ones to only download once scrolled into view.
A topic I would like to tease apart is whether lazy-loading of images alone is the most compelling use-case to focus on vs. a solution that allows attribute-based priority specification for any type of resource (e.g <iframe lazyload>
or <video priority="low">
).
I know <img lazyload>
addresses a very specific need, but I can imagine developers wanting to similarly apply lazyload to other types of resources. I'm unsure how much granular control may be desirable however. Would there be value in focusing on the fetch prioritization use-case?
It would definitely be useful to have this for iframe
s, object
s and embed
s as well!
As for video
and audio
, correct me if I'm wrong, but unless the preload
or autoplay
attributes are used, the media resource won't be downloaded anyway until it's initiated by the user. However, if they are specified, it might be useful to be able to use lazyload
so they don't start buffering until they are scrolled into view.
When you mention a more general priority specification, do you mean something like the old Resource Priorities spec? What kind of behaviour are you thinking of?
I believe Edge already does lazy image loading. For out-of-viewport images, it loads enough of the image to get metadata for size (with byte-range requests?), but not the entire image. The entire image is then loaded when visible to the user.
Would lots of small byte-range requests for out-of-viewport images be acceptable?
@mcmanus I think the previous comment in this thread is of interest to you.
I have two questions:
Will the css background image use a similar attribute?
.box { background-image: url("backgorund.gif") lazyload; }
The image async attribute is discussed here https://github.com/whatwg/html/issues/1920. I think the 'lazyload' and the 'async' attributes are very related. The first one postpone loading the image till it is needed. The second attribute moves the decoding to a separate thread and this will skip drawing the image till the decoding finishes. They both have almost the same effect if the image source or the decoded image is not available: the image will not be drawn; only the background of the image element will be drawn. When the image source and the decoded image are available, the image will be drawn.
I can't think of any use of these cases:
async="on" and lazyload="off" async="off" and lazyload="on"
If any of them is "on", the browser will be lazy loading or decoding the image. In any case, the user won't see the image drawn immediately. So should not a single attribute be used to indicate the laziness for loading and the decoding the image?
<img src="image.png" lazy>
and
.box { background-image: url("backgorund.gif") lazy; }
Will the css background image use a similar attribute?
.box { background-image: url("backgorund.gif") lazyload; }
I guess that would be a separate discussion in the CSS WG, but at least in the case of BBC websites, the few background images that are used are visible at the top of the page, and therefore need to be loaded immediately anyway.
If any of them is "on", the browser will be lazy loading or decoding the image. In any case, the user won't see the image drawn immediately. So should not a single attribute be used to indicate the laziness for loading and the decoding the image?
It also depends on if these attributes would prevent the image from being downloaded entirely, or whether it would just affect the order in which the images are downloaded. (I think the latter would be much less useful.)
The content performance policy draft suggests <img>
lazy loading, although they only mention lazy loading of images and no other embeds it seems that their idea is to enable developers to opt-in for site-wide lazy loading.
For a complete proposal, we probably need not just a way to mark an image as lazy loading but also a way to provide a placeholder. Sometimes colors are used as placeholders but often it's a more data-compact form of the image itself (a blurry view of the main image colors seems popular). Placeholders are also sometimes used for non-lazy images, e.g. on Medium the immediately-visible splash images on articles briefly show a fuzzy placeholder.
Also: Apple is interested in a feature along these lines.
@othermaciej in early 2014 I proposed CSS placeholder
(similar to background
property only applied until loaded/failed) https://lists.w3.org/Archives/Public/www-style/2014Jan/0046.html and still there haven't been any progress related to it.
The Chrome team's proposal is a lazyload=”” attribute. It applies to images and iframes for now, although in the future we might expand it to other resources like videos.
“lazyload” has the following states:
In Chrome we plan to always respect on and off. (Perhaps we should make them always-respected in the spec too, instead of being strong hints? Thoughts welcome.)
Deferring images and iframes delays their respective load events until they are loaded. However, a deferred image or iframe will not delay the document/window's load event.
One possible strategy for lazyload="on", which allows lazily loading images without affecting layout, is to issue a range request for the beginning of an image file and to construct an appropriately sized placeholder using the dimensions found in the image header. This is the approach Chrome will take. Edge might already do something similar.
We’re also open to the idea of supporting developer-provided placeholder images, though ideally, lazyloaded images would always be fully loaded before the user scrolls them into the viewport. Note that such placeholders might already be accomplishable today with a CSS background-image that is a data URL, but we can investigate in parallel with lazyload="" a dedicated feature like lowsrc="" or integration into
Although we won’t go into the details here (unless you’re interested), we also would like to add a feature policy to flip the default for lazyload="" from auto to off. This would be used for example for a particularly important
@bengreenstein It is great to hear your proposal. I have a couple of questions:
on: a strong hint to defer downloading BTF content until the last minute
Does this imply images will not be downloaded until visible in the viewport (at least on metered network connections)?
One possible strategy for lazyload="on", which allows lazily loading images without affecting layout, is to issue a range request for the beginning of an image file and to construct an appropriately sized placeholder using the dimensions found in the image header. This is the approach Chrome will take.
If width and height attributes are already provided by the author, will that negate the need for this request?
Here's a number of thoughts on this proposal:
I wish there was a way to make this a boolean instead of a tristate, since boolean attributes have much sweeter syntax in HTML.
Bikeshed comment: If it has to be a tristate, maybe we can have more meaningful keywords than on
and off
. How about load=lazy
, load=eager
, load=auto
? This also makes it feasible to add other values if a fourth useful state should ever be discovered. And it's also a bit more consistent with the decoding
attribute (on which see more below).
Many developer-rolled versions of lazy loading use some form of placeholder so that seems like an essential feature. CSS background-image
with a data:
URL seems like a pretty inelegant (and potentially inefficient) way to do it.
It's always possible to see the placeholder state during an initial load or when scrolling fast soon after load on a slow network. So it can't be assumed that "lazyloaded images would always be fully loaded before the user scrolls them into the viewport". This is a good goal but not always achievable. Concretely, I frequently see the placeholder image on Medium posts when on LTE and can sometimes even see flashes on my pretty good home WiFi.
It would be good to figure out how this interacts with async decoding. Should lazy-loaded images be be asynchronously decoded as if decoding=async
was specified? I think probably yes, as the use cases for sync decoding don't seem to be consistent with lazy loading. At the extreme you could think of lazy
as an additional decoding state, though that might be stretching the attribute too far.
Some additional thoughts:
Good point. We need to consider <picture>
too, where different <source>
images may need different placeholders, since they may not all have the same size.
img { placeholder-transition: opacity 1s linear; }
When the actual image is available, the opacity of the placeholder will go from the current_img_opacity to zero and the opacity of the actual image will go from zero to the current_img_opacity in 1 second.
@JoshTumath: thanks!
Does this imply images will not be downloaded until visible in the viewport (at least on metered network connections)?
No. We plan to start loading when the element gets close to the viewport due to scrolling, allowing some padding so that the resource finishes loading before actually reaching the viewport.
If width and height attributes are already provided by the author, will that negate the need for this request?
For most images, yes. In Chrome we plan to make the request anyway to avoid deferring tracking pixels, but we may eventually experiment with heuristics to special case them.
@othermaciej: thanks as well!
I wish there was a way to make this a boolean instead of a tristate
That would be nice, but we are really trying to support three cases: 'on' is a hint to the browser that lazyloading of an element is desirable/safe where, perhaps, the browser would have presumed it to be unsafe or undesireable. 'off' is a hint that lazyload is undesirable/unsafe, where, perhaps, the browser would have presumed it to be safe and desireable.
If it has to be a tristate, maybe we can have more meaningful keywords than on and off. How about load=lazy, load=eager, load=auto
on
and off
seemed clearest to me, but I take your point that it limits extensibility. I'm not sure I like 'eager' because it might require some explaining. I'm open to discussing this more.
Many developer-rolled versions of lazy loading use some form of placeholder
Indeed. I think it's useful to separate our thinking about the specific case of lazyloading outside-the-viewport content from lazyloading in general. Our proposal mainly deals with the former, and there we hope never to have the opportunity to show a placeholder, because, hopefully, we will have loaded the resource by the time it is scrolled to. For images, Chrome creates its own placeholder just in case it hasn't. It makes sense to me to provide a way for developers to specify their own placeholders.
The latter case, iiuc, is usually about getting a low res image (e.g., a data url) onto the screen as quickly as possible, while or before a high res replacement loads. I agree that is a common use case and something that we might want to support. I'm not sure what lazyload means in this case, though: "load after everything else?" Maybe instead we want a placeholder and a priority API?
It would be good to figure out how this interacts with async decoding.
The lazyload attribute is different from (and orthogonal to) images with decoding=async. lazyload means defer loading until needed, whereas decoding=async means do not block presentation.
One reason that lazy loading needs to be opt-in: if the author is relying on the page load event to know when content images are loaded, and then painting a content image into a canvas, that image has to have been loaded. I'm not sure what should happen if such an image has load=lazy
; maybe the author needs to remove the attribute and wait for the image's load event to fire?
@bengreenstein
Maybe instead we want a placeholder and a priority API?
I suppose @addyosmani's priority hints (https://github.com/WICG/priority-hints) would apply here?
But IMO those blurry low-res placeholder images are terrible, I'd rather have a spinning loading indicator. I think most authors (and end-users) would agree:
... we hope never to have the opportunity to show a placeholder, because, hopefully, we will have loaded the resource by the time it is scrolled to.
But IMO those blurry low-res placeholder images are terrible, I'd rather have a spinning loading indicator.
At the site I'm working on, we're more inclined to use the dominant colour of images as a solid background until they've loaded. I recall seeing this technique on Google Images at one point, and it appears to me like it provides a better experience for the user than a blank box or loading indicator.
While we could potentially generate a small image with the solid colour, it would be more efficient if we also had the option to supply a placeholder colour through an attribute. Due to a strict CSP, inline CSS can not be used to supply colours in the markup.
We might want to avoid using "lazyload" as its name. IE11 shipped lazyload
attribute based on the old Resource Hints proposal.
But the IE implementation has different semantics (not a boolean attribute but takes 0
or 1
for its value). So at least we should avoid boolean attribute if we go with lazyload
.
Hi all,
@bengreenstein has put up a proposal for this in https://github.com/whatwg/html/pull/3752. Review and thoughts appreciated.
As I mentioned before in an earlier comment, it might be desirable having a way to set the default value of lazyLoad
for all embedded resources e.g:
<meta name="lazyLoad" content="on|off|auto">
.
I'd love to hear some thoughts on that :+1:
I think @clelland might be working on a way to do that using feature policy?
I think there still some unanswered questions here:
@Malvoz -- the proposal right now is to define a policy-controlled feature that sets the default behaviour, so <img lazyload="on">
and <img lazyload="off">
would be explicit, but <img>
by itself would follow the policy.
This would let you opt-out of lazyload for an iframe, for instance, and because the feature policy is inherited by subframes, any content embedded by that iframe would also be opted-out.
Similarly, you could set the default policy your own page with a Feature-Policy
header.
@bengreenstein how does your proposal relate to #3670?
@smfr Chrome's implementation will fetch enough of the image to get dimensions to aid in layout. I don't see a need to expose control over metadata fetching to the page. As for the canvas scenario you raise, the plan in Chrome is disable lazy loading of images by default in frames that have at least one
@annevk The lazyload attribute is different from (and orthogonal to) the importance attribute. Lazyload and importance can be used together. For example, lazyload=on importance=low means defer loading until everything else has loaded if in the viewport; if outside the initial viewport, defer until scroll. Likewise lazyload=on, importance=high means load with high priority if in the viewport, but defer loading otherwise and when loading resumes (on scroll), load with high priority.
Chrome's implementation will fetch enough of the image to get dimensions to aid in layout. I don't see a need to expose control over metadata fetching to the page.
Well, but will such fetches go through a service worker? If so, we may need to specify them. If not, we may need to specify that they shouldn't... This does seem to be an interesting case.
Concretely, how do vendors interoperably get the behavior you're planning to implement in Chrome here? Probably we should write a spec for that fetch.
As for the canvas scenario you raise, the plan in Chrome is disable lazy loading of images by default in frames that have at least one tag that's accessible via JavaScript from that frame.
(I'm assuming by "tag" you mean "canvas tag".) Again, is this something we'd want interopability on?
I wonder what Edge does, since they implement lazy loading already?
It seems pretty weird (for authors) that adding a <canvas>
would suddenly change the image loading behavior of your page, and I bet many common pages have a few canvas tags for non-important content. Can we make this more explicit somehow?
@smfr and @domenic I see your points.
lazyload="metadata": I don't see the fetching of metadata as being something we should require of UAs. The loading of metadata is the default lazyload behavior for images in Chrome and I'd recommend that loading behavior for other UAs. Do you think other UAs should be required to fetch metadata? If so, and we made it the default behavior, do you think it would also be useful to provide lazyload="on-but-please-no-metadata"? Related, do you think we should improve interopability w.r.t. tracking pixels? E.g., Chrome will use some heuristics to determine which images to not lazyload, e.g., display:none, etc.. We could also require other UAs to do the same. Wdyt?
canvas: By default, the UA should lazyload when it is reasonably safe to do so. The presence of a canvas tag makes image lazyloading unsafe, so by default Chrome won't lazyload images when there's a canvas tag. The developer can explicitly enable lazyload on any image to override the default behavior. I can see a few other options: The first is to not change the default lazyloading behavior and instead require developers to disable lazyloading on any images that will be used by the canvas. The second is to provide a canvas attribute to say that the canvas is not affected by lazyloading. Wdyt?
@bengreenstein even if the metadata fetch is optional, it should be written into a standard. This could be a "may" series of steps.
If other browsers want to fetch metadata, this should be interoperable.
This is especially important if range requests are used. Range requests have historically been a source of security issues due to their lack of standardisation, and we shouldn't make the same mistake here. From in-person discussions, I believe the plan is to issue a range request for the metadata, but then make a standard non-ranged request for the full resource. This avoids the kind of security issues related to joining partials, which is exactly why it should be written down.
I'm happy to review. Recent additions to the fetch spec such as https://fetch.spec.whatwg.org/#concept-request-add-range-header may help.
https://wicg.github.io/background-fetch/#validate-partial-response-algorithm may also be interesting, which is something I'm working on at the moment.
Thinking about the interaction between image lazyloading and canvas, we don't lazyload images that are outside of the document, so I believe this would only be an issue if the canvas uses an image that is in the document and outside the viewport. This seems to me to be an edge case. To address it the developer can explicitly disable lazyload for that image.
On the topic of fetching image metadata, I don't think we should expose this choice to the web platform, nor should we require a user agent to fetch it. However, I do think we should agree on a common way to fetch metadata if a user agent chooses to do so. I've updated the PR with an algorithm.
@jakearchibald I'd appreciate a review.
I added the algorithm for fetching image metadata. Please take a look. Are there any further concerns with this being interoperably implementable? We hope to ship this in Chrome soon.
Has the use case of lazy loading CSS background images been addressed yet? Might we need something like this?
<div style="background-image: url(img.jpg); background-lazyload: on;"></div>
Edit: Nevermind, I can just use an <img>
tag. Now that we have object-fit: cover;
I don't need to use a background image. Are there other cases where lazy loading background images makes sense? I can't think of any.
In general non-scripting agents are not a very compelling argument to get browsers to support a proposal, given that they all support scripting :).
@domenic The issue is more whether these scripts fail to download, which does lead to an odd experience. It's becoming harder and harder these days to progressively enhance websites as we seem to depend on scripting more and more for the core functionality of our websites.
It's also worth noting that things like Firefox Reader View are impacted by lazy-loading scripts: mozilla/readability#299
What if the person viewing the site doesn't want lazy-loading? A person should have the option to disable lazy-loading at the browser level to override what a website has declared.
@amazingrando, maybe what @bengreenstein said makes you feel more comfortable with lazyloading:
We plan to start loading when the element gets close to the viewport due to scrolling, allowing some padding so that the resource finishes loading before actually reaching the viewport.
Problem
Many websites are very image heavy, but not all of those images are going to be viewed by visitors. Especially on mobile devices where most visitors do not scroll down very much; it is mostly the content at the top of the page that is consumed. Most of the images further down the page will never be viewed, but they are downloaded anyway.
This is slowing down the overall page load time, unnecessarily increasing mobile data charges for some visitors and increasing the amount of data held in memory.
Example workaround
For years, the BBC News team have been using the following method to work around this problem. Primary images at the top of the page are included in the HTML document in the typical way using an
img
element. However, any other images are loaded in lazily with a script. For those images, they are inidially included in the HTL document as adiv
which acts as a placeholder. Thediv
is styled with CSS to have the same dimensions as the loaded image and has a grey background with a BBC logo on it.Eventually, a script will replace it with an
img
element when it is visible in the viewport.Doing this with a script is not ideal, because:
Solution
There needs to be a native method for authors to do this without using a script.
One solution to this is to have an attribute for declaring which images or objects should not be downloaded and decoded until they are visible in the viewport. For example,
<img lazyload>
.*Alternatively, a
meta
element could be placed in thehead
to globally set all images and objects to only download once they are visible in the viewport.* An attribute with that name was proposed in the Resource Priorities spec a few years ago, but it didn't prevent the image from downloading - it just gave a hint to the browser about the ordering, which is probably not as useful in a HTTP/2 world.