Open triblondon opened 8 years ago
I also thought about that and I also saw the polyfill already. My plan was to at least play with the API to have some know how and maybe give some input in the design process of this young API. But I have currently plenty of work and it seems that this won't change until end of August.
In general, I'm an extreme fan of new web platform features as also of performance features, but I have a simple rule: I only add a feature and relay on polyfills as soon as two different browser engines are supporting them. (It doesn't have to be a stable browser version though). (And yes: There are more, but also vague rules).
This is why I'm now preparing the switch over to requestIdleCallback
. Which is already supported in Chromium and soon will be supported by Firefox. My tests are very promising although I hoped rIC
would give me not only an idle time, but also a time, where the layout is guaranteed to not be dirty.
With other yes, but not now ;-).
Big question here: Is any other browser interested to implement this API soon?
But let's talk about the thing here:
I must say, that I have nearly no knowledge about the full capabilities of IntersectionObserver, but I noticed some parts.
Here are some parts about the polyfill:
getBoundingClientRect
can be an extreme fast method that can be called on hundreds of elements (if you have more you might want to use efficient script yielding) on a high frequency without causing jank. On the other hand it can be very slow and causes jank if it is called even only on one element, if you call it at the wrong time.After I refactored my throttle method to: Do a normal throttle using setTimeout, then do a rAF and then call a timeout with 0. I had dramatic performance improvements.
A cleaned up version could look like this:
var throttle = function(fn){
var running;
var lastTime = 0;
var run = function(){
running = false;
lastTime = Date.now();
fn();
};
var afterRAF = function(){
setTimeout(run);
};
var getRAF = function(){
rAF(afterRAF);
};
return function(){
if(running){
return;
}
var delay = 125 - (Date.now() - lastTime);
running = true;
if(delay < 9){
delay = 9;
}
setTimeout(getRAF, delay);
};
};
This has two interesting effects by scheduling 3 jobs instead of just one, the job takes longer if the browser is currently busy (mostly only in the loading phase), which results in slower frequency. The second thing, that seems to heavily improve things is the fact that I often get a position in the frame life cycle right after frame commit (i.e.: rAF + timeout of 0. Basically this is also the position, where rIC
is scheduling tasks.) (The chance, that layout is dirty is much lower.)
If you are interested the upcoming throttle function can be seen here.
About the IntersectionObserver spec and its use case:
lazysizes is different in the way, that I don't use one static expand (margin). It uses a dynamic one. If there are currently images in view loaded only other images in view are loaded. Images that are near the view are not downloaded. If network is idling the view is expanded to preload images, that are not in view.
This way you get two important things:
I have difficulties to think how I can implement this model with IntersectionObserver because of two things.
The only thing I can think of, is to use one observer with the large expand option and than if the callback is called to sort these records for in view and near view images and then decide how to do things.
Use case: I don't doubt, that this API improves developer convenience, battery lifetime and also performance for developers, which don't use the right tools to implement in view detection. (No or non efficient throttled event handlers for example.)
But I really doubt, that it would noticeable improve jank with good current implementations. Especially, I don't think, that it would dramatically improve performance compared to the upcoming version of lazysizes.
The way how scroll events do work now has dramatically changed in the last years. FF is just making the switch and now all browser have async scroll events. The actual visual scroll is now always decoupled from the scroll event itself.
This doesn't mean, that it is bad (I think it can be very good for battery lifetime), but I don't think it will produce wonders.
Wow, this is incredibly useful feedback for IO. I'll cross-post to the IO polyfill issue. I think to summarise the issues you raise which the polyfill should address:
requestIdleCallback
-like scheduling behaviourThe use case of prioritising image downloading based on distance from the visible viewport is interesting, and I believe it's not within scope of IO to help you with that, but seems not too hard to build on top. The data you would get from a callback would include the intersection ratio, and allow you to easily determine whether the object was in view or just within the margin, and prioritise accordingly.
One use case you may not have covered here is where lazySizes is included in an iframe (eg an ad) which is not scrollable, but which is scrolled into view as part of a parent document which does not use lazySizes. Do you currently have any way to determine whether the content is in the viewport? I'm aware that advertisers are currently using some pretty horrible hacks to instrument this kind of thing - mostly for analytics, but ideally we'd like them to use this kind of detection to improve performance and bandwidth usage.
The data you would get from a callback would include the intersection ratio, and allow you to easily determine whether the object was in view or just within the margin, and prioritise accordingly.
Yeah, just started to play with the API instead of looking into the docs and it will make fun to work on top of it. Can't wait for the second browser ;-).
.. Do you currently have any way to determine whether the content is in the viewport? ...
This is tough. No. But nearly any advertiser script looks like a collection of hacks written by a junior php guy, who was forced to write javascript.
I realise that the quality of ad code is notoriously poor, but surely that's a reason for us to try and help those developers improve it. Probably writing ad code is how many newly qualified web developers get started, and advertising is a primary use case for the web.
Anyway I thought it was worth bringing up that use case because it is the best example of how IO can add capabilities to lazySizes beyond perf improvements which obviously are always going to be arguable.
Can't wait for the second browser ;-).
Seems like Mozilla are showing some intent to implement once previous work this would rely on is completed.
@aFarkas your plan to have a larger margin and keep track of the images in that window seems reasonable to me. A thing you could do is to have a second IntersectionObserver with no margin and you only observe elements in that one that the first observer said were in the larger margin. That way you can efficiently tell if things are in the larger window and the smaller window.
Regarding code efficiency: The big performance win with using the native IntersectionObserver here would be avoiding both polling, MutationObservers and listening to all these events. That will come with a significant performance cost. IntersectionObserver performance will scale with the number of elements you're observing, but I expect that on the vast majority of pages it will be considerably more performant.
Whether you use the polyfill or not, I recommend that you use the native IntersectionObserver when available.
Regarding adoption: https://blog.mozilla.org/futurereleases/2016/07/20/reducing-adobe-flash-usage-in-firefox/ implies Firefox is aiming to ship this year. We've heard positive sentiment from other browser vendors, but not specific date targets.
1/2400 page loads in Chrome is using it now. I expect that number to grow quickly. https://www.chromestatus.com/metrics/feature/timeline/popularity/1368
I already have a began to write a version based on IntersectionObserver. In most cases it will make sense to use it, if you are already using a IntersectionObserver polyfill or if you target only browser, which do support IntersectionObserver.
Some notes:
The big performance win with using the native IntersectionObserver here would be avoiding both polling, MutationObservers and listening to all these events.
IntersectionObserver won't help to ditch a MutationObserver because you will need to add new elements to the IntersectionObserver.
I expect that on the vast majority of pages it will be considerably more performant.
Lazysizes without IntersectionObserver already works with 60fps on the vast majority of pages. This can't be beaten by an even more performant implementation.
I already have a began to write a version based on IntersectionObserver. In most cases it will make sense to use it, if you are already using a IntersectionObserver polyfill or if you target only browser, which do support IntersectionObserver.
\o/
IntersectionObserver won't help to ditch a MutationObserver because you will need to add new elements to the IntersectionObserver.
Oh, I didn't realize it was intended to automatically observe things. You might want to consider a future iteration of this library where folks could opt to use custom elements and avoid the need for MutationObservers by using custom element callbacks, e.g.
I have a (very rough) first draft of something similar where you make a whole subtree lazy here https://github.com/ojanvafai/lazyelements/blob/master/lazy-elements.js. I'm definitely glad to see more lazy loading libraries emerging. This probably isn't the right forum to discuss this, but let me know if there are things you think browsers could expose to make any of this better or lighter-weight.
Lazysizes without IntersectionObserver already works with 60fps on the vast majority of pages. This can't be beaten by an even more performant implementation.
I didn't mean this as a criticism of your library. 60fps is a good minimum bar for libraries, but it's not the best judgement of library performance, especially given that most pages include many libraries that all need stay withing the frame budget when combined together.
If it's 60fps, but using 90% of the frame budget, then that means the page author has no headroom to write their own code. I'm not making any claims that this code is particularly slow, just that it could be faster and that the less of the frame budget you use, the more likely it will be that the page will actually hit 60fps with all their own code and other libraries mixed in.
Firefox ships the support of intersection observer in the version 52. source : https://developer.mozilla.org/en-US/Firefox/Releases/52.
@abarre I'm aware of this. I just added an intersectionobserver version, that you can use to test you app with. Note you will need IntersectionObserver and MutationObserver polyfill for browsers, that don't support these.
Don't expect a performance improvement though. The assumption by @ojanvafai that lazysizes would use "90% of the frame budget" is simply not true.
As long as lazysizes doesn't find any images to transform. It takes about 0.5-2ms to search for those elements throtteled by a combination of requestIdleCallback
and setTimeout
. (Maximum frequency 1 "search" per 125ms, but only if rIC
is called.).
Which basically means, lazySizes sometimes uses 10% of the frame budget. Unfortunately work of the IntersectionObserver is hidden from the Chrome timeline, so it is hard to compare it. I assume, that battery consumption is improved.
Thank you for this excellent feedback. I will see if it can replace our current lazyloader.
@aFarkas I think you're misinterpreting my feedback as saying that your library is bad and that people shouldn't use it. I wasn't meaning to imply that at all. I'm really happy to see lazy loading libraries become more popular. I wish more of the web was built with lazy loading.
I wasn't saying that lazysizes would use 90% of the framebudget by itself. I was commenting that measuring performance using FPS is not the best method for generic libraries like this. Modern web pages regularly include multiple libraries like this that each use part of the budget and you quickly get to 90%. So, even if you're hitting 60fps, it's still worth improving performance. I see death by a dozen cuts on a lot of pages (mostly due to ad networks to be fair). :)
Your measurement of milliseconds is more meaningful. Were you testing on a mobile device? Unfortunately, the budget available to JS is somewhere closer to 6ms because the browser itself uses a big chunk of your 16ms per frame (this is highly dependent on the content though).
Regarding showing the work of IntersectionObserver, we should fix it to expose something in the timeline. Filed https://bugs.chromium.org/p/chromium/issues/detail?id=668332 to track that.
I'm seeing this issue still marked as open but notice the intersection observer js file in my lazysizes npm package. Is it being used in this plugin or not?
Edit- nevermind. I see what you're doing and just added this in my webpack.config for testing:
resolve.alias = {
'lazysizes': path.resolve( __dirname, 'node_modules/lazysizes/src/lazysizes-intersection.js')
}
@afarkas Just FYI. Blink browsers (Chrome, Android, Opera), Edge 15+ have support for intersection-observer and Firefox 55 (release date on 8th August) will enable it too.
Any updates on this? Looks like you have some competition now https://github.com/ApoorvSaxena/lozad.js.
@t-kelly I already have an IntersectionObserver version of lazysizes included in this repo. You can use it with the polyfill. But be aware that the polyfill produces memory leaks and is by fare slower than the normal lazysizes version in such unimportant browsers like Safari for iOS.
It is also easy to get me to create a totally new IntersectionObserver version of lazysizes and promote it as the main script, if you can simply produce a simple test case that demonstrates that lazysizes can produce jank under some circumstances, while IntersectionObserver based versions doesn't.
Otherwise I will simply wait, until all latest browsers support this technology.
@aFarkas What do you think about a progressively enhanced version which uses the current behavior by default but uses IntersectionObserver if available? Of course this would still need some kind of test-case to show potential advantages/benefits.
@aFarkas Given the following situation on a page:
intersection observer
to trigger animations and track visibility of DOM nodes on scroll.lazysizes
and some of its plugins (bgset
, progressive
, object-fit
, and respimg
).Which of the following scenarios would you go for:
lazysizes
alongside Intersection Observer Polyfill
.lazysizes-intersection
alongside Intersection Observer Polyfill
.@caillou
Tough questions. I would still take 1. because the plugins are currently mainly tested with the normal version of lazysizes and lazysizes-intersection
doesn't give you much more performance.
Hey @afarkas - hope you’re well!
Is the IntersectionObserver version still being developed or is this abandoned now?
I know IntersectionObserver is really picking up ground now, but I notice the file variant hasn’t been updated in a while. I assume it’s a lot of work to convert?
@willstocks-tech
Yes the IntersectionObserver version is still being developed. But I indeed concentrate more on the main file.
You have to understand that IntersectionObserver is really made for people who want to either write a simple and fast lazyloader with less than 50/100 lines of code or who simply do not understand how to get element layout information in a performant way. But as soon as you want to have more control and know what you are doing it is much better to use the low level APIs.
For example I added the feature to change the expand
options afterwards. In the IntersectionObserver version I would need to change the rootMargin
option unfortunately that's not possible rootMargin
becomes a readOnly
property. Similar there is no direct control included for visibility: hidden
or opacity: 0
elements.
At the end: I currently think about writing a non-backwards compatible version of lazySizes. Only reason is about writing more modern code (so for current lazySizes users it shouldn't be that interesting its more for me as a maintainer a thing). This new version might not have IntersectionObserver version at all. While I had some perf problems in some very hard circumstances at the start of this project I have never seen a perf problem that can be solved with IntersectionObserver. I also bet that if you would create such a website IntersectionObserver would have actually more problems than my technique.
Thanks @afarkas - that all makes complete sense! TL;DR = IntersectionObserver is good for “quick and simple use cases” but if you want fine-grain control, or complex configurability and low-level access, it’s not the right tool?
I’m still new to JS, so am still learning the ropes. I know IntersectionObserver is a “go-to” at the moment for a fair few things, especially with the new “v2 spec” - which includes support for transform, opacity, filter, etc. (Worth a read: https://developers.google.com/web/updates/2019/02/intersectionobserver-v2)
I wouldn’t know where to begin to even validate whether the performance trade off is worth the development effort (is there actually any performance benefit? I know IntersectionObserver is a bit of a buzzword at the moment, but I’m not sure why as it seems to replicate a lot of existing functionality - as you said, you’re already doing what it can do and more, potentially in a more performant way...), or whether it’s more supportable! It is all super interesting though!!!!
Side note: love lazysizes - you do an amazing job, thank you 😊
Hello.
Google's documentation says that this library uses IntersectionObserver API (which is the method recommended by them for lazy loading). But I see there is a separate version of the library for that. Is still correct that if I want to use IntersectionObserver API I need to use that separate version (lazysizes-intersection.js) instead of the main one? Would that separate version include the polyfills as well?
Thanks a lot.
Yeah you can use intersection observer version if you want. But you won't see any noticeable perf improvements and of course have to add a polyfill (the official one has some obvious memory leak issues). Please understand that some tech people like to exaggerate especially some perf things to push new technologies.
Additionally it drives me crazy that developers - people who should be able to understand some tec information - blindly follow stupid advices only because the source is google or what ever. I could point you to some really crazy stupid stuff on google's web fundamentals page written by so called google experts.
Hi,
Thanks for you answer. I've been testing the latest version of Lazysizes using Puppeteer without scroll events and it works well.
This all comes because quite a lot of Google's documentation stress the need of using the IO API and because they say in quite a few places that Lazysizes uses the API (without specifying that it's a different version). They're reviewing some documentation and they have already done some changes.
See more context here:
I hope you don't mind I have mentioned this thread.
Thanks for your work!
I've been testing the latest version of Lazysizes using Puppeteer without scroll events and it works well.
Yes this test is a good example. This artificial test is tailored in a way that lazy loaders with scroll events do not pass. Then they make the false claim that search bots work the same way. But this is not true. The fact that lazysizes (with and without IO) works with search bots has different reasons and in fact a normal IO based lazyloader would not work with search bots.
They're reviewing some documentation and they have already done some changes.
This is something that really hurts me. I tried to get some corrected information into google's web fundamentals some years ago. And the currently wrong information are pretty laughable if you think a little bit deeper about them. But as it turns out the ego of the author of these articles was too big to acknowledge any mistake. So downgrading the recommendation of a third party library is easy but correcting wrong information and acknowledging mistakes that a lot of developers are blindly following is quite hard. To be honest it's hard for me to overcome this double standard.
Why IO is not the holy grail for lazysizes
Let me please explain in a longer way why I currently don't consider to use IntersectionObserver (IO) for the normal lazysizes version.
Someone who is seeing this issue might think that I'm against modern technology or I'm not interested in performance. In fact people who know my other older projects know I'm an early adopter of modern standard compliant technologies and I'm always interested in performance.
And I must say that I also like IO. With IO it is possible to create a blazingly fast lazyloading script with just a few lines of JS. That is awesome.
But what I'm against is the narrow minded notion that a developer must use IO to create a performant lazyloader or otherwise thousands of puppies/unicorns will die. This is simply not true. And for me it is important that people do use their brain and never follow advices especially if they say, "always do this"/"never do this". Instead of the education of "what" educate more "how" and especially "why".
Status quo Before we compare IO with lazySizes implementation. Let's just talk about the absolute numbers. Lazysizes uses around up-to 3ms (on my device this number is 0.5ms) every 8-9 frames for the intersection detection task and our performance budget for each frame during scroll is above 10ms (and in most cases much much more). The user does not see any difference no matter wether you are using for example 6ms or just 1ms (although it is a stunning improvement by 6x) and again lazysizes already only uses a small fraction of the budget.
The performance advantages of IO compared to lazySizes: Now let's see what main reasons gives IO a performance boost:
tocuhmove
listeners work. But this problem was solved some years ago. There is no browser which still uses sync scroll events anymore.getBoundingClientRect()
directly or at least the same underlying methods to obtain the layout data. Which is equal to the lazysizes implementation. The "only" magic thing is that this method is executed at a specific point inside of a frame (right after the layout was flushed/calculated) where the browser can guarantee that it doesn't produce layout thrashing.
It is good to know for developers that the JS work inside of the IO callback can still produce layout thrashing.(And a lot of IO using lazyloaders do a pretty bad job here.) But here comes the big point: lazySizes uses a simple workaround that is close to guaranteed no layout thrashing. Of course lazysizes can always guarantee no layout thrashing with "itself". But there is always some conflict potential if other scripts are involved. But I can not stress this enough: the simple workaround works so reliable that I'm still amazed how well it works. Layzsizes does not only separate style/layout reads from DOM writes. Lazysizes schedules these two different tasks to different points in a frame where they have the least conflict potential with any other third party script. And it is amazing how consistent it works. (Side note: I wish we would have a simple "official" requestLayoutFlushed
method that allows us to use any of the many existing potential layout forcing methods in a performant way.)With other words the two main advantages brought forward for IO are either not existent anymore or solved by lazysizes.
The disadvantages of IO compared to lazySizes:
As I developed lazysizes I tried to add some features that are not normal for other lazyloaders. One of those features I call dynamic expand (Expand is basically called rootMargin
in IO). Normal lazyloaders use a static expand/rootMargin
. What does this dynamic expand solve? Consider the following situation: If you have an expand
value 0
only visible images are loaded. Which means as soon as the user starts to scroll the user sees not yet loaded images and has to wait until those images are loaded, quite bad UX. The other option is to expand the "search area". This results in the following problem. Especially during the initialization phase. You might have 1-2 images in view and 3-4 images that are not in view but are inside of the expanded search area and this means the users bandwidth is literally divided by the image downloads that are inside of this expanded search area.
This is solved by lazySizes dynamic expand option. LazySizes constantly switches between different expand modes to load in view images with a high priority and preload not in-view images with a low priority.
This technique is hard to implement using IO because IO's rootMargin
can only be set once for all images during initialization of the IO and is readonly afterwards.
So using IO with a static expand option would not only be premature performance optimization. It would actually degrade the network performance in a significant way. Or with other words it costs me flexibility without giving me a huge performance boost. But of course if you want to write a performant lazyloader with under 2kb of code you must use IO.
Misleading layout thrashing information by google The other thing that drives me crazy is the misinformation of possible solutions to the layout thrashing problem spread by google. The main reason why I should switch to IO!?! These misinformations are expressed in the two articles Avoid Large, Complex Layouts and Layout Thrashing (ALCLLT) and Debounce Your Input Handlers (DYIH).
From a lazyloading point of view you can boil these two articles down to the following code pattern advertised by google experts in the DYIH article:
let scheduledAnimationFrame;
const readAndUpdatePage = () => {
scheduledAnimationFrame = false;
// do something
};
window.addEventListener('scroll', () => {
// Prevent multiple rAF callbacks.
if (scheduledAnimationFrame) {
return;
}
scheduledAnimationFrame = true;
requestAnimationFrame(readAndUpdatePage);
});
And in fact many lazyloading projects that are now using IO have used exactly the code pattern above to "improve performance". Funny thing: No it does not improve performance.
Let's go through the two main problems:
It has no debounce/throttle effect: First of all let me state this: It is a good idea to throttle/debounce all render code using requestAnimationFrame
. The issue though is that there are not many events that have such a high frequency. This is especially true for the scroll
event because it is inherent to the scroll
event. The scroll
event is a UI/render specific event. This means the browser has to synchronize it with the V-Sync and the requestAnimationFrame
is also synchronized with the V-Sync. The max frequency of the scroll event equals the frequency of the rAF
! You can simply test this by adding a console.log
to the abort condition to see how often this code rescues you from to many executions:
// Prevent multiple rAF callbacks.
if (scheduledAnimationFrame) {
console.log('success: i was executed - not');
return;
}
Think about how f------ crazy this is. This code is out there for years and copied many times by developers and no one asks themself: why the special trick is to write dead code that is never executed?
The second problem of the script is even crazier.
requestAnimationFrame is the worst time inside a frame to read style/layout information:
Before I go into the details: Let's recapitulate what layout thrashing is. Reading style and layout informations are normally blazingly fast. But can be also extremely slow if the DOM or CSSOM was changed previously inside of the same frame. The browser has a normal cycle where it calculates the style/layout information - if needed - according to the refresh rate of your monitor. After that layout calculation is done the style/layout informations are cached as long as no change happens. The requestAnimationFrame
schedules a callback right in front of this "normal" style/layout calculation.
Let's try to understand why google performance experts think it is a good idea to read layout inside of a rAF
regarding to layout thrashing which is explained in the ALCLLT article:
First the JavaScript runs, then style calculations, then layout. It is, however, possible to force a browser to perform layout earlier with JavaScript. It is called a forced synchronous layout.
The first thing to keep in mind is that as the JavaScript runs all the old layout values from the previous frame are known and available for you to query. So if, for example, you want to write out the height of an element (let’s call it “box”) at the start of the frame you may write some code like this:
// Schedule our function to run at the start of the frame.
requestAnimationFrame(logBoxHeight);
function logBoxHeight() {
// Gets the height of the box in pixels and logs it out.
console.log(box.offsetHeight);
}
END quote
Did this explanation made sense to you? It sounds indeed plausible, but rethink what rAF
does. rAF
doesn't bring you to the start of the frame. It brings you to a later point inside of the same frame and the longer you wait the higher the potential that any other JS has invalidated your layout. The crazy thing here is that rAF
doesn't only bring you to a later point. It brings you right in front of the normal style and layout calculation. This means if you read style/layout inside of a rAF then you are basically at the end of the life time of the previously calculated style/layout values and you are maximizing the potential of layout thrashing to the greatest possible point. rAF
is the worst point to read style values!
This is a fantastic tool and really positive for the performance of the web. I'd love to see this using IntersectionObserver to compute the intersection of the viewport and the lazyload elements, which will avoid the need to use a scroll handler in browsers that offer support for IntersectionObserver.
I'm currently working on adding an IntersectionObserver polyfill to the polyfill service, which will provide IO support on all the browsers you support for lazySizes, so a nice-to-have would be a version of lazysizes that relies on IO with no onscroll fallback, on the basis that it can be polyfilled where needed.