whatwg / html

HTML Standard
https://html.spec.whatwg.org/multipage/
Other
8.01k stars 2.62k forks source link

A way to run code before the next frame #10113

Open jakearchibald opened 7 months ago

jakearchibald commented 7 months ago

What problem are you trying to solve?

Deferring code until the render steps, but before paint, is a good way to optimise code related to rendering. It allows you to avoid running code too frequently, and run it just in time.

What solutions exist today?

requestAnimationFrame() lets you schedule a callback for the render steps. This will be before next render unless the current point in the event loop is between requestAnimationFrame callbacks and paint, in which case the callback will run after the next paint.

A piece of code may not know which of these states the event loop is in, so requestAnimationFrame() cannot reliably be used to run code before the next paint.

How would you solve it?

Some way of knowing if requestAnimationFrame(callback) will run callback before the next paint, or after the next paint. This could be a boolean, or something which offers more insight into the current event loop position.

Anything else?

A current sort-of hack is:

let rafWillBeAfterPaint = false;

(async () => {
  while (true) {
    await new Promise(r => requestAnimationFrame(() => r()));
    rafWillBeAfterPaint = true;
    await new Promise(r => setTimeout(() => r(), 0));
    rafWillBeAfterPaint = false;
  }
})();

Although this isn't totally reliable, and doesn't feel great performance-wise.

Kaiido commented 7 months ago

Quite interesting, but I'm not sure to completely understand the premise of this proposal. Paraphrasing comment 0, I understand that this would be useful only to scripts that are called from another script and thus don't control when they've been called, but still want to hook to the update the rendering phase, and to be rendered no later than in the next paint, right?
Is this such a common problem? Could you provide some example?

Now, if I get it correctly, this builds on the assumption that moving all the "code related to rendering" at the end of the event-loop would allow engines to optimize the code. Is it true? I remember the requestPostAnimationFrame proposal was built on the idea that pushing too much pressure on the update the rendering steps could actually be detrimental since you'd have less time for the code to run before presentation and you may lose a frame.
But even assuming it's used properly with only pure rendering stuff that don't take long to compute and that there is an actual optimization of the code there, if your caller didn't call you in the update the rendering phase, how can you be sure that it didn't execute other rendering stuff in whatever phase they were? Would the optimization still work if only your code is moved there or would it be better if it was still "packed" with other rendering-related code, even if it's not in the update the rendering phase?

Then I must admit I don't quite understand the point made in "It allows you to avoid running code too frequently". If you're not in control of when your script is ran, how can you ensure it's not ran multiple times per frame by just looking at that property?

jakearchibald commented 7 months ago

I understand that this would be useful only to scripts that are called from another script and thus don't control when they've been called, but still want to hook to the update the rendering phase, and to be rendered no later than in the next paint, right?

Correct.

Is this such a common problem? Could you provide some example?

I believe so. Systems that control the creation and mounting of elements are common and popular (React, VueJS, Svelte to name a few). If your library / component is called from within one of those systems, you're in the middle of something else's rendering system.

Now, if I get it correctly, this builds on the assumption that moving all the "code related to rendering" at the end of the event-loop would allow engines to optimize the code. Is it true?

Yes. Particularly with code that reads styles and modifies content as a result. If you do this as soon as possible, you risk layout thrashing with other code doing the same thing. If you do it after the next paint, you get a flash of incorrect content/layout. The ideal time is rAF timing, but if we're between rAF and paint, immediately (or microtask) is the next best option.

If you're not in control of when your script is ran, how can you ensure it's not ran multiple times per frame by just looking at that property?

You can't, but that seems less likely in the systems I've worked with. You're more likely to be called immediately, or once within the render steps.

Kaiido commented 7 months ago

Systems that control the creation and mounting of elements are common and popular

I must admit I'm not very well versed in these "systems" myself, but aren't these popular exactly because they're supposed to do the right thing with their "virtual DOM" and to remove all that burden from their users? And if they don't do it right, wouldn't it be better to fix each of these frameworks/libraries instead so that their users keep not worrying about it?

Particularly with code that reads styles and modifies content as a result. If you do this as soon as possible, you risk layout thrashing with other code doing the same thing. If you do it after the next paint, you get a flash of incorrect content/layout. The ideal time is rAF timing

I have to disagree here. Layout thrashing is caused by the interleaving of code that dirties the layout, and code that reads it. Whether it's in a fetch task, in a scroll callback, or in rAF doesn't change much. The only browsers where this would have a slight incidence are Webkit based ones where IIRC they do update the layout when the event loop is free for a few rounds. In other browsers that changes absolutely nothing and is possibly even detrimental since it would take the time of other code that should be running in there.
If you want to prevent layout thrashing you need to sort your code execution in 3 steps:

I'm using something like that, and it works pretty well most of the time[^1], limiting to 2 relayouts per frame. But it also means that you must know perfectly what will dirty the layout and what will trigger a reflow, and that you have control over all the code that do dirty&read the layout. And this is probably asking a lot to the developers, moreover since some triggers are really sneaky. Though a good place where such optimizations can be made is probably in the systems you were talking about.

But I'm not sure to see how this proposal would really help with this.

[^1]: It doesn't work so well with the APIs that do both trigger a layout and dirty the layout at the same time like scrollTo().

jakearchibald commented 7 months ago

I guess I can't ask you to trust me that I'm aware of what layout thrashing is, and that I don't need it explained to me 😄

This might be easier if we take a step back and look at the wider picture. Please read The Extensible Web Manifesto if you haven't already - a better solution here is a low level solution.

requestAnimationFrame(callback) does one of two things:

Do you accept that these behaviours are substantially different? Do you accept that there could be situations where one of those behaviours is the intended behaviour of the caller, and that getting the other behaviour would be undesirable?

The behaviour you get depends on the point within the event loop that requestAnimationFrame is called.

Do you accept that a single piece of code, such as an attributeChangedCallback, may not know what stage in the event loop it's being called from, given that code from another owner may have performed an action to trigger such a callback?

Do you think it's reasonable for the caller of requestAnimationFrame to know which behaviour they're going to get ahead of calling requestAnimationFrame?

This is a piece of information the browser knows, but simply doesn't expose. It can already be observed, just not in advance. This could be as simple as:

interface mixin AnimationFrameProvider {
  // …
  readonly attribute RequestAnimationFrameTiming requestAnimationFrameTiming;
};

enum RequestAnimationFrameTiming { "before-next-paint", "after-next-paint" };
Kaiido commented 7 months ago

I do accept all of these yes, and I do agree this is something that's hard to achieve and easy to implement. That's why I engaged in the first place and why I said it's an interesting proposal.

I simply don't see how you are making it relate with your use case, and thus I can't judge the solution you already had in mind. I'm sure I don't have to remind you that new features proposals are supposed to be the presentation of a problem and then we discuss a solution.

And given that in your second comment you said the actual problem is layout thrashing, and that this problem would not be solved by this, I'm still confused.

The attributeChangedCallback case is quite compelling as for when a "piece of code" can run without knowing when, but if the goal is to prevent layout-thrashing then just moving in rAF, before the internal layout, possibly interleaved with a bunch of other code that will dirty the layout because, e.g. they need to update styles in an animation loop, doesn't help much. And I'm sure there is a better solution for this issue.

But if there is another reason to absolutely want to be in the rAF callbacks in this scenario then I'm all ears, and I'll be very supportive of that proposal.

jakearchibald commented 7 months ago

Let me try again, but a solution that's hyper focused on one use-case, without solving "will my rAF callback run before or after the next paint?", is not what I'm looking for, as being able to answer that question is what I'm looking for.


I am a framework author, but rendering within my framework may be triggered by the actions of another framework.

The list of other frameworks that may trigger these actions is not known, and may change over time, so requiring changes in these frameworks is not practical.

Sometimes, rendering within my framework needs to respond to the current page layout.

This leaves with me with two options:

Option A

Read the current style and update immediately.

Pros:

Cons:

Option B

Debounce via requestAnimationFrame.

Pros:

Cons:

Option B is closer to the ideal, but the potential flash of incorrect content/layout is unacceptable. If I knew which requestAnimationFrame behaviour I was going to get, I could avoid this case by running immediately (but still batching style reads and writes to reduce layout calculations).

Kaiido commented 7 months ago

Ah, an rAF debouncer that makes a valid case 😃. I'll still note that other pertinent changes could also happen in later rAF callbacks or in RO callbacks, but now I'm on board.

jakearchibald commented 7 months ago

I'll still note that other pertinent changes could also happen in later rAF callbacks or in RO callbacks

Of course. Any system that lets you run "after everything else" falls down once two things run there, since the first is no longer "after everything else". But later is better than sooner.