Why was requestAnimationFrame removed from HTML 5.1?

[siusin]

The requestAnimationFrame API has been widely implemented... perhaps we can keep it in the HTML spec?

[travisleithead]

Looking at: http://www.w3.org/TR/animation-timing/ it seems that a standalone spec for requestAnimationFrame has been put into a Note. So, this is something that should probably integrated into HTML5.1 (at least temporarily--I think the various timing and event loop pieces could probably be pulled into a module at some point--and this would likely go with it.)

[LJWatson]

@travisleithead If part of the work to create a module has been done as a consequence of putting it in a note, would it be worth going the rest of the way to create a separate module now? Conscious this is in the grey area between new feature/module and existing/partially integrated feature.

[travisleithead]

Reading over the note, the spec is roughly divided into thirds: explanatory and examples, API definition, and processing model. As far as preparing it for a module, I would think that the explanatory and example content could be expanded, the API definition could probably be reduced in size, but the processing model section would be the most troublesome to dissect. As-is the task queuing and management algorithms of HTML have evolved since this note was written, and effectively merge quite a few task-related features into one combined processing model very efficiently (Promises, mutation observers, timeouts, and probably something I'm not thinking of.) It would be most efficient to carry the whole processing model into the module, or to omit it from the module, leaving it in HTML. To reduce the cognitive load of HTML, my preference would be to carry the processing model out into a module as well, but I think that brings various other features along for the ride (those mentioned earlier).

So, I'd be willing to try to start a module for this--I just figured staging the work first by bringing RAF into HTML (so that we at least have it defined) and then starting the module extraction would be the best strategy given the module work could take some time.

[siusin]

@travisleithead ++

@takenspc - Thanks for raising the issue. Does this all make sense to you? Comments, edits, or any kind of contributions will be appreciated :)

[blurbusters]

requestAnimationFrame() runs synchronized to refresh rate

One important aspect of requestAnimationFrame() is that it runs at the same rate as the refresh rate (end of Section 5.1 of W3C Animation Timing Spec). The module needs access to VSYNC in order to do this -- is there a VSYNC API built into HTML 5.1? It's important for things like animations, games and VR.

Almost all web browsers do this, see http://www.testufo.com/browser.html

Non-60Hz displays becoming more common

Various motion tests that demonstrate this are at http://www.testufo.com -- run these tests on displays other than 60Hz. This can include gaming displays (e.g. www.blurbusters.com/faq/120hz-monitors ) as well as VR displays (Oculus 90 Hz, Vive 90 Hz), as well as displays capable of displaying HFR (120Hz).

Should strongly discourage locked framerates

These displays are getting more and more common as time passes, so this is an important consideration -- even simply trying to run 60fps at 50Hz or 75Hz results in jumpy motion -- so browsers need to be strongly discouraged away from locked framerates (Edge is worse than Internet Explorer in this respect -- see www.testufo.com/browser.html ).

[plehegar]

422 adds requestAnimationFrame.

Note that I took into account document.hidden. See https://www.w3.org/2016/05/raf.html for a test. Firefox seems to have a bug and will run RAF even if document.hidden is true, unlike Chrome/Safari/Edge.

I didn't manage to get bikeshed to link from the IDL to the method definition. Not sure why...

[plehegar]

Regarding VSYNC, the spec says "If there is a top-level browsing context B that the user agent believes would not benefit from having its rendering updated at this time". It then talks about "if the browser is attempting to achieve a 60 Hz refresh rate, then these steps are only necessary every 60th of a second (about 16.7ms)." Lots of hand-waving there. If we change that to link it to the VSYNC instead, it's not clear that all steps under "Update the rendering" ought to run on VSYNC. Eg do one need to update the resize and scroll at 120Hz? This might be worth a separate issue.

[blurbusters]

Yes, you need to do the resize & scroll at 120fps @ 120Hz.

IE11, FireFox, and Chrome already smooth-scrolls with keyboard up/down at a screen update rate matching refresh rate.. With a smooth scroller extension, Chrome also smooth-scrolls with the mouse wheel as well.

Note to those people "assuming 60fps is enough" (in a "640K ought to be enough fashion): There is a lot of incorrect assumption that the human eye can only see 60 images per second when it is not really that simple -- lots of /indirect/ artifacts can occur that are still visible at far over >200fps@200Hz -- (1) For example, motion side effects like eye-tracking-based motion blur caused by a non-infinite refresh rate (e.g. demo at www.testufo.com/eyetracking ...). (Note: I'm the creator of all the TestUFO.com motion tests, including this finite-refresh-rate-induced optical illusion) -- it is a great example of an indirect side effect caused by the finiteness of the refresh rate -- and behavior changes at 60Hz, 120Hz, 144Hz 165Hz... (2) Vision researchers has long found this out, and a great article is written by Oculus' Michael Abrash, "Down the VR Rabbit Hole: Fixing Judder" at http://blogs.valvesoftware.com/abrash/down-the-vr-rabbit-hole-fixing-judder/ -- demonstrating discrete-refreshrate-related artifact still shows up in some situations even at 1000fps@1000Hz

I frequently reported to FireFox and Chrome about refreshrate/framerate bugs: https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=(site:mozilla.org+OR+site:chromium.org)+(rejhon+OR+%22blurbusters%22)+60+120++

A popular animation test pattern used by browser programmers (at least the FireFox and Chrome teams) for requestAnimationFrame() debugging is this one http://www.testufo.com/animation-time-graph --

It uses Javascript High Resolution Time http://www.w3.org/TR/hr-time/ to benchmark the time between VSYNC intervals (by using requestAnimationFrame() as the signal-tick of VSYNC) ...

Two to three years ago, the Chrome programmers obtained a gaming monitor (capable of multiple refresh rates up to 144Hz), and fixed many refresh-rate and fluidity problems, and to this date, Chrome has been the favourite web browser for smooth motion.

Fluidity tests: Also, here are web browser fluidity comparison benchmarks (how long browsers stay smooth with increasing screen rendertimes): http://www.blurbusters.com/blur-busters-120hz-web-browser-tests/

These were done by using these tests to enable a hidden "Delay" setting. http://www.testufo.com/#test=animation-time-graph&easteregg=1 Increasing the delay causes the javascript to do a busywait-loop for the specified number of milliseconds before beginning to paint. It simulates longer-than-expected rendering (e.g. intensive 3D graphics). At the time, Chrome maintained fluidity longer than most of the others.

Hope this helps in motion fluidity testing!

P.S. OpenGL already provide options to synchronize to VSYNC (OpenGL "swap interval", which WebGL also seems to use) but not everyone wants to do animations using WebGL. So what do we do, when requestAnimationFrame() disappears?

---

Related bug report https://github.com/w3c/html/issues/375 for W3C standardization (but not exactly the same, since it may be a separate replacement API instead of requestAnimationFrame() ---). If requestAnimationFrame() is removed, we need an alternate industry-standard method of accessing the VSYNC signal to set the metronome for animations (including non-WebGL)...

To borrow a RFC2119 normative keyword, there "MUST" be an alternate method of synchronizing to VSYNC (e.g. https://github.com/w3c/html/issues/375) before the removal of requestAnimationFrame() from HTML standardization.

[travisleithead]

Can this issue be closed since we have #375?

[blurbusters]

That depends.on what W3C decides because requestAnimationFrame() is the defacto HTML VSYNC API currently -- unbeknownst to many within W3C. Many HTML5 videogames use this VSYNC API.

159 -- Reinstatement: requestAnimationFrame() instead. Don't remove it. It's there, implementations exist, and saves work for web browser programmers. Improve this existing VSYNC API to enhance its VSYNC features.

--or--

375 -- Alternative API: if requestAnimationFrame() is actually truly discontinued, we definitely need an alternate standardization of synchronizing to VSYNC (a VSYNC API per se).

Also, #375 mentions possible new side features such as suggesting/permitting requestFullScreen() to use a lower-latency full-screen mode that skips window compositing (e.g. gaming modes) -- it can be very important for VR. This may be ideally opened as a separate issue -- but is mentioned in #375 because because screen synchronization options are often tied to screen refresh latency options in some screen mode APIs on some platforms (e.g. a faster, lower-latency fullscreen mode via Windows DirectX APIs and its Mac/Linux equivalents, as an example). Such low-latency full screen modes includes compromises like not being able to handle overlapped/popup windows -- in exchange for having lower latency very important for gaming, VR, science apps, etc. This might ideally be a separate W3C issue to open, but often the same APIs also accepts synchronization (VSYNC-related) parameters.

Because of all the above, #159 and #375 are not true duplicates because of the above. It needs careful decision how this is to be handled -- how will W3C handle choosing #159 versus #375?

[travisleithead]

Does @plehegar's merged pull request #422 address #159 (this issue)? Seems like it's [generally] reinstated... Is there followup work regarding the VSYNC that is requested for interop?

For #375 that sounds like future work (e.g., not for our immediate 5.1 publication). Does that sound right?

[blurbusters]

Yes, #422 seems to supersede #159, but I would like to verify. I'll need to git-pull this weekend and see if it looks good.

If reinstatement of requestAnimationFrame() is what is going to happen, then yes, #375 ends up being future work. That said, it can be broken into separate opportunities:

(1) Lower-latency mode merged into an enhanced requestFullScreen() API. This can be a separate feature, as video games (which often use requestAnimationFrame()) would also benefit from this. This is a very simple change, like .requestFullScreen("gaming") and just leave it to browser developers to interpret the flag. Perhaps it needs to be incubated...

SOLUTION IDEA: Keep #375 or split it into multiple issues, for 5.1 or for future

(2) Although most platforms use .requestAnimationFrame() as the defacto "VSYNC API", ...Unfortunately not all browsers do, and not at all refresh rates -- www.testufo.com/browser.html -- and also see the browser animation fluidity benchmark tests at http://www.blurbusters.com/blur-busters-120hz-web-browser-tests/

SOLUTION IDEA: 2a -- Strengthen the end of section 5 of W3C Animation Timing Standard, to make it REQUIRED that requestAnimationFrame() synchronizes to the refresh rate 2b -- Alternatively add a new API called "doesRAFsyncToVSYNC()" == true|false ... or "doesRAFsyncToRefresh()" or "isVSYNCsupported()" .....so that HTML5 apps don't need OS/useragent detection (like www.testufo.com/browser.html unfortunately needs to do to figure out if your browser supports VSYNC / synchronization to refresh rate -- very important for precision motion such as www.testufo.com/eyetracking ). 2c -- Or programmable rate ([i] sync to VSYNC/refreshrate, [ii] callback at a custom rate, or [iii] run as fast as possible by requesting a new frame immediately after finishing the current repaint (like chrome://flags is able to let you do) ...

As it stands now, Microsoft is the biggest aberration here for requestAnimationFrame (worse with Edge than IE10/IE11) while others (FireFox/Opera/Chrome/Safari/iPhone/iPad/some Android) properly synchronizes to refresh rate. Android devices are inconsistent, some Linux systems, and IE/Edge.

I'd like to see requestAnimationFrame() specification strengthened very slightly so we know we're successfully syncing to VSYNC (or not), or even have a flag that turns on/off. There is already ~80%-90% compliance with synchronizing to refresh rate. Thus I really think W3C needs to either (A) make it mandatory to solve the remaining 10% or (B) allow detection of synchronization (in order to avoid needing to do useragent detection to figure out which browsers/platform don't support VSYNC -- www.testufo.com/browser.html ...).

[chaals]

We added requestAnimationFrame() in 5.1 with #422. Think we can close this - #375 is the remaining feature request and is open, although perhaps it should be moved to incubation for now.