Strengthen requestAnimationFrame() spec for VSYNC / refresh rate matching in HTML 5.2

[blurbusters]

UPDATE: Change committed on #785. Please see #375 instead (rewritten/newer) for further standardization efforts on this.

PROBLEM

• requestAnimationFrame() is not sufficiently rigidly enough defined.

The older W3C "Animation Timing" spec had a better definition (https://www.w3.org/TR/animation-timing/) Section 5 says "The expectation is that the user agent will run tasks from the animation task source at at a regular interval matching the display's refresh rate. Running tasks at a lower rate can result in animations not appearing smooth. Running tasks at a higher rate can cause extra computation to occur without a user-visible benefit." -- However the newer W3C HTML 5.1 no longer correctly describes the desire to match requestAnimationFrame() exactly to the refresh rate.

DETAIL

• Browser support of refresh rates are mostly consistent except for a few browsers: http://www.testufo.com/browser.html
• Some browsers use an arbitrary hardcoded framerate limits (e.g. 105fps), such as found in Microsoft Internet Explorer: https://connect.microsoft.com/IE/feedback/details/794072/internet-explorer-animations-fails-on-120hz-computer-monitors-works-at-60hz-75hz-100hz?siteID=rGMTN56tf_w-1PqoBW8wrx7DKpzuXQ.Wbg
• A listing of many problems with requestAnimationFrame() timing: http://www.vsynctester.com/howtocomputevsync.html and at http://www.testufo.com/browser.html
• Firefox issue where it intentionally computes incorrect VSYNC times: http://www.vsynctester.com/firefoxisbroken.html
• Differing amounts of time spent inside requestAnimationFrame() can create jerkiness/stutters or lower frame rates despite using only a fraction of CPU. There were some benchmarks done: http://www.blurbusters.com/blur-busters-120hz-web-browser-tests/ Stutter began to occur in some browsers even at 10% CPU, while Chrome remained stutter-resistant all the way to 75% CPU used up (e.g. 12ms inside requestAnimationFrame() during a 1/60sec period). This disparity is unreasonable.

WHY IT IS IMPORTANT

• Games
• High-refresh-rate monitors (120Hz+ monitors, often found in GSYNC / FreeSync gaming monitors)
• Animations, motion tests, etc.
• Virtual reality. Note that Oculus/Rift is 90Hz

EASY SOLUTIONS

• Discourage hardcoding of numbers such as "60Hz" or arbitrary maximum framerates such as "105". This does a disservice to high-refresh-rate monitors and future HFR displays (including future 4K120Hz and 8K120Hz standards)
• Strengthen specification to make synchronization to display refresh rate a "MUST" instead of a "RECOMMENDED"
• Strengthen specification to make it return exact vsync time rather than now(). Chrome is the only one that does it correctly.
• [This lineitem has been moved to #375] Strengthen specification to reduce allowable jitter, to permit smoother animations, given the increasingly real-time nature of websites, games, VR, etc. Things should be able to be stutter/jank-free all the way to ~50%-75% CPU used up inside requestAnimationFrame()
• [This lineitem has been moved to #375] Add options of flexibility such as to maximize smoothness (when fluidity is priority) versus minimizing lag (when minimum latency is priority).

COMPLIANCE TESTING WEBSITES FOR requestAnimationFrame()

• http://www.vsynctester.com (click the gear icon for lots of tweakable settings)
• http://www.testufo.com/#test=animation-time-graph&easteregg=1

SUPERSEDES (IN PART): https://github.com/w3c/html/pull/422, https://github.com/w3c/html/issues/375, https://github.com/w3c/html/issues/159

[chaals]

@plehegar assigning you to take a first look as the person who might know best. Feel free to pass on, or others please fell free to take it if PLH is busy.

[duckware]

Work in this area is needed. Please do address the issues raised.

• rAF callback time arg: Setting the time argument to 'now' adds no value. Because of this, web browser vendors have already started to switch over to passing in the actual vsync time (Chrome / Firefox) as the argument to the rAF callback (which actually does add a lot of value). However, Chrome is currently the only browser that does this correctly. Firefox actually passes in a faked time (http://www.vsynctester.com/firefoxisbroken.html). Please strengthen the specification to make the rAF time argument be the exact vsync time.

As just as a very interesting FYI. On Windows, all rAF animations in all web browsers have a two frame input lag caused by how all browsers interact with the GPU. Apparently what is happening is that vsync triggers the rAF callback, but 'what was drawn' is not presented until the next vsync, but at that point, it is too late to make it to the screen (because the Windows OS composites). What was drawn in the rAF callback actually makes it to the screen one entire frame later. It sure would be nice if web browser vendors took notice, and (under Windows), presented what was drawn in a rAF callback immediately, instead of waiting for the next vsync (which then delays an entire frame).

[blurbusters]

The two biggies are likely:

1. [Easy spec change] Strengthen specification to make synchronization to display refresh rate a "MUST" instead of a "RECOMMENDED"
2. [Easy spec change] Discourage hardcoding of numbers such as "60Hz" or arbitrary maximum framerates such as "105"

Currently, Microsoft's EDGE browser doesn't run at 100Hz nor 120Hz -- in fact, it supports fewer refresh rates than Microsoft's Internet Explorer. The developers at Microsoft should get a BENQ XL2720Z which can support custom refresh rates in 1Hz increments from 50Hz through 144Hz, to do validation testing.

Chrome does it almost perfectly (even 240fps @ 240Hz! Video https://www.youtube.com/watch?v=oJakpZbvm3I#t=4m0s -- this is a YouTube video of www.testufo.com/frameskipping in Chrome browser on a 240Hz monitor!

Opera / FireFox / Safari can do any refresh rate too (Albiet Chrome is the clear winner at the moment), as they ran successfully at 120fps@120Hz on a 120Hz monitor.

Microsoft IE 10/11 goes up to 105fps (my tests apppear to show a hardlimit of 105, no matter how slow or fast CPU or GPU is, even on a GEFORCE GTX TITAN). Microsoft EDGE doesn't even do 100fps@100Hz nor 120fps@120Hz.

[blurbusters]

FireFox Update: I am pleased that since the earlier 120Hz Browser Tests done -- FireFox has dramatically improved fluidity (elimination of jank/judder) and it now stays smooth even up to around ~70% of a refresh cycle spent inside requestAnimationFrame() at least on Windows systems.

EDGE Update: Alas, Microsoft Edge 15 in the Windows Creator's Update, still does not work correctly. It does not remove its own internal artifical cap on the rate of requestAnimationFrame(), still running at 60fps at 120Hz -- and 72fps at 144Hz.

There is a huge growth in gaming monitors, and there are now over 100 models of gaming monitors on the market (GSYNC and FreeSync). They are becoming more common, with several models now available at ordinary places like the Staples office-supplies store, Best Buy, etc.

[blurbusters]

I see that W3C Editor's Draft (18 April 2017), section 7.1.4.2 Processing Model, uses this text:

NOTE: Whether a top-level browsing context would benefit from having its rendering updated depends on various factors, such as the update frequency. For example, if the browser is attempting to achieve a 60 Hz refresh rate, then these steps are only necessary every 60th of a second (about 16.7ms). If the browser finds that a top-level browsing context is not able to sustain this rate, it might drop to a more sustainable 30Hz for that set of Documents, rather than occasionally dropping frames. (This specification does not mandate any particular model for when to update the rendering.) Similarly, if a top-level browsing context is in the background, the user agent might decide to drop that page to a much slower 4Hz, or even less.

There appears to be fluff words ("....would benefit from having its rendering updated depends on various factors...") that could easily be shortened with some good English rewriting.

Implied assumptions Being Made We're running on battery power Also, I observe an excessive focus on concerns of saving battery power, i.e. mobile devices -- when it should be also factored in that modern browsers on desktop GPUs (NVIDIA/AMD) can run in excess of >250fps and even 2000fps (screenshot!) while using less than 20% CPU power. There is a false preconception that browsers are unable to run at high framerates. Also desktop computers are not battery-powered. There are now over 100+ high-Hz gaming monitors on the market.

120Hz mobile phones While most browsing worldwide now occurs on mobile devices, there are now certain mobile phones starting to experiment with 120Hz (e.g. Sony Xperia X Premium) -- so this needs to be considered.

Wording doesn't mention matching framerate to refresh rate Also, the browser often runs its update (e.g. scrollbar, 120fps video playback) frequency differently from the frequency of requestAnimationFrame -- even when less than 5% of CPU is used. Microsoft EDGE manages to run testufo.com at 72fps using only 0.8% of CPU, and doesn't bother using the extra 0.8% of CPU needed (total 1.6% CPU) to run at 144fps (at 144Hz) on my AC-powered non-battery-limited non-performance-limited 5-year-old mid-range desktop computer. Why does EDGE rudely limit its requestAnimationFrame() rate to 72 callbacks per second, even though it can use its scrollbar scrolling at 144 frames per second? IE/EDGE plays back 120fps videos in

Wording is Problematic for Input Lag Reducing Edge Cases Also, sometimes, lag-reducing advocates recommend running at an uncapped rate on high-performance computers to minimize input lag. For example, rendering at 1000fps can reduce the input-latency to 1ms per frame, e.g. between the mouse click (of a 1000Hz mouse) and the actual display of the result. If updating only 60 times a second at 60Hz, this can still be a full 16.7ms delay between the timing of a mouse click (if it occurs right after the input read, keyboard or mouse) and the actual action. This is why many gamers (especially in eSports gaming). .... There is even a setting in chrome://flags (in Chrome) to uncap the update rate, and enable the browser to update in excess of 500 frames per second, for special experimential cases. Ideally, in a perfect world, there should be an W3C-compliant browser API to control this (e.g. turn VSYNC ON versus VSYNC OFF). Obviously, this is an edge case, but this illustrates the problematic wording of saying "....these steps are only necessary every 60th of a second..."

Wording should not dictate a preference of framerate-slowdown algorithm. This should be developer choice Yes, sometimes this is best. But even for mainstream situations, blanket-recommending 60fps suddenly go to 30fps is very problematic. Sometimes this is actually better, but in many cases, this suddenly adds input lag that throws off aiming in certain kinds of gaming. In many use cases, like brief framerate slowdowns in video games, it's preferable to go in a sequence 60-60-60-60-58-55-50-54-57-59-60-60-60fps -- during brief performance issues, like a momentary complex moment in a video game. A gradual slowdown to 55fps has less input-lag than suddenly going to 30fps, for HTML gaming. However, at other times, it's definitely preferable to go to 30fps than 60fps. In a perfect ideal world, this is a developer-specified behavior (e.g. HTML5 API to specify preferred update-rate degradation behavior). But in the abscence of this, it's not W3C's place to suggest one-size-fits-all specific update-rate degradation behavior. Also, I've developed simple HTML games before -- Here's a simplified mathematical example: Trying archery at a moving-target of 960 pixels/second, 60fps is 16 pixel steps per frame (combined with 1/60sec lag) and 30fps is 32 pixel steps per frame (combined with 2/60sec lag). If you aim your arrow at the moment after an unexpected 30fps slowdown due to performance -- your archery-shoot (aimed during 60fps) may have an unintentional browser-enforced "+2/60sec" lag modifier instead of a "+1/60sec" lag modifier. Your arrow misses by 16 pixels because of a 1/60sec time delta of aimed button press versus aimed moving target. In this situation, sudden 60-to-30fps slowdown created an unexpected input lag that threw off your aiming. Remember, human reaction times are typically 100-200ms, and if a browser slowdown occurs during this time window, your aiming is off because of an unwanted mandated browser half-framerate slowdown (i.e. a mere millisecond missing a 60fps deadline due to performance, enforces another forced 1/60sec wait before displaying the frame (if browser doing a mandated 60-to-30 slowdown) -- unwanted sudden lag change). As a result, for this specific HTML5 game, you want to override that, and enable the option of "play at maximum framerate as browser performance allows, without half-framerate slowdowns". In this case, user experience improves during this specific particular use-case of "usually 60fps with a few slowdowns". While many use cases are more complex than this, hopefully this simplified example helps developers understand. One has to realize the "60-to-30-slowdown" technique can be great but it is not a one-size-fits-all-apps solution.

As a long-term future plan, consider the eSports use case for a future VSYNC OFF mode Chrome can do it via a flag, "--disable-gpu-vsync" but this should be accessible in JavaScript full-screen gaming applications. Many people falsely claim that running framerates above refreshrate has no benefit. There is a side consideration of input lag. This is why VSYNC OFF is favoured by many competition gamers -- the latency between a button press or key press, and the screen update. Future standards ideally should make framerate-degrade behaviour a developer-specifiable API behavior instead (along with things like future "VSYNC OFF" API support for future uncapped-1000fps "ultralow-lag" games in web browsers -- scientifically benchmarked, 1000fps@60Hz has approximately 1/2 of a 60Hz refresh cycle less lag than 60fps@60Hz -- due to fresher delivery of fresher frame right before a fixed refresh cycle interval (closer to last-minute WDM compositing) or even splicing into the current refresh cycle (tearline effect in exchange for lower latency) that some games are able to do -- competitive eSports game players (on average) tend to prefer this (and may someday be important for any future browser-spaced eSports-league type games where latency is numero uno, with Olympics-style "millisecond-close" finish line reaction times -- eSports video gaming competitions has reached almost $1 billion dollars in 2016, and slated to cross that threshold in 2017. By making browsers more open and friendly to future developer API control of preferred framerate-limiting behaviour ("VSYNC ON drop to half framerate when performance constrained", "VSYNC ON drop framerate incrementally when constrained", "VSYNC OFF uncapped ultralow-lag mode running screen updates & callbacks at maximum performance rate", etc) -- this makes web browsers potentially more compatible with future eSports programming and special niches. Chrome has a hidden VSYNC OFF mode in chrome://flags that has to be manually accessed -- www.testufo.com has successfully run in Chrome browser at >500fps-1000fps on new NVIDIA GPUs with this mode is toggled. In high speed video input-latency tests, VSYNC OFF (framerate uncapped to ~1000fps / 1ms rendertimes) can have ~8ms less input lag than VSYNC ON at 60Hz (roughly half of a 60Hz refresh cycle less). Although visual updates aren't smoother with framerates running higher than refresh rate -- due to less time delta between final frame & the timing of monitor refresh -- there are input latency reductions in button-to-pixels response. So, this is not 100% advantage-less, and is useful in very niche situations. This is not of importance to most people, but it illustrates another reason not to arbitrarily specify a framerate behavior (e.g. recommending against framerates higher than refreshrate, e.g. recommending a 60-to-30 sudden slowdown) and leave that to developer choice. EDIT: A few years ago, before EDGE came out, Microsoft once responded to an email of mine saying browsers cannot do 120fps. Hogwash! Here's Google Chrome running at 2000fps on a 5-year-old c omputer (two thousand frames per second) running "chrome.exe --disable-gpu-vsync" mode, using less than 20% of CPU. Modern GPUs are very fast.

The mention of milliseconds (about 16.7ms) may be counterproductive The technique that many browsers use to synchronize to the display frequency is via a VSYNC trigger -- so browser engines don't measure time between refresh cycles (e.g. 16.7ms timer is STRONGLY discouraged -- as it's not synchronized, to refresh -- some systems are running at 59.94Hz, others 60Hz, and others another fraction thereof, such as 60.031Hz)... If you load a Custom Resolution Utility on a PC, you'll often notice that the computer's refresh rate is not always an exact integer. Avoiding mention of milliseconds is recommended, to repeat the mistake that some old web browsers did to use a timer rather than use VSYNC. The operating system's window compositing manager always synchronizes to the VSYNC too...

Consider future variable refresh rate standards There are now variable-refresh-rate monitors on the market (GSYNC, FreeSync) that dynamically changes their refresh rate every single refresh cycle. There is now an upcoming HDTV standard for variable refresh rates coming to big-screen TVs too, not just computer monitors. Microsoft is adding variable refresh rate support to the next XBox video game consule. For example, if motion runs at 58 or 57 frames per second, the display is running at 58Hz or 57Hz, and motion remains stutter-free (example simulated animation of stutter-free changes in framerate -- 100% stutterfree changes in framerates! -- this animation of stutterfree framerate changes is accomplished via frame interpolation but VRR displays do this natively). It is possible within 5 or 10 years, W3C may need to have a migration path to variable refresh rates permitting "Variable refresh rate displays may mean that browser vendors may decide to support them, updating the display at the same rate as the browser update rate of the top-level browsing context". Often on such VRR monitors, an OpenGL glFlush() command delivers the frame directly to the monitor (immediately refreshing the monitor, rather than waiting for refresh cycle). It would go to surmise, that this could happen with WebGL, and could happen with browser update contexts -- that a commit-to-display instantly activates a display refresh cycle, rather than buffers for next fixed refresh cycle. This is done at the API level, window compositing manager, or DirectX / OpenGL, etc.

However, matching update frequency to refresh rate (Or vice versa, in the case of VRR displays), is definitely indeed the recommended method for the smoothest, jank-free judder-free operation.

The existing W3C HTML 5.2 DRAFT 7 wording is very clearly written flawed with more than half-a-dozen ugly implied (accidental) assumptions (see below newer post for bullet list), so I propose a rewording of that particular paragraph.

[EDIT: Proposed update text has been moved to new post below]

[blurbusters]

CHANGE 1 of 2

Another consideration: The wording relating to timing requestAnimationFrame HTML 5.2 Draft 8 feels worse and less consistent/lower-quality (visual animation-wise recommendation of match-to-refreshrate) than current higher-quality browser practice (e.g. current versions of Chrome & FireFox). Even IE/Edge manages to surpass current HTML 5.2 Draft 8 wording, so let's not set the bar that low.

Thus, since this is github, and I apparently am allowed to submit a pull request (that should compel discussion on this :) ...) I'm going to attempt to submit a pull request that changes:

Proposal to CHANGE this (7.1.4.2):

NOTE: Whether a top-level browsing context would benefit from having its rendering updated depends on various factors, such as the update frequency. For example, if the browser is attempting to achieve a 60 Hz refresh rate, then these steps are only necessary every 60th of a second (about 16.7ms). If the browser finds that a top-level browsing context is not able to sustain this rate, it might drop to a more sustainable 30Hz for that set of Documents, rather than occasionally dropping frames. (This specification does not mandate any particular model for when to update the rendering.) Similarly, if a top-level browsing context is in the background, the user agent might decide to drop that page to a much slower 4Hz, or even less.

Into this:

NOTE: There are many factors that affect the ideal update frequency for the top-level browsing context including performance, power, background operation, quality of user experience, refresh rate of display(s), etc. When in foreground and not constrained by resources (i.e. performance, battery versus mains power, other resource limits), the user agent normally prioritizes for maximum quality of user experience for that set of {{Document}}s by matching update frequency and animation frame callback rate to the current refresh rate of the current display (usually 60Hz, but refresh rate may be higher or lower). When accommodating constraints on resources, the update frequency might automatically run at a lower rate. Also, if a top-level browsing context is in the background, the user agent might decide to drop that page to a much slower 4Hz, or even less.

Improvements

• Explains user experience factor sometimes prioritizes over reasons to slow update rate (e.g. no battery limit, no performance limit)
• Removes "16.7ms" (good) -- to eliminate temptation of wrongly using a timer instead of current mostly-prevailing practice to synchronize to VSYNC or window manager. See earlier post above.
• "Constraints on resources" is neutral. It can be natural (e.g. insufficient performance) or artificial (e.g. a predefined target on power consumption) or somewhere in between (e.g. a virtual machine with preconfigured parameters). The word "resources" can apply to any unforseen resource limitation -- like lack of appropriate OS APIs, lack of timer precision, etc. -- as a catchall of any limit a developer may run into. I was very careful to be 100% neutral what "constraints on resources" means, this is left up to the developer.
• Removes well-intentioned but flawed "60-goes-to-30" -- because this isn't one-size-fits-all as sometimes specific apps require gradual degradation in framerate (e.g. 60-59-58-57-etc) -- and also is simultaneously incompatible with upcoming variable refresh rate (GSYNC/FreeSync/VESA Adaptive-Sync/HDMI 2.1 VRR/etc) technologies. See earlier post above.
• Clearly points out higher-than-60Hz exists. Necessary mention that other refresh rates than 60hz exists. Unfortunately, I've met very smart people who didn't know that other refresh rates (than 60Hz) exists, so this actually needs to be mentioned, given the issue of several browsers (including pre-Chromium Opera) historically being locked to 60Hz regardless of current display refresh rate. And of course, Microsoft IE/Edge -- possible flawed assumption that Microsoft unfortunately made (IE/Edge issue)
• Brings back neglected mention about motion quality may be more important than battery in certian circumstances (a mention of motion quality exists in old W3C Animation Timing Standard, but is missing in this document)
• Discourages cap/limit when there's no constraints necessary. Remember: 120 frames per second at www.testufo.com in Chrome on my 5-year-old desktop gaming computer only uses 1% CPU, and less than 10% GPU (in many cases, now less than 2% on newer NVIDIA gaming GPUs)
• Rewords it in a way that preserves a long-term compatibility path towards future emerging variable refresh rate support such as GSYNC/FreeSync/VESA Adaptive-Sync/HDMI 2.1 VRR/etc -- where display refresh rate dynamically changes. Dozens of times a second -- aka every refresh cycle! Display refresh-rate changes in real-time to match current frame-rate, rather than the traditional vice-versa (traditionally trying to match frame rate to display refresh rate). For over a decade, laptops often temporarily lowers refresh rate on the fly as a power-saving measure, but the new variable refresh rate technologies eliminate visible stutter of framerate changes.
• Illustrates quality-versus-performance/battery tradeoff choice for useragents. Inclusion of motion quality (that was mentioned in old W3C Animation Timing standard)
• Removes flawed assumption of a single refresh rate. Example is multiple-monitor systems. While this behaviour is not strictly defined, Chrome uses the refresh rate of the monitor that the largest surface area of window is within. (e.g. if 80% of the window is on the 120Hz monitor instead of a second 60Hz monitor, then Chrome automatically uses a 120Hz update frequency on the top-level browser context -- this might be an automatic WDM behavior that Chrome is synchronizing to).
• Easier to read language (might need a bit of adjustment).

While not perfect wording, and may not completely solve #785, I think this wording is a big improvement for many reasons.

@chaals / @siusin -- Can you assign this github issue to me?

[blurbusters]

CHANGE 2 of 2

Right now, Chrome/FireFox/Opera is compliant already with my proposed changes, with only a minor change needed by Microsoft IE/Edge needed (remove the arbitrary hard-coded internal cap).

That said, browser vendors have frequently errored into this territory in the last five years (And Microsoft EDGE still does, as of Windows 10 Creator's Update), so I've made an additional wording update to make this more airtight, as there are some developers in this world surprised "There are displays above 60Hz?" -- so unfortunately it has to be mentioned.

2nd Change (in addition to earlier change)

CHANGED:

Another example of why a browser might skip updating the rendering is to ensure certain tasks are executed immediately after each other, with only microtask checkpoints interleaved (and without, e.g., animation frame callbacks interleaved). For example, a user agent might wish to coalesce timer callbacks together, with no intermediate rendering updates.

INTO:

Another example of why a browser might skip updating the rendering is to ensure certain tasks are executed immediately after each other, with only microtask checkpoints interleaved (and without, e.g., animation frame callbacks interleaved). For example, a user agent might wish to coalesce callbacks together, with no intermediate rendering updates. However, when are no constraints on resources, there must not be an arbitrary permanent user agent limit on the update rate and animation frame callback rate (i.e., high refresh rate displays and/or low latency applications).

You'll also notice I changed "coalesce timer callbacks together" into "coalesce callbacks together". The requestAnimationFrame() is not a timer per se in the specific case of Chrome and FireFox, they synchronize to an external trigger -- the VSYNC signal has often multiple non-timer methods of detection on multiple different platforms -- and can be a different technique in different rendering pipelines in the same browser (e.g. hardware versus software)

• Callback signal from the OS WDM (window desktop manager) if there's a callback for VSYNC (not available on all OSes)
• Polling a GPU register (VSYNC true|false) or an API (e.g. RasterStatus.InVBlank).
• Or timing of a return from a graphics-commit call such as glFlush(). This often pauses until VSYNC, flips a buffer then immediately returns. The timing of the return from call closely corresponds to the timing of the VSYNC.

While a monitor refresh rate is often sort of like a timer signal -- it's not always a program timer. So I removed the word "timer" (as that word only sometimes true, not always true) while also adding 1 sentence about arbitrary limits.

[blurbusters]

I did talk to a few industry representatives (gaming industry, display related, display engineers). So far they've agreed the changes "keeps doors open" far better, as my previous long description states. Feel free to trial-balloon my suggestion around to other people who have an intimate understanding of this topic matter...

My experience includes a peer reviewed paper on similar subject matter and also past experience writing industry standards documents, along with past experience submitting VSYNC-related and refresh-related bug reports in BugZilla-type systems to Chrome/FireFox (that got successfully fixed, e.g. this, this, this, and a few dozen others. I'm username "mdrejhon" and "blurbusters" on various bug tracking systems)

As a result, I believe I'm a great candidate to participate in improving W3C standardization involving this sphere of update-frequencies including requestAnimationFrame() given my extensive knowledge of display technologies. I'd like to participate in future work, possibly #375.

If not sure, please read my earlier (bigger) post from a few days ago -- to realize the complexity of the huge number of use cases that makes the orignal wording flawed.

[blurbusters]

I have commited to my fork.
Ready to submit a pull request, if I can be approved. https://github.com/mdrejhon/html/commit/bb06ea15f46c7d622ce089e067967380929ddddb

EDIT: I've been contacted by @chaals -- now privately communicating to go through the proper channels & application process to be invited in this little corner of standardization expertise. That said, further discussion is welcome regarding my proposed patch

EDIT2: I've successfully joined the W3C Web Platform Working Group (as an Invited Expert) and look forward to future collaboration in this specific sphere of expertise. The commit appears ready for official review

[blurbusters]

For testing purposes, I purchased a new 240Hz gaming monitor (Acer Predator XB252Q) and did browser tests on it.

Between late 2016 and April 2017 there are five new 240Hz gaming monitors on the market that are capable of a native 240Hz refresh rate (not "fake frames" / not interpolated).

(Interesting video cable bandwidth note: The bandwidth of 4K 60Hz is the same as 1080p 240Hz)

The current versions of Chrome and FireFox synchronized perfectly in TestUFO motion tests at 240fps (on a NVIDIA GPU accelerated desktop) without using much CPU or GPU at all.

However, Microsoft Edge exhibited framerate-divisor results on requestAnimationFrame() callback rate. 60Hz -- Microsoft Edge runs at 60fps 100Hz -- Micrsoft Edge runs at 100fps 120Hz -- Microsoft Edge runs at 60fps 144Hz -- Microsoft Edge runs at 72fps 200Hz -- Microsoft Edge runs at 100fps 240Hz -- Microsoft Edge runs at 80fps

SIDE NOTE -- For readers who doubt 240Hz makes a visible difference I can confirm: Visually, during TestUFO tests there is unamious confirmation (all visitors confirmed they could see a difference in TestUFO tests -- 6 out of 6 people so far -- myself included) that 240Hz syunchronized animations appear clearer than 120Hz -- there is roughly half the amount of motion blur at 240fps on this LCD panel than at 120fps at www.testufo.com/photo tests. Mathematically, on non-flicker displays (sample-and-hold displays such as LCD) -- and as long as LCD GtG response can keep up -- the amount of eye-tracking-based display motion blur is directly proportional to static frame visibility time in a moving eye-tracking situation. Static frames are blurred across retinas as eyes tracks across a static frame. (and motion demo of reducing motion blur by reducing frame visibility time via black frame insertion). Increasing refresh rate (and matching framerate) is another method of shortening frame visibility time, reducing eye-tracking-based motion blur even further. The motion clarity improvement of 240fps versus 120fps also clearly showed up in other fast-pan-motion situations (e.g. fast scrolling, fast Google Maps panning, etc) as long as the mouse report rate was higher than the refresh rate (many new midrange & up mice are now capable of 1000Hz mouse cursor position report rate)

In Microsoft Edge, scrolling runs at full frame rate, so the updaterate for scrolling is different from the artificially-capped callback rate for requestAnimationFrame(). The new commit discourages an arbitrary artificial cap in unconstrained situations (An example of an unconstrained situation would be a desktop computer on 120Hz-and-up monitors).

(BTW, in an extreme test, I ran Chrome with the command line option "--disable-gpu-vsync" and Chrome managed to run requestAnimationFrame almost 2,000 frames per second on a five year old gaming computer with NVIDIA card -- for those readers doubting browsers are fast enough to keep up)

Here is a zip file of all Microsoft Edge screenshots, for downloading.

[blurbusters]

@duckware, following up:

Propose for inclusion in #785:

rAF callback time arg: Setting the time argument to 'now' adds no value. Because of this, web browser vendors have already started to switch over to passing in the actual vsync time (Chrome / Firefox) as the argument to the rAF callback (which actually does add a lot of value). However, Chrome is currently the only browser that does this correctly. Firefox actually passes in a faked time (http://www.vsynctester.com/firefoxisbroken.html). Please strengthen the specification to make the rAF time argument be the exact vsync time.

I am also familiar with this. Prevailing practice clearly appears to be heading in the direction you're indicating. In theory, this warrants a separate github issue, but I could make a second pull request if there's industry agreement. We'll have to fish it out at least to a few outsiders as I'm not as dependant on this detail.

I definitely know correct handling of this parameter is critical to much better stutter-detection heuristic algorithms in TestUFO too (in theory, it's also possible #375 someday makes that unnecessary, but regardless -- this matter is clearly a flaw/inconsistency that needs to be fixed).

It's super easy to mathematically detect stutters in Chrome because the frame-drops and stutters clearly show up as aberrations in Performance.now() microsecond timer readings (a single stutter can interfere with scientific motion tests running in browsers) -- but it would be preferable to trust the parameter instead. To me, it is amazing that today, Chrome can support precision motion tests with enough Performance.now() accuracy that I feel confident which frames made it to which refresh cycle (even confirmed by co-relation to my high-speed camera tests) -- enough self-analysis accuracy to feel confident I'm getting an exact frame delivered to an exact chosen refresh cycle. This wasn't something JavaScript/ECMAScript was designed to be able to do but is already being done by both vsynctester (your creation) and testufo (my creation). This was formerly the exclusive domain of executable files such as "PixPerAn" or "DisplayMate Motion Bitmaps Edition" as well as other software that runs with $30K lab equipment (e.g. MotionMaster) where one single stutter invalidates a specific test run. Now already being done today in Chrome!

I think THIS part belongs as a separate (second) commit under this #785. So your minor recommendation change will strengthen the standard further, IMHO. Now it's a wording matter, and I'd like your assistance on this, @duckware.

(I've reached out to duckware by email too)

---

Propose moving this to #375:

As just as a very interesting FYI. On Windows, all rAF animations in all web browsers have a two frame input lag caused by how all browsers interact with the GPU. Apparently what is happening is that vsync triggers the rAF callback, but 'what was drawn' is not presented until the next vsync, but at that point, it is too late to make it to the screen (because the Windows OS composites). What was drawn in the rAF callback actually makes it to the screen one entire frame later. It sure would be nice if web browser vendors took notice, and (under Windows), presented what was drawn in a rAF callback immediately, instead of waiting for the next vsync (which then delays an entire frame).

This probably could be addressed as part of #375 to make browsers accelerate compositing (reduced lag at more stutter risk) or delay compositing to minmize stutter. For a long time, compositing was a very slow operation, but right now, midrange Geforce cards can composite all windows and framebuffers in less than one millisecond.

According to Microsoft Research, there's a very human-noticeable difference between 100Hz and 1000Hz when it comes to touchscreen, as seen in their YouTube video.

A cursor lagging behind by 1/100sec versus a cursor lagging behind 1/1000sec can be quite noticeable during fast drawing operations.

Especially when browser apps are used on touchscreens (e.g. pencil sketching app), there's much more lag on touchscreen Windows monitors than on Apple iPad touchscreens, due to the slower Windows 10 WDM compositing than Apple's low-latency iPad compositing. I believe the iPad probably composites in the same refresh cycle as app's creating of frame buffer (e.g. HTML5 canvas or WebGL canvas), since nowadays GPUs are fast enough to draw (3D graphics, even!) AND composite in the same refresh cycle.

While Windows WDM has long been stuck with compositing in the next refresh cycle (adding another refresh cycle of lag) -- browser compositing, THEN windows compositing -- when ideally it could have all been done in the same refresh cycle even on a 5-year old Intel Integrated GPU. And consider, modern GPUs can now composite at up to half a terabyte per second on a GTX Titan (4K now composited in less than a millisecond on the fastest GPU on the market, and experimental 8K120Hz managed to run at full framerate in Google Chrome on TestUFO on an experimental display at CES 2017 in a clandestine "does it work?" test)... So the idea that compositing belongs in a separate subsequent refresh cycle is now an outdated idea...

The advantage of the Windows 10 approach is requestAnimationFrame() can have nearly a full refresh cycle of drawing, without worrying about additional delays caused by compositing. This can make things smoother especially on slower GPUs, but as you already pointed out -- increases input latency. Ideally, browser apps should choose (as executable files are currently able to).

Today, compositing is approaching near negligible overhead on modern GPUs, even mobile GPUs -- e.g. the current A7s and up manages to composite in less than a millisecond.

Even browser capability (without Javascript access to it) is already emerging in a hidden way: Chrome seems to provide command-line options that affects this calculus partially, so making it a simpler matter (for some browser vendors) to consider letting JavaScript level be able to control this...

That said, I think this part of the change belongs under #375 as a VSYNC API related topic. I think it's an excellent idea to touch upon Javascript ability to decide how much latency they want (stutter-versus-fluidity tradeoffs, etc) just as native applications currently are able to nowadays, and even useragents (Chrome) able to via command line options (such as --disable-gpu-vsync) that should eventually be able to (in some form) be exposed to the script level.

Once we're satisfied with #785 (easy, no new API), as an Invited Expert with some pre-existing contacts at Blur Busters, I may be able to invite some industry representatives (e.g. Microsoft, Google, Valve Software, Epic Megagames, 2K, Oculus, etc) to comment on #375 (harder, new APIs) standardization matters. Please note, since it's new APIs added to HTML, I am not sure if we can get enough traction on #375 by HTML 5.2 but certainly by HTML 5.3.

[blurbusters]

Thanks for merging the commit. So this is now part of HTML 5.2 DRAFT 8.

Current status of browsers as of 5/1/2017:

-- PASS: Blink/Gecko/Webkit aka Chrome/Opera/FireFox/Safari are the ones that are currently compliant. (Chrome/FireFox even work full framerate on 240Hz monitors.)

-- FAIL: Trident/EdgeHTML aka IE/Edge has the arbitrary hardcoded cap (probably 1-line code) that becomes hit in unconstrained situations such as high-performance desktops. As soon as I exceed exactly 105Hz, the framerate suddenly halves (even at <1% CPU).

EDIT -- An article on Blur Busters about a VSYNC API has been created to help generate further industry discussion on this topic matter.

[smfr]

WebKit also has interest in extending requestAnimationFrame given that the newly announced iPads Pro have both variable refresh-rate displays, with up to 120Hz screen updates.

However, we have to balance giving developers access to higher refresh rates with power and security considerations.

First, we avoided simply allowing rAF to fire at 120Hz on the new iPads because it would have significant impacts on battery life for almost no visible benefit on most pages. I think anything > 60Hz needs to be opt-in.

Second, I think it's almost more important to allow developers to opt in to lower-frequency rAF, by telling the browser that their animation only needs, say, 24fps, which would allow for significant power savings by reducing the number of process wake-ups to avoid rads which do nothing.

Third, we have to avoid making it easier for content to do timing attacks (which usually measure how long some specific set of drawing commands take). Disabling VSYNC would make these kinds of attack much easier.

[blurbusters]

Hello,

Note: For those unaware I am the author of: http://www.blurbusters.com/blur-busters-working-on-changes-to-html-5-2/ And I submitted a change to HTML 5.2 for the 120Hz era: https://github.com/w3c/html/issues/785

I should note that on desktop computers, some WebKit based browsers already support for 120Hz requestAnimationFrame, so it appears to be browser-vendor specific.

When Chrome was using WebKit, it was working great with 120Hz. In fact, it works great with 240Hz too (five new true 240Hz gaming monitors have hit the market in the last few months, including the BenQ Zowie XL2546 and the ASUS ROG PG258Q).

>>"First, we avoided simply allowing rAF to fire at 120Hz on the new iPads because it would have significant impacts on battery life for almost no visible benefit on most pages. I think anything > 60Hz needs to be opt-in."

Disabling VSYNC can be permission-based (for security reasons as well as battery-consumption reasons, as indicated). What you suggest is a very theoretical very rare kind of certain timing attacks are theoretically possible (e.g. timing the length of draw operations) but this is something already possible with shader benchmarking as well as forcing animation frames to take longer than 1/60sec (to time draw operations without needing VSYNC OFF). However, WebGL is far more insecure than supporting VSYNC OFF, but this is being mentioned anyway for completeness' sake -- alongside full screen mode security considerations -- and another rationale for permission-based operation for requesting VSYNC OFF.
Workarounds exist such as adding random jitter (random microseconds) to VSYNC OFF can also reduce security considerations too (to becoming far safer than WebGL or alternate techniques, for example). However, generally supporting fixed-rate 120fps/144fps/240fps should not be permission-based; the existence of a high-Hz monitor is defacto permission.

Personally, I disagree about end-user opt-in except when a "Power Saver" mode is enabled. It can be configurable, and can be scaled back, e.g. power management, but it should be 120Hz-permitted-by-default. rAF at a higher rate should be made available unconditionally "IF AND ONLY IF" (A) the screen is already 120Hz (B) hardware acceleration is available

Today, GPUs can do 120Hz for almost no extra power consumption than 60Hz, according to my power-consumption tests and Moore's Law still marches on. Desktop Chrome browser can run at 2000 frames per second with the command line "--disable-gpu-vsync", so rendering at 120 frames per second does not use much more battery than 60 frames per second.

I feel that the plain fact is if the user has a 120Hz display, the user has strongly and loudly declared a very strong opt-in for 120Hz, and therefore. Metaphorically, we should not be the policeman to prevent 120 frames per second on the end user's 120Hz display. I feel it is tantamount to limiting framerate to 30fps on a 60Hz display.

Too often, standards designers design by committee, often never having seen the use-case before (e.g. never having seen 120Hz). I'm very familiar with this: People who have often never having seen 30fps-vs-60fps in 1990, and now today, often never having seen 60fps-vs-120fps in year 2017. One declares an assumed cap, and the user experience is affected. (Back in year 1992, I programmed a 30fps-versus-60fps app to prove the human eye could see the difference, and put it on a BBS)

Yes, "Power Saver Mode" could include framerate slowdowns (even all the way down to 15 frames per second, where it actually makes a real difference compared to 60fps-vs-120fps)

Yes, when pages become static (no scrolling, no animations), system can lower refresh rate to save power regardless of 60Hz or 120Hz. The idle moments between scrolls and between animations can be made quite power miserly.

Moral of story: Gain everyday 120Hz experience first, and most will agree asking user permission for it is silly.

>>"Second, I think it's almost more important to allow developers to opt in to lower-frequency rAF, by telling the browser that their animation only needs, say, 24fps, which would allow for significant power savings by reducing the number of process wake-ups to avoid rads which do nothing."

Yes... I think this is important too.

However, I want to respond to the "almost more important" phrase, which warrants a long reply due to the possible "red herring" factor.

For example -- have you seen 120fps before? Many people have not yet. But as owner of Blur Busters, we are intimately familiar with it. It's amazingly a huger difference than the current 3:2 pulldown we use to display 24fps on a 60Hz display. There's half as much motion blur during fast swipe-scrolling, making it much easier to read text while the browser screen scrolling. With properly perfectly synchronized scrolling (stutter free smooth scrolling -- e.g. up/down arrow key in desktop browsers, or finger scrolling on newest phones/tablets) -- the move from 60fps to 120fps makes a much more visible user experience improvement than removing 3:2 pulldown from 24fps videos. (BTW, I'm the developer of the world's first open-sourced 3:2 pulldown detection algorithm -- back in 1999 for the "dScaler" deinterlacing app). Yes, the "24p" playback fluidity is wonderful, but "120Hz scrolling" is even more amazing-er. Personally, get a few 120Hz devices, play with them, THEN make a decision by consensus. I think you'll agree with me by then, that none of us should cap to 60fps and none of us should ask the user for permission to do 120fps.

The user experience improvement of finding text faster during scrolling, is tremendously huge, since the doubled refresh rate mathematically halves eye-tracking-based motion blur (for a demonstration of this, see www.testufo.com/eyetracking -- an optical illusion I invented). It's also written in several scientific papers that the sample-and-hold effect (eye-tracking across statically displayed frames, blurs the frame across the retinas).

However, the power management aspects of slowing framerate from 60Hz to 24Hz, during movie playback, can actually add a few tens extra minutes to the playback time (That said, in my experience, the H.264 / H.265 decoding uses more power than the refresh rate difference between 24Hz and 60Hz), But yes, what's important is Variable Refresh Rate support:

• Synchronize to the framerate of video (fixed custom refresh rate)
• Synchronize to the framerate of video games (dynamically varying framerate).

.......This eliminates stutter; see software simulation of stutter-free varying framerate at www.testufo.com/stutter&demo=gsync if you don't have experience with FreeSync, GSYNC, VESA Adaptive-Sync, or HDMI 2.1 Variable Refresh Rate. Now that Apple has VRR support, this will bring VRR into the mainstream and we may need to support WebGL canvas / requestAnimationFrame() at a variable refresh rate in the future. Basically, refresh cycles are delivered immediately at the instant of the exit of those respective canvas-refresh functions, keeping time presentation of screen in sync with render time, reducing stutter in the light of continuously variable refresh rate situations. The refresh rate can change over 100 times a second in this situation.

It's possible that iPads will only support selectable variable refresh rate (app-selectable, once per startup of video playback) but the LCD controller also supports dynamic variable refresh rate (refresh rate changes every single refresh cycle). I'm not sure of iOS 11 supports continuously dynamic variable refresh rate for video games (much like NVIDIA GSYNC and AMD FreeSync, HDMI 2.1 VRR, and VESA Adaptive-Sync standards) but knowing Apple, they will eventually enable this support for iPad video games.

So one needs to support two things simultaneously and separately:

(1) Ability to sync to a specific refresh rate (request Hz synchronization to a fixed-framerate source, like video)

(2) Ability to continually vary the refresh rate (refresh the display exactly on completion on renders)

Most VRR displays will support both (1) and (2) but some may only support (1) due to an artificial OS limitation (e.g. iOS11 supporting (1) then iOS12 supporting both (1) and (2) ....). (Side Question: Does iOS11 supports both (1)+(2) or only (1) via API support?

Also responsibility may not even be in the browser at all except to select the synchronization source. (e.g. two simultaneous videos). What browser standards designers should probably do is:

• Enable variable refresh rate in full screen mode only (both fixed-Hz and dynamic-Hz)
• Full screen mode should sync to the framerate of the first

Tricky situations:

• Multiple videos and multiple canvas
• Multiple windows side by side

Potential Simplifications:

• Enable VRR support only in full screen mode

Possible rules:

• Browser vendor can choose to sync the Hz to the dominant viewport (first
• Things like requestAnimationFrame() is a trickier case that needs a specific sets of rules. By default, it should run at a fixed Hz (highest Hz possible of display, 120Hz, when not in a power saver mode). But should be able (at browser vendor's choosing) to dynamically sync Hz whenever framerate slows down. The browser programmer should also be able to decide to specify a fixed frequency (e.g. 24fps, 30fps, 60fps, etc) and the Hz will sync exactly to framerate. (Just as it does today on GSYNC and FreeSync).

Current prevailing practice on PC: In videogames, many videogames aren't even aware they're running at a variable refresh rate. Because there's no difference between framerate and refreshrate, they are the same thing when it comes to a dynamically-changing VRR display. At least when running in VRR range (e.g. 30fps-144fps on a 144Hz GSYNC display -- and 30fps-240fps on a 240Hz GSYNC display). If the game runs at 43.7 frames per second, the display is 43.7Hz, there's no difference between frame rate and refresh rate in that situation, and the refresh rate can change every single refresh cycle / every single frame.

Cheers, Mark Rejhon

[blurbusters]

Current prevailing practice on PC: In videogames, many videogames aren't even aware they're running at a variable refresh rate. Because there's no difference between framerate and refreshrate, they are the same thing when it comes to a dynamically-changing VRR display. At least when running in VRR range (e.g. 30fps-144fps on a 144Hz GSYNC display -- and 30fps-240fps on a 240Hz GSYNC display). If the game runs at 43.7 frames per second, the display is 43.7Hz, there's no difference between frame rate and refresh rate in that situation, and the refresh rate can change every single refresh cycle / every single frame.

In other words, on a variable refresh display, individual refresh cycles are triggered by software (e.g. graphics driver) -- like the completion of a draw cycle, or an OpenGL glFlush() -- or theoretically the return from requestAnimationFrame() -- or the completion of a video frame from the video decoder.

Important Questions about iOS 11 ProMotion refresh workflow So on this note, before we decide on standardization, I have some important questions about how iOS 11 handles ProMotion:

(A) Does ProMotion sync rendering to display? aka iPad tells display "I want to run at 24Hz, and I'll deliver video frames synchronously"

(B) Does ProMotion sync display to rendering? (like GSYNC, FreeSync, HDMI 2.1 VRR, VESA Adaptive-Sync) aka iPad tells display "Please refresh immediately every time I deliver this frame buffer"

(C) Does ProMotion support continuously dynamic variable frame rate? (like GSYNC, FreeSync, HDMI 2.1 VRR, VESA Adaptive-Sync) aka iPad can change refresh rate every frame render.

We probably need to answer all the above questions before we decide exactly how to standardize this in HTML 5.2. Ideally, I think we should standardize for all possible eventualities.

Also it's theoretically possible both (A) and (B) could be a yes -- my experience is GSYNC and FreeSync displays are capable of any random fixed refresh rate too, in addition to letting software trigger individual refresh cycles.

Thanks, Mark Rejhon

On Tue, Jun 6, 2017 at 10:46 AM, Mark Rejhon mark@realjabber.org wrote:

Hello,

Note: For those unaware I am the author of: http://www.blurbusters.com/blur-busters-working-on- changes-to-html-5-2/ And I submitted a change to HTML 5.2 for the 120Hz era: https://github.com/w3c/html/issues/785

I should note that on desktop computers, some WebKit based browsers already support for 120Hz requestAnimationFrame, so it appears to be browser-vendor specific.

When Chrome was using WebKit, it was working great with 120Hz. In fact, it works great with 240Hz too (five new true 240Hz gaming monitors have hit the market in the last few months, including the BenQ Zowie XL2546 and the ASUS ROG PG258Q).

>>"First, we avoided simply allowing rAF to fire at 120Hz on the new iPads because it would have significant impacts on battery life for almost no visible benefit on most pages. I think anything > 60Hz needs to be opt-in."

Personally, I disagree about end-user opt-in except when a "Power Saver" mode is enabled. It can be configurable, and can be scaled back, e.g. power management, but it should be 120Hz-permitted-by-default. rAF at a higher rate should be made available unconditionally "IF AND ONLY IF" (A) the screen is already 120Hz (B) hardware acceleration is available

Today, GPUs can do 120Hz for almost no extra power consumption than 60Hz, according to my power-consumption tests and Moore's Law still marches on. Desktop Chrome browser can run at 2000 frames per second with the command line "--disable-gpu-vsync", so rendering at 120 frames per second does not use much more battery than 60 frames per second.

I feel that the plain fact is if the user has a 120Hz display, the user has strongly and loudly declared a very strong opt-in for 120Hz, and therefore. Metaphorically, we should not be the policeman to prevent 120 frames per second on the end user's 120Hz display. I feel it is tantamount to limiting framerate to 30fps on a 60Hz display.

Too often, standards designers design by committee, often never having seen the use-case before (e.g. never having seen 120Hz). I'm very familiar with this: People who have often never having seen 30fps-vs-60fps in 1990, and now today, often never having seen 60fps-vs-120fps in year 2017. One declares an assumed cap, and the user experience is affected. (Back in year 1992, I programmed a 30fps-versus-60fps app to prove the human eye could see the difference, and put it on a BBS)

Yes, "Power Saver Mode" could include framerate slowdowns (even all the way down to 15 frames per second, where it actually makes a real difference compared to 60fps-vs-120fps)

Moral of story: Gain everyday 120Hz experience first, and most will agree asking user permission for it is silly.

>>"Second, I think it's almost more important to allow developers to opt in to lower-frequency rAF, by telling the browser that their animation only needs, say, 24fps, which would allow for significant power savings by reducing the number of process wake-ups to avoid rads which do nothing."

Yes... I think this is important too.

However, I want to respond to the "almost more important" phrase, which warrants a long reply due to the possible "red herring" factor.

For example -- have you seen 120fps before? Many people have not yet. But as owner of Blur Busters, we are intimately familiar with it. It's amazingly a huger difference than the current 3:2 pulldown we use to display 24fps on a 60Hz display. There's half as much motion blur during fast swipe-scrolling, making it much easier to read text while the browser screen scrolling. With properly perfectly synchronized scrolling (stutter free smooth scrolling -- e.g. up/down arrow key in desktop browsers, or finger scrolling on newest phones/tablets) -- the move from 60fps to 120fps makes a much more visible user experience improvement than removing 3:2 pulldown from 24fps videos. (BTW, I'm the developer of the world's first open-sourced 3:2 pulldown detection algorithm -- back in 1999 for the "dScaler" deinterlacing app). Yes, the "24p" playback fluidity is wonderful, but "120Hz scrolling" is even more amazing-er. Personally, get a few 120Hz devices, play with them, THEN make a decision by consensus. I think you'll agree with me by then, that none of us should cap to 60fps and none of us should ask the user for permission to do 120fps.

The user experience improvement of finding text faster during scrolling, is tremendously huge, since the doubled refresh rate mathematically halves eye-tracking-based motion blur (for a demonstration of this, see www.testufo.com/eyetracking -- an optical illusion I invented). It's also written in several scientific papers that the sample-and-hold effect (eye-tracking across statically displayed frames, blurs the frame across the retinas).

However, the power management aspects of slowing framerate from 60Hz to 24Hz, during movie playback, can actually add a few tens extra minutes to the playback time (That said, in my experience, the H.264 / H.265 decoding uses more power than the refresh rate difference between 24Hz and 60Hz), But yes, what's important is Variable Refresh Rate support:

• Synchronize to the framerate of video (fixed custom refresh rate)
• Synchronize to the framerate of video games (dynamically varying framerate).

.......This eliminates stutter; see software simulation of stutter-free varying framerate at www.testufo.com/stutter&demo=gsync if you don't have experience with FreeSync, GSYNC, VESA Adaptive-Sync, or HDMI 2.1 Variable Refresh Rate. Now that Apple has VRR support, this will bring VRR into the mainstream and we may need to support WebGL canvas / requestAnimationFrame() at a variable refresh rate in the future. Basically, refresh cycles are delivered immediately at the instant of the exit of those respective canvas-refresh functions, keeping time presentation of screen in sync with render time, reducing stutter in the light of continuously variable refresh rate situations. The refresh rate can change over 100 times a second in this situation.

It's possible that iPads will only support selectable variable refresh rate (app-selectable, once per startup of video playback) but the LCD controller also supports dynamic variable refresh rate (refresh rate changes every single refresh cycle). I'm not sure of iOS 11 supports continuously dynamic variable refresh rate for video games (much like NVIDIA GSYNC and AMD FreeSync, HDMI 2.1 VRR, and VESA Adaptive-Sync standards) but knowing Apple, they will eventually enable this support for iPad video games.

So one needs to support two things simultaneously and separately:

(1) Ability to sync to a specific refresh rate (request Hz synchronization to a fixed-framerate source, like video)

(2) Ability to continually vary the refresh rate (refresh the display exactly on completion on renders)

Most VRR displays will support both (1) and (2) but some may only support (1) due to an artificial OS limitation (e.g. iOS11 supporting (1) then iOS12 supporting both (1) and (2) ....). (Side Question: Does iOS11 supports both (1)+(2) or only (1) via API support?

Also responsibility may not even be in the browser at all except to select the synchronization source. (e.g. two simultaneous videos). What browser standards designers should probably do is:

• Enable variable refresh rate in full screen mode only (both fixed-Hz and dynamic-Hz)
• Full screen mode should sync to the framerate of the first

Tricky situations:

• Multiple videos and multiple canvas
• Multiple windows side by side

Potential Simplifications:

• Enable VRR support only in full screen mode

Possible rules:

• Browser vendor can choose to sync the Hz to the dominant viewport (first
• Things like requestAnimationFrame() is a trickier case that needs a specific sets of rules. By default, it should run at a fixed Hz (highest Hz possible of display, 120Hz, when not in a power saver mode). But should be able (at browser vendor's choosing) to dynamically sync Hz whenever framerate slows down. The browser programmer should also be able to decide to specify a fixed frequency (e.g. 24fps, 30fps, 60fps, etc) and the Hz will sync exactly to framerate. (Just as it does today on GSYNC and FreeSync).

Current prevailing practice on PC: In videogames, many videogames aren't even aware they're running at a variable refresh rate. Because there's no difference between framerate and refreshrate, they are the same thing when it comes to a dynamically-changing VRR display. At least when running in VRR range (e.g. 30fps-144fps on a 144Hz GSYNC display -- and 30fps-240fps on a 240Hz GSYNC display). If the game runs at 43.7 frames per second, the display is 43.7Hz, there's no difference between frame rate and refresh rate in that situation, and the refresh rate can change every single refresh cycle / every single frame.

Cheers, Mark Rejhon

[blurbusters]

Moved discussion to #375

Please note #375 description was completely rewritten (after this 785), so #375 is much more recent than 785. Here are some important comments to catch up on:

Proposed new API is written here: PHASE 1 and PHASE 2. Comments are encouraged in those new w3c github threads over there.

June 6th, 2017:

Potential HTML 5.2 Variable Refresh Rate Standardization Path

PHASE 1

Simplified Goals

• Zero new APIs added initially
• Theoretical browser-vendor path without additional discovery APIs, command arguments, etc
• Leave most Variable Refresh Rate (VRR) responsibilities to OS / drivers (including automatic VRR-related power management).

Simplified Rules

• Automatically engage javscript-controlled VRR support only in full screen mode
• Automatically sync refresh to the first or biggest visible viewport (e.g. <video> or <canvas> tag)
• At top level, compositing allowed to use VRR to eliminate performance stutter & to idle on non-animated pages (if not done automatically by OS).
• Limiting VRR support to full-screen mode simplifies platform considerations for certain platforms (e.g. needing to use Direct3D/OpenGL to signal the display to immediately begin an asynchronous refresh cycle, even for plain text browsing or playing videos)

Satisfies The Following

PARTIAL - (1) Easiest support for frame rates intentionally decoupled from refresh rate (higher & lower)

In full screen or maximized mode, the browser vendor makes decision to synchronize to the dominant viewport. The refresh cycle workflow is either:

1. individually triggered refreshes: triggered on delivery of framebuffer (video frame, webGL frame, requestAnimationFrame); if supported. OR
2. custom fixed Hz: triggered by OS automatically (e.g. automatically by ProMotion fixed H.264 decoder rate during full screen videos)

No HTML coding changes to existing HTML apps are necessary. "Partial" support denotes lack of support for frame rates higher than max Hz of display

RESOLVED - (2) Easiest support for fixed-custom-Hz (e.g. video, animations)

Resolved via (1) above.

RESOLVED - (3) Easiest support for dynamically-varying-Hz (e.g. games)

Resolved via (1) above.

Unresolved - (4) Easiest support for regular HTML

The easy path has no support for JavaScript specifying a custom HTML5 compositor rate (frame rate = refresh rate) since that requires adding a new API.
Exception: However, the top-level compositor may gracefully slow down the rate (for power management or for stutter-reduction), letting the graphics driver or OS immediately refresh the screen. A device "power saver" mode may automatically use a lower refresh rate, or performance-demands spikes may slow down the compositor frame rate (and thus, the refresh rate of a variable refresh rate)

RESOLVED - (5) Easiest support for <video>

Resolved via (1) above.

RESOLVED - (6) Easiest support for <canvas> via WebGL

Resolved via (1) above.

RESOLVED - (7) Easiest support for <canvas> via requestAnimationFrame()

Resolved via (1) above.

Special Note: about App-requested refresh rate: requestAnimationFrame already can be made to run via timer (e.g. if Javascript wants a 45Hz refresh rate, it simply runs requestAnimationFrame() at 45 times a second as already possible today; the OS/driver simply executes the refresh cycle upon delivery of a frame buffer). You'd simply call requestAnimationFrame inside a timer event, and the browser would immediately trigger the call to the callback. That way, no HTML API change is needed to allow a browser app developer do a custom refresh rate with <canvas>. Five years ago, you can already do this with existing HTML, run animation at a timer-based 45fps, but it stutters a lot (45fps at 60Hz is bad) -- however, if you do 45fps on a variable refresh rate display, it plays naturally stutterfree, with no app or browser intervention, except simply by delivering the framebuffer to the operating system on a timer. That's it. Very easy.

Special Note: about Gracefully slowed-down frame rates due to performance: Say, a game tries to run at 60fps but doesn't have enough performance, and it only runs at 53fps instead. This is what happens if requestAnimationFrame() is called inside the requestAnimationFrame() callback, then the rate will max out at the display's maximum refresh rate except when throttled for any reason. On a variable refresh rate, whenever framerate slows down, it is not necessary to run requestAnimationFrame at fixed intervals. Variable refresh rate can de-stutter / de-jitter erratic frame-delivery timings, as long as game time rendering is in sync with refresh cycle times. Prevailing practice is most games & animations already do this, in order to keep motionspeed consistent independently of frame rates. As a side-effect, such logic is automatically compatible with stutterfree variable-framerate motion on a variable frame rate display. For those unfamiliar -- it's actually almost miraculous to see existing 15-year-old source code, not invented for variable refresh rate -- whenever it is struggling to run at its fastest framerate -- still remarkably successfully stutterlessly run at exactly 53 frames per second at exactly 53 Hertz on a variable refresh rate display. This is in thanks to the operating system simply triggering refresh cycles on frame buffer delivery. Even if the software is 15 years old, the operating system can successfully give it variable refresh rate support! As long as the animations are already designed to run off gametime (and automatically adapt to fluctuating frame rates as they already do 15 years ago), it's usually beautifully compatible with variable refresh rate: The stutters miraculously disappears, much like this animation demo: www.testufo.com/stutter#demo=gsync

Unresolved - (8) Adds improved discovery (e.g. query for the display refresh rate).

There is often a need to query for the refresh rate of a display (fractionally too -- use a float!).
This is in Phase 2 (which can be done concurrently with Phase 1)

Potential HTML 5.2 Variable Refresh Rate Standardization Path

PHASE 2

Please see Phase 1 first.

Phase 2 Addition of APIs to browsers

APIs can initially have prefixes during incubation ("moz", "o", "webkit", etc)

• Add API to discover current refresh rate
• Add API to discover whether display supports VRR
• Add API to discover whether VSYNC is ON/OFF
• Add API to disable VSYNC to permit frame rates above refresh rate.
• Add API to discover whether DWM compositor is being bypassed for lower lag

API to discover refresh rate: screen.hz

Existing Similar Practice: Currently we already have screen.width and screen.height which returns the dimensions of the monitor. (For a multi-monitor setup, existing browsers currently returns the resolution of the monitor that the browser window is on).
Proposed API: The API is proposed to be "screen.hz" (readonly) because "hz" is (mostly) more language neutral than "refreshrate" and easier to type.

• It SHOULD return a floating-point value where possible, since refresh rate can be fractional.
• On a variable refresh rate display, this always returns the current maximum bounds of the variable refresh rate range. (e.g. If the current range is set to 24.000Hz-120.000Hz, then it should return "120.000000"). This is the prevailing practice on other pre-existing VRR APIs on other platforms, to simply return maximum refresh rate, and let software decide on a lower refresh rate simply by timing frame buffer deliveries (even erratically/asynchronously too!).
• If this is a fixed-frequency refresh rate change (e.g. 15Hz power saver mode, 24Hz movie mode, etc) where the software still needs to synchronize to the refresh cycles, then screen.hz should update to return this value.
• The value "screen.hz" shall be read-only. On a true continuously variable refresh rate display, there is no need to change to a lower refresh rate. Simply delivering framebuffers slower to the graphics driver, is all that is needed, since the refresh cycles are triggered upon frame buffer deliveries from the application to the OS/driver.

API to discover Variable Refresh support: screen.vrr

Existing Similar Practice: Currently we already have screen.width and screen.height Proposed API: The API should be screen.vrr (readonly) returning true|false whether the display is capable currently in variable refresh rate mode. This does not necessarily mean that Javascript currently has control over the VRR, since it may only be available to videos. VRR stands for Variable Refresh Rate, which is the current generic terminology. Be noted, that on a multi-monitor system, not all monitors may support VRR. Both screen.hz and screen.vrr will apply to current monitor that the browser window is on (moving the window already updates screen.height and screen.width in prevailing practice)

API to discover or configure VSYNC ON/OFF: screen.vsync

See this original post for more info why VSYNC OFF can be beneficial for certain important use cases.

Existing Similar Practice: The Chrome browser has a VSYNC OFF command line option, "--disable-gpu-vsync" to permit frame rates far above refresh rates. Basically, renders will run at an unthrottled rate (as much as the current CPU allotment & power management plan permits).
Proposed API: The API should be screen.vsync (readonly OR read/write) returning true|false whether VSYNC is turned off.

• Attempting to write screen.vsync = false would be the JavaScript equivalent of launching Chrome with "--disable-gpu-vsync".
• During screen.vsync = true browser frame rates are limited to max refresh rate.
• During screen.vsync = false browser frame rates would run at the fastest rate permissible, much like a video game's VSYNC OFF setting.
• During VSYNC OFF, the compositor would run at the fastest rate, WebGL, and requestAnimationFrame() would callback at the fastest rate. At the render completion, the next render cycle begin immediately. The net result is framerates far above refresh rate, with correspondingly reduced keyboard/mouse/touchscreen lag useful for critical fast-response / eSports / etc.
• If VSYNC is successfully enabled/disabled, the "screen.vsync" setting will update to the value written to it. If not successful, "screen.vsync" will behave as a read-only value (much like "screen.width" and "screen.height") -- the next read of the value would result in an unchanged value from the attempted write.
• USER PERMISSION RECOMMENDED: Running at uncapped frame rates far above refresh rates, potentially >1000fps, can be very power-consuming, and hog system resources. One person suggests a theoretical very rare kind of certain timing attacks are theoretically possible (e.g. timing the length of draw operations) but this is something already possible with shader benchmarking as well as forcing animation frames to take longer than 1/60sec (to time draw operations without needing VSYNC OFF). However, WebGL is far more insecure than supporting VSYNC OFF, but this is being mentioned anyway for completeness' sake -- alongside full screen mode security considerations -- and another rationale for permission-based operation for requesting VSYNC OFF. Thus, this should be opt-in. This should be a user-permission request similiar to full screen mode requests. Upon first call by a specific web page to screen.vsync = true should automatically prompt the user whether to permit VSYNC OFF, for ultra-high framerates needed for low-latency / eSports applications.

API to discover whether DWM compositor is being bypassed: screen.dwm

Existing Similar Practice: Full-screen-exclusive mode for Direct3D and OpenGL already bypasses desktop window manager compositor (DWM) in full-screen applications (Even most browsers currently don't use the full-screen-exclusive mode, even in full screen mode -- it's more a borderless window stretched to full screen of the window manager instead) Proposed API: The API should be screen.dwm (readonly) returning true|false whether desktop window manager compositor is currently being bypassed.

• This could be made to happen only during full screen mode
• It happens that variable refresh rate is supported under Windows only via graphics drivers via Direct3D and OpenGL frame buffer presentation APIs (such as Present() or glFlush() triggering the immediate beginnings of a refresh cycle).
• Using exclusive full screen mode presents a brand new opportunity to reduce keyboard/mouse input lag by using Direct3D/OpenGL to display the browser framebuffer (even for text, or for video frames). You'd gain variable refresh rate support, and lower input lag.
• screen.dwm might become true only during full screen mode.
• Discovery of this permits browers to automatically detect if the system is currently configured to the lowest latency. The simultaneous existence of "screen.vsync = false" and "screen.dwm = false" confirms the absolute theoretical lowest input-lag situation possible, and confirms that the web browser is eSports-ready for ultra-high-performance competitive purposes (where Olympics-style millisecond differences in reaction times matter a lot, such as simultaneous-draw situations in FPS shooter games)
• To be discussed: Should this be a read/write? By default, full screen mode still lets other windows overlap (if you Alt+Tab another window onto the top of a browser window). In exclusive-full-screen-mode, used by videogames for reduced lag, you lose this ability, and also the ability to bring up the Start Menu as quickly (During bringing up the Start Menu in the middle of a PC videogame -- the game either minimizes, goes windowed, or goes "borderless full screen windowed" with compositor temporarily reenabled). Perhaps the JavaScript developer should do a "screen.dwm = false" right after requestFullScreen() and make it permission-based too, as part of the requestFullScreen() call, since it can be somewhat disruptive to use the exclusive full screen mode.
• To be discussed: The behavior of "screen.vsync = false" simultaneously with "screen.vrr = true" will need to be defined later. Ideally "screen.vsync = false" may simply turn off VRR, or simply produce non-VRR behavior once framerates exceeds maximum refresh rate (like GSYNC + VSYNC OFF currently does -- tearing only occurs when framerates are above refresh rate).

Also, bypassing the compositor enables tearing during VSYNC OFF (which is normal). The ability to know we're in the lowest possible-latency mode can signal that the browser is "eSports compatible" (competitive gaming).

Important: Further replies should go to #375.

(The HTML5.2 modification commit was successfully accepted -- see original post at the very top of #785 -- so further VSYNC related changes are under scope of #375)

[blurbusters]

Hello all,

If there is anything I can do for W3C to create new motion-fluidity or jank-verification tests, that are also suitable for high refresh rates -- I am ready to help.

I also did a successful 480 Hz test in FireFox: http://www.blurbusters.com/4k-120hz-with-bonus-240hz-and-480hz-modes/

It is time for a new rAF() compliance benchmark web browsers within 6 months for their high-refresh-rate supports. Browsers that follow the HTML 5.2 standard running full-refresh-rate requestAnimationFrame() when on an A/C powered computer running in high priority mode. This is when power constraints do not matter and frame rate caps [e.g. like those found in IE, Edge] should not apply. This is important for scientific & educational testing, and for writing peer reviewed papers (I already have one. Also, Viewpixx now has a 1440Hz DLP research projector, but that is currently out of my price budget at the moment.

Regarding requestAnimationFrame support for high-refresh rates for scientific motion testing: I'm pleased to mention that FireFox apparently succeeded a 480 Hz test once I forced FireFox to disable it automatic CPU power management (via Minimum CPU = 100% via OS). Using the peer reviewed test technique, improvements are still indeed human-eye visible -- but that's offtopic.

I am going to re-benchmark all web browsers before the end of the years with brand new tests for 120Hz, 240Hz -- and now our new 480Hz capability. This will be the new HTML 5.2 rAF() compliance test.

Thanks, Mark Rejhon

On Tue, Jun 6, 2017 at 10:46 AM, Mark Rejhon mark@realjabber.org wrote:

Hello,

Note: For those unaware I am the author of: http://www.blurbusters.com/blur-busters-working-on- changes-to-html-5-2/ And I submitted a change to HTML 5.2 for the 120Hz era: https://github.com/w3c/html/issues/785

I should note that on desktop computers, some WebKit based browsers already support for 120Hz requestAnimationFrame, so it appears to be browser-vendor specific.

When Chrome was using WebKit, it was working great with 120Hz. In fact, it works great with 240Hz too (five new true 240Hz gaming monitors have hit the market in the last few months, including the BenQ Zowie XL2546 and the ASUS ROG PG258Q).

>>"First, we avoided simply allowing rAF to fire at 120Hz on the new iPads because it would have significant impacts on battery life for almost no visible benefit on most pages. I think anything > 60Hz needs to be opt-in."

Personally, I disagree about end-user opt-in except when a "Power Saver" mode is enabled. It can be configurable, and can be scaled back, e.g. power management, but it should be 120Hz-permitted-by-default. rAF at a higher rate should be made available unconditionally "IF AND ONLY IF" (A) the screen is already 120Hz (B) hardware acceleration is available

Today, GPUs can do 120Hz for almost no extra power consumption than 60Hz, according to my power-consumption tests and Moore's Law still marches on. Desktop Chrome browser can run at 2000 frames per second with the command line "--disable-gpu-vsync", so rendering at 120 frames per second does not use much more battery than 60 frames per second.

I feel that the plain fact is if the user has a 120Hz display, the user has strongly and loudly declared a very strong opt-in for 120Hz, and therefore. Metaphorically, we should not be the policeman to prevent 120 frames per second on the end user's 120Hz display. I feel it is tantamount to limiting framerate to 30fps on a 60Hz display.

Too often, standards designers design by committee, often never having seen the use-case before (e.g. never having seen 120Hz). I'm very familiar with this: People who have often never having seen 30fps-vs-60fps in 1990, and now today, often never having seen 60fps-vs-120fps in year 2017. One declares an assumed cap, and the user experience is affected. (Back in year 1992, I programmed a 30fps-versus-60fps app to prove the human eye could see the difference, and put it on a BBS)

Yes, "Power Saver Mode" could include framerate slowdowns (even all the way down to 15 frames per second, where it actually makes a real difference compared to 60fps-vs-120fps)

Moral of story: Gain everyday 120Hz experience first, and most will agree asking user permission for it is silly.

>>"Second, I think it's almost more important to allow developers to opt in to lower-frequency rAF, by telling the browser that their animation only needs, say, 24fps, which would allow for significant power savings by reducing the number of process wake-ups to avoid rads which do nothing."

Yes... I think this is important too.

However, I want to respond to the "almost more important" phrase, which warrants a long reply due to the possible "red herring" factor.

For example -- have you seen 120fps before? Many people have not yet. But as owner of Blur Busters, we are intimately familiar with it. It's amazingly a huger difference than the current 3:2 pulldown we use to display 24fps on a 60Hz display. There's half as much motion blur during fast swipe-scrolling, making it much easier to read text while the browser screen scrolling. With properly perfectly synchronized scrolling (stutter free smooth scrolling -- e.g. up/down arrow key in desktop browsers, or finger scrolling on newest phones/tablets) -- the move from 60fps to 120fps makes a much more visible user experience improvement than removing 3:2 pulldown from 24fps videos. (BTW, I'm the developer of the world's first open-sourced 3:2 pulldown detection algorithm -- back in 1999 for the "dScaler" deinterlacing app). Yes, the "24p" playback fluidity is wonderful, but "120Hz scrolling" is even more amazing-er. Personally, get a few 120Hz devices, play with them, THEN make a decision by consensus. I think you'll agree with me by then, that none of us should cap to 60fps and none of us should ask the user for permission to do 120fps.

The user experience improvement of finding text faster during scrolling, is tremendously huge, since the doubled refresh rate mathematically halves eye-tracking-based motion blur (for a demonstration of this, see www.testufo.com/eyetracking -- an optical illusion I invented). It's also written in several scientific papers that the sample-and-hold effect (eye-tracking across statically displayed frames, blurs the frame across the retinas).

However, the power management aspects of slowing framerate from 60Hz to 24Hz, during movie playback, can actually add a few tens extra minutes to the playback time (That said, in my experience, the H.264 / H.265 decoding uses more power than the refresh rate difference between 24Hz and 60Hz), But yes, what's important is Variable Refresh Rate support:

• Synchronize to the framerate of video (fixed custom refresh rate)
• Synchronize to the framerate of video games (dynamically varying framerate).

.......This eliminates stutter; see software simulation of stutter-free varying framerate at www.testufo.com/stutter&demo=gsync if you don't have experience with FreeSync, GSYNC, VESA Adaptive-Sync, or HDMI 2.1 Variable Refresh Rate. Now that Apple has VRR support, this will bring VRR into the mainstream and we may need to support WebGL canvas / requestAnimationFrame() at a variable refresh rate in the future. Basically, refresh cycles are delivered immediately at the instant of the exit of those respective canvas-refresh functions, keeping time presentation of screen in sync with render time, reducing stutter in the light of continuously variable refresh rate situations. The refresh rate can change over 100 times a second in this situation.

It's possible that iPads will only support selectable variable refresh rate (app-selectable, once per startup of video playback) but the LCD controller also supports dynamic variable refresh rate (refresh rate changes every single refresh cycle). I'm not sure of iOS 11 supports continuously dynamic variable refresh rate for video games (much like NVIDIA GSYNC and AMD FreeSync, HDMI 2.1 VRR, and VESA Adaptive-Sync standards) but knowing Apple, they will eventually enable this support for iPad video games.

So one needs to support two things simultaneously and separately:

(1) Ability to sync to a specific refresh rate (request Hz synchronization to a fixed-framerate source, like video)

(2) Ability to continually vary the refresh rate (refresh the display exactly on completion on renders)

Most VRR displays will support both (1) and (2) but some may only support (1) due to an artificial OS limitation (e.g. iOS11 supporting (1) then iOS12 supporting both (1) and (2) ....). (Side Question: Does iOS11 supports both (1)+(2) or only (1) via API support?

Also responsibility may not even be in the browser at all except to select the synchronization source. (e.g. two simultaneous videos). What browser standards designers should probably do is:

• Enable variable refresh rate in full screen mode only (both fixed-Hz and dynamic-Hz)
• Full screen mode should sync to the framerate of the first

Tricky situations:

• Multiple videos and multiple canvas
• Multiple windows side by side

Potential Simplifications:

• Enable VRR support only in full screen mode

Possible rules:

• Browser vendor can choose to sync the Hz to the dominant viewport (first
• Things like requestAnimationFrame() is a trickier case that needs a specific sets of rules. By default, it should run at a fixed Hz (highest Hz possible of display, 120Hz, when not in a power saver mode). But should be able (at browser vendor's choosing) to dynamically sync Hz whenever framerate slows down. The browser programmer should also be able to decide to specify a fixed frequency (e.g. 24fps, 30fps, 60fps, etc) and the Hz will sync exactly to framerate. (Just as it does today on GSYNC and FreeSync).

Current prevailing practice on PC: In videogames, many videogames aren't even aware they're running at a variable refresh rate. Because there's no difference between framerate and refreshrate, they are the same thing when it comes to a dynamically-changing VRR display. At least when running in VRR range (e.g. 30fps-144fps on a 144Hz GSYNC display -- and 30fps-240fps on a 240Hz GSYNC display). If the game runs at 43.7 frames per second, the display is 43.7Hz, there's no difference between frame rate and refresh rate in that situation, and the refresh rate can change every single refresh cycle / every single frame.

Cheers, Mark Rejhon