[css-will-change-1] proposal: will-change: integer-transform

[progers]

Proposal: will-change: integer-transform. This allows the browser to use high quality anti-aliasing when initially rasterizing the text, and continue using the same rasterized text as the transform changes. Engines that do not use subpixel anti-aliasing would treat this as an alias for will-change: transform.

In https://crrev.com/645003, chromium/blink stopped using subpixel anti-aliasing when will-change: transform is used, as it avoids needing to re-raster if the pixel alignment of the text changes. This idea is explicitly referenced in the spec:

If any non-initial value of a property would cause rendering differences on the element (such as using a different anti-aliasing strategy for text), the user agent should use that alternate rendering when the property is specified in will-change, to avoid sudden rendering differences when the property is eventually changed.

This change caused the Monaco editor (used by VSCode) to lose high-quality text rendering (see: https://crbug.com/1016062). This proposal is a standard way to maintain high-quality text while hinting that the transform will change.

[ewilligers]

I assume this is only about integer translation - not rotate/scale/skew.

Should it be integer-translate, and be an alias for will-change: transform translate?

[progers]

@ericwilligers, I like your idea; will-change: integer-translate is a better name.

[emilio]

cc @mstange / @jrmuizel / @mattwoodrow in case they have opinions

[progers]

@tabatkins, do you have thoughts on this?

[tabatkins]

This seems reasonable to me at first glance.

If the page actually does a non-integer-px translate, or does a scale/rotate/etc, we'll still switch to grayscale antialiasing, right? This is just will-change: transform; without forcing anti-aliasing to grayscale immediately.

[jrmuizel]

@progers Can you elaborate on how you see this being implemented? i.e under what conditions should the browser retain vs rerasterize the text?

What about opacity? Today if you have something like:

<div style="will-change: transform">Foo</div>

You potentially lose subpixel-aa because of transparency not because of a non-integer transform.

[emilio]

Also, a quick question, how is "integer-transform" defined if the device scale is fractional? An integer number of CSS pixels doesn't always correspond to an integer number of device pixels... Does that affect the proposal here?

[progers]

@tabatkins, exactly! This is just will-change: transform without forcing greyscale AA immediately.

@jrmuizel, blink is unable to use subpixel antialiasing in a lot of cases, such as for content that is not known to be opaque or is not known to have an integer to-screen transform. These implementation-specific restrictions are why I'd like to use a will-change hint rather than an approach like font-smoothing: subpixel-antialiased which blink cannot honor in the general case.

In terms of the actual implementation: all engines have an optimization to avoid work when scrolling the main document, and this hint allows for a similar optimization when changing transform. Blink implements this by rasterizing content into a texture on the GPU and then moving that texture instead of re-generating it when scroll offsets or transforms change. If we are unable to maintain subpixel antialiased text, such as if opacity is applied, the text would fall back to greyscale antialiasing permanently.

@emilio, that is a great question. If the engine is unable to re-use high quality text with changing integer transforms, the hint would be equivalent to will-change: transform.

[tabatkins]

Yeah, I think the value would be defined as something like:

This value hints that the page author will only be transforming this element with whole-pixel translates, and no other transforms. If the UA has higher-quality rendering paths (especially for text rendering) that depend on the element being positioned on whole-pixel boundaries, this value allows them to retain that rendering path when possible. (As opposed to will-change: transform, which requires the UA to be pessimistic and use rendering paths which apply to all possible transforms.)

[tabatkins]

Probably with a following note reminding that these are just rendering hints, and UAs can revert to more pessimistic renderings if you violate them, or if they just feel like it, anyway.

[jrmuizel]

@progers why does transform need to be used in the first place? Couldn't regular scrolling serve this use case?

[progers]

@jrmuizel, that's a great question. A Monaco editor dev gave several reasons in crbug.com/1016062#c34. The one I found most compelling was that they did not want the browser's scroll behavior (including scrollbars). More fundamentally, it doesn't seem ergonomic to require high-quality movable text be wrapped in a scroller. Another reason the Monaco developer gave was a performance problem in blink because scrollTop forces a layout (this is a bug).

[jrmuizel]

Monaco is implementing scrolling though, so I wonder if it would be better to try and improve the capabilities there (like adding a way to disable scrollbars) and only pursue this option if we can't make that work well.

[progers]

@jrmuizel, I do think it is important to continue improving and extending scrollers but scrolling is not sufficient to cover the usecase of will-change: transform that preserves high-quality text. The Monaco editor was the most recent example that came up, but this is a long-standing problem currently worked around with implementation-specific hacks like transform: translateZ(0). Slide-in transitions (e.g., settings "hamburger"/"hidey" menus) and "pick up and drag" interactions (e.g., re-ordering list items) are examples from the material design gallery that have high-quality moving text.

[progers]

cc @alexdima (Monaco developer) in case they have thoughts.

[alexdima]

Hi, I'm a developer on the VS Code team and author of the Monaco Editor.

@progers Thank you for looping me in and for driving this proposal! I think the proposal makes sense, feels familar if you are familar with will-change: transform, and covers our use-case.

@jrmuizel We have spent a lot of time building a code editor that uses DOM nodes to render the text. Not everything is great with our implementation (we have trouble with long lines), but it is possible to open files of 2GB or more, or of 1 million or more lines, and we can really compete with some other editors that are written in C++.

We can mainly do that by rendering only the lines in the viewport, otherwise browsers freeze for minutes or crash when inserting a large amount of HTML to the DOM. So we have implemented virtualized vertical scrolling, so only the visible (or partially visible) lines are actually in the DOM. Our main users are developers and many of our users use touchpad or precision trackpad devices which allow for smooth scrolling (i.e. as opposed to using a more traditional mouse wheel that scrolls a few lines at a time). Our users are also way more sensitive to the font rendering of their code relative to the general browser users.

We have tried to use scrollTop, but have run into the following limitations. I'm sorry if these are not 100% accurate anymore, as this was some years ago and I write them from memory:

• browsers would scroll to the position before we had a chance to render. So we would get smooth, native-like scrolling, but you would always have 1 frame where content was missing, it's as if the browser would first scroll, and then emit DOM events so we had a chance to render at the new viewport position. We would call this "white flashing" as the background was always white and we'd always see that white before the lines were rendered. This made it feel as a toy and not a real code editor.
• browsers would not accept very high numbers for positions / heights. At the time, IE would not allow heights over 1533908px. Maybe this is no longer a limitation, but I also was not curious enough to find which browser now has the smallest limit. In any case, we currently use modulo 500k to avoid very large numbers for the height and line positions.
• we actually don't really want the native scrollbars, because we cannot participate and render things on top them, and they end up wasting space. We render all kinds of editor decorations in the scrollbar, and we also have non-standard behaviors where clicking on a region in the scrollbar does not scroll up or down by one page, but rather directly jumps to that position. Here I have included a screenshot to better exemplify what I mean: from the scrollbar area alone, it is easy for me to spot that the file has at least one error, has one deletion and one insertion, and that there are 2 hit results somewhere below the current viewport:
• IIRC, setting or getting scollTop would cause synchronous layouts.

So we have decided to "fake" scrolling by simply moving things around. We would do that by changing the top position of the lines container. That would translate in one go all the lines up or down. Also, at each frame, we would remove the dom nodes which fall outside of the viewport and insert dom nodes which fall inside the viewport. We tweaked a lot of things until we managed to do this in as few dom interactions as possible, one trick we do is that we don't keep the lines ordered in the dom in the same order as you see them, but rather always the new lines are added at the end via a single insertAdjacentHTML call. In any case, old lines go out and new lines go in to the dom as you scroll.

A while later we have discovered the translate3d trick which would create a new layer for the lines and which reduced the paint time substantially when scrolling. Unfortunately we could not use the translate3d trick in all browsers, Firefox has an open issue here. But, over time, will-change: transform has been standardized and we switched from translate3d to will-change: transform. I will include the two gifs here to give a feeling of the paint area magnitudes:

Expand - large GIFs

Without a layer:  With a layer: 

We might be unique in our problem of wanting both high quality text rendering and high performance scrolling, but we are willing to adopt anything you come up with to make our users (fellow developers) happy.

Here you can try out the editor in a web browser -- https://microsoft.github.io/monaco-editor/playground.html -- and you can create a large amount of text using this snippet:

let value = "function hello() {\n\talert('Hello world!');\n}\n";
for (let i = 0; i < 22; i++) {
    value = value + value;
}

monaco.editor.create(document.getElementById("container"), {
    value: value,
    language: "javascript"
});

As we have our own API to maintain in VS Code, I can understand the push back against new APIs, and can definitely appreciate how difficult it is to add something to a standard which must then be supported possibly indefinitely.

But at the same time I find this proposal simple and clear.

P.S. Thank you for your work! :heart:

[progers]

@jrmuizel, do you think the reasons in https://github.com/w3c/csswg-drafts/issues/4560#issuecomment-566690654 and https://github.com/w3c/csswg-drafts/issues/4560#issuecomment-567529765 support an approach like will-change: integer-translate?

[yisibl]

I agree with the name integer-translate.

For a long time, the hack of transform: translateZ(0) has brought a lot of confusion to developers, and standardized solutions are helpful to change this situation. It also helps VS Code render with high-quality text.

[jrmuizel]

@jrmuizel, do you think the reasons in #4560 (comment) and #4560 (comment) support an approach like will-change: integer-translate?

I haven't had a chance to think through those comments yet but thought it was worth noting that you can disable scrollbars by using scrollbar-width: none;. Chrome doesn't seem to implement that yet so perhaps that's an obvious place for Chrome to start improving the situation for web authors trying to have more control.

[wangxianzhu]

What do you think about the relationship between will-change: integer-translate and global zoom or device scale factor?

I think integer should mean whole physical pixels instead of integer css translation values. With will-change: integer-translate, even if the developer specifies integer translation values, the physical translation may not be integer if the zoom/scale factor is not integral. To achieve physical-pixel-aligned rendering, the browser will have to round the physical translations. Should we define a property to specify whether the browser should snap to physical pixels for translations, which is not limited to will-change? Currently it seems that Firefox snaps to pixels for translations, while Chrome and Safari don't.