[css-transform] Provide way to scale internal resolution of layers

[Awendel]

Link to proposal repository: https://github.com/Awendel/css-pixel-density

BACKGROUND Every year devices become higher and higher resolution (especially smartphones, most have very high dpi displays). At the same time the scope and scale of what web apps can render (think Figma, data visualisation, ...) becomes larger and larger. Moreover, the amount web users increases every year with largest growth coming from developing countries where sadly the majority have access to only low end hardware.

THE PROPOSAL Offer a way for a given layer to adjust its internal resolution by a multiplying factor. The default of course should be "1" or "100%" which is the resolution that is currently chosen. If I set the value to 0.5 or 50% it should half the internal resolution of the layer (and hence put less pressure on the rasterizer hardware and gpu memory etc). The CSS property could be named "resolution", "raster-scale" or "pixel-density" and units should be normalized floats or percentage.

USE CASES

• make graphics intensive animation run smooth (60fps) even on low end devices, by temporarily lowering the resolution for the duration of the animation
• offer way for low end devices to use graphics intensive application with useable framerates ( similar to how common it is to lower the resolution in gaming, especially on 4K displays etc)

COMPATIBILITY & EASE OF INTEGRATION The proposed property would integrate very well with the following existing properties:

• transform, esp. scale
• will-change, especially when set to "transform". Will-change "locks" the internal resolution of a layer already, but it doesn't provide a way to lower it, especialling when "zooming" out. It is currently not enough to ensure every animation runs smooth. I will attach some test cases to prove this and show the only way to run it smooth would be to lower the resolution.

Moreover, it would be very simple to integrate into the existing render logic of browsers. Since there already is a mechanism to calculate the layer resolution (multiplying screen resolution by devicePixelRation and transform:scale), it would be just another multiplier in that computation.

CURRENT (LACK OF) ALTERNATIVES The only way to achieve this currently is by pushing all render logic to canvas, where one can control the resolution. Sadly this has led to a situation where all graphics intensive application use a custom WebGL render stack (Figma, Google Maps etc), even though all rendering could be done with plain HTML. Plain HTML is also by nature more accessible and eases the barrier to entry for new developers.

[Awendel]

TEST CASE FILE (use 4k or other high resolution display ideally) Here is a test case that shows the problem and why existing CSS properties cannot handle this case:

https://jsfiddle.net/9wsry1g0/

This renders some procedural graphics intensive content and starts a "zooming out" animation. It uses transform scale and will-change:transform in order to rasterise the content and "make it smooth"

Chrome and Firefox both handle it in slightly different, but both insufficient ways:

Chrome respects the will-change:transform property which leads to the animation running smooth. Yet, it locks the resolution at the start resolution which leads to a GPU memory explosion and henceforth it creates visual artifacts and doesn't render a lot of the content when zoomed out (since it runs out of GPU memory very quick).

Firefox seems to ignore the will-change:transform property. This leads to it not having a GPU overflow, it shows all content but because it re-rasterises on every frame, it has a very low framerate (15fps or worse).

This "zooming" out effect would be a very common scenario in something like Figma / Design tool, Maps or Whiteboarding tool. The only way to achieve running this smooth in a 4k display, would be to lower the resolution during the "zooming" out. Ideally also in conjunction with locking the raster resolution (using will-change: transform)

[Awendel]

USE CASE FOR INCREASING LAYER RESOLUTION I have previously only outlined the use case for lowering the resolution of a layer to increase performance. Yet, there is also a compelling use case for being able to increase the resolution:

Say I want to do a "zoom in" animation on content and I want it to be performant. Hence I lock the rasterisation resolution via will-change:transform. The issue is that as I zoom in the content becomes blurry (since the animation started at a lower scale etc).

With the new property I could instead double the resolution + then zoom in still with rasterisation enabled, but the content would be a lot less blurred than before.

This woul also make the following issue mostly redundant: https://github.com/w3c/csswg-drafts/issues/236

Since now the developer can plan ahead the target resolution before an animation

[Awendel]

REASONS WHY ADDING THE PROPERTY WOULD BE NET POSITIVE FOR WEB

• inline with new Houdini approach of exposing low level primitives to web developers which can be combined to greatly customise render experience
• simplicity + intuitiveness of API
• simplicity of integration into existing browser rendering pipelines (devicePixelRatio etc)
• addresses 2 main concerns for web developers who do animations:
• (1) animations not running smooth (fix via lowering / downSampling resolution)
• (2) animation not being sharp (fix via increasing / upSampling resolution)
• making web more accessible by allowing lower end devices to render graphics intensive content
• making web more accessible because now plain HTML can be used without having to implement this in custom WebGL / Canvas rendering stack
• has potential to reduce loading time of HTML pages when a lot of content is rendered, by lowering the resolution before content is rendered, hence taking pressure of UI thread, it will appear faster on the screen, then one could increase resolution async in order to sharpen it again but not blocking anything

[fantasai]

The question I have here is, does this need to be an author-controlled property, or would it make more sense for the implementations to just be smarter about these things?

[Awendel]

I definitely believe this needs to be an author controlled property, since only the developer will know when animation / graphics content is about to be rendered.

All implementations currently attempt to match the layer resolution as close as possible to the screen resolution. Lowering the resolution in an automatic fashion would be too much of an unexpected behaviour for developers.

But if we exposed this as a low level primitive, then library developers could create libraries on top of this that maybe do apply automatic heuristics (e.g. by measuring fps, device resolution etc.).

This chain of though is also in line with the Houdini approach of exposing more low level primitives, on top of which then library developers can build higher level features.

[Loirooriol]

In the related https://github.com/w3c/csswg-drafts/issues/236#issuecomment-272585914, @smfr said

I agree this would be a useful property. WebKit tried to do it automatically, and reverted the change because it had bad perf impacts. Putting this under author control would be good.

[Awendel]

I created a public repository to summarise the spec for the proposed feature: https://github.com/Awendel/css-pixel-density

Over time I will add more examples, edge cases + implementation details there.

Feel free to open Issues there and write Pull requests, should you feel like adding details / comments

For now pixel-density seems to be the most descriptive term, that will also be intuitively understood by new developers, compared to more technical terms such as "rasterization-scale"