Issues and possible improvements to transparency rendering

[mvaligursky]

The default framebuffer WebGL uses for rendering is RGB(A)8 buffer. The values written to it are expected to be in the display color space, which typically means sRGB. As this framebuffer is created by the canvas, we have no control over it, and it is not created with sRGB (EXT_sRGB) flag. Meaning the shader writing to it needs to manually encode values from linear to sRGB space.

LDR rendering

• when rendering directly to the default framebuffer, the engine correctly encodes values to sRGB space at the end of shaders. This causes alpha blending to not be correct, as the blending takes place in sRGB space, instead of correctly in linear space, with the result getting encoded to sRGB. This is how PlayCanvas currently operates if rendering directly to the default framebuffer.
• proposal to support linear workflow: https://www.w3.org/Graphics/Color/Workshop/slides/Russell.pdf (it does not seem to be progressing, see https://groups.google.com/a/chromium.org/g/blink-dev/c/tA7Av4fLQFY/m/oUw_k949AAAJ).
• a workaround is rendering to a created framebuffer using sRGB, which would alpha blend correctly, and blit the framebuffer to the default framebuffer at the end of the frame. This is expensive, and so not typically used.
• we also implement dithered transparency. As this does not depend on alpha blending, it's not affected by blending in gamma space and works correctly. The only problem is that when enabled, this looks visually different to alpha blending, as that is incorrect.

HDR rendering

• When we render in HDR mode, we use float format of the internal render target in linear space. This makes alpha blending to work in linear space, which is correct.
• We have problem with transparency dithering here if tone mapping is used - it does not match the correct alpha blending. The problem is that with dithering, the discarded and visible pixels are tone mapped, but as tone-mapping is not linear, it does not match the alpha blending, as non-blended values are tonemapped.
• Transparency dithering works correctly if tone mapping is used, as long as TAA is used as well - as TAA resolves transparency by blending with history buffer, and this blending takes place before the tone mapping.

References: https://github.com/KhronosGroup/WebGL/issues/2474 https://github.com/mrdoob/three.js/issues/23019 https://stackoverflow.com/questions/51032480/how-do-you-implement-a-gamma-correct-workflow-in-webgl2 https://groups.google.com/g/angleproject/c/5ITMg4_m8Ug https://github.com/playcanvas/engine/issues/5287

[MAG-AdrianMeredith]

its not necessarily srgb though right? https://ccameron-chromium.github.io/webgl-examples/p3.html. but I guess this would actually help here as we're letting the hardware do it

[mvaligursky]

True, but until we implement https://github.com/playcanvas/engine/issues/5287, it's sRGB. Handling P3 is part of that issue.

[mvaligursky]

Related Chromium issue I created exploring the sRGB in WebXR: https://partnerissuetracker.corp.google.com/issues/352345526?pli=1

[mvaligursky]

conclusion regarding WebXR, from the issue above:

The mechanism for specifying a texture format for WebXR is the Layers API, which is not yet supported by Chrome but does have support in the Meta Quest browser. WebXR is not compatible with buffers allocated by drawingBufferStorage.

[mvaligursky]

https://github.com/playcanvas/engine/pull/6838 implements support for sRGB framebuffer on WebGPU, but also support for sRGB render target on all platforms (just not the default framebuffer on WebGL), allowing linear rendering.

[mvaligursky]

Update on this.

• the linear blending can be used on both WebGL and WebGPU if the color buffer of a render target uses some of the sRGB formats, for example SRGBA8
• sRGBA8 format can be used as a canvas framebuffer format on WebGPU as well, using displayFormat member of createGraphicsDevice options
• on WebGL the support is not implemented currently and the options are more limited:
  • drawingBufferStorage can be used to set it, but this is only supported on Chrome 122+, other browsers do not implement it yet (https://github.com/playcanvas/engine/pull/6840)
  • Layers API can be used to set this for WebXR, but Chrome does not implement this (see more above)

[mvaligursky]

relevant topic: https://www.fractolog.com/2024/07/color-space-correctness-in-alpha-blending/