HDR and alpha-blending in the composition layer: some thinking needed

[benstevens48]

I've recently been experimenting with HDR in my app. The HDR content is rendered to a 16-bit float swap-chain, and Windows HDR mode is on.

However, this presents a previously unthought of problem. I have some panels that can overlay the swap chain, which are slightly transparent - the opacity is about 0.9. With SDR content, this means that the content of the panel is always legible. However, the HDR content in the swap chain can have pixel color values much larger than 1, especially if not clamped to match the max display luminance. When Windows HDR mode is on, the alpha blending formula applied when composing the swap chain and the panel overlay is completely dominated by the swap chain content if the pixel color values are much larger than 1, meaning that it is no longer possible to read the content of my panel!

You might say that this is a design flaw in my app. But I think this warrants further consideration. In the future, HDR will probably become more common. You could potentially even have translucent windows or windows using background acrylic or something. If the user puts one such window over the top of an HDR window, they would still expect to be able to see the content of that window and for it not to be eclipsed by the HDR one. (Note I haven't actually tested what happens in this case). Therefore, I think it might be worth considering whether to implement any solutions to this. For example, you could have an app setting that tells it to clamp before alpha blending, so any alpha blending done by the app composition layer would clamp colors to the [0,1] range before blending, but ideally only in the regions which need blending (so it may need to be done on a per-pixel basis or could be based on rectangular regions if really necessary). Alternatively, it could be a property of the swap chain to say, when composing this, clamp any pixels which need to be alpha-blended (and only these pixels) to the range [0,1]. Or there could be other modifications to the alpha blending formula in order to give more weight the 'over' input (I haven't thought it through very much).

Alternatively, I guess you could make this the responsibility of the developer. But then it would be nice if there was an easy way to clamp the pixel colors of a region of the swap chain, or perhaps this could be done be adding an effect to the composition layer or something... I'm not sure if there's an easy way to do this currently?

[StephenLPeters]

@jeffstall and @codendone I'm not sure which team owns this area, thoughts?

[benstevens48]

I've come up with a partial workaround to this problem which works reasonably well. The idea is basically to add a composition backdrop brush via an XamlCompositionBrushBase brush to the outermost element of the translucent panels which overlay the HDR content. This allows clamping of the pixels. Code for the brush can be found here - https://gist.github.com/benstevens48/a32c1871e22d722142d3c27622e64507.

Interestingly, it appears that the composition backdrop brush clamps pixel values without a further clamping effect (e.g. color matrix effect plus clamp output = true) required - I don't know if this is deliberate or if it just happens to work like that.

It would be nice to have a built in boolean property which effectively does the same thing as this workaround. This would be better because the workaround has a number of caveats such as having to restructure the visual tree in some cases, adding a background brush changes hit testing when you might not want to, and a built-in solution could presumably be more efficient.

[benstevens48]

Not sure I'd class this as a bug, but I guess this still requires some consideration. I'm not sure how this manifests itself in WinUI3 because I know the composition is done differently for swap-chains.