Sergio0694 commented 3 years ago

Proposal: bring back global composition (UWP regression)

Follow up from https://github.com/microsoft/microsoft-ui-xaml/issues/4947

Summary

As described in the related issue, currently CompositionBackdropBrush cannot sample from over a SwapChainPanel. This makes it impossible to apply effects such as GaussianBlurEffect over content being rendered in a SwapChainPanel. This proposal is about allowing content in a SwapChainPanel to be sampled just like other normal WinUI content in a window, so that it can be used as input in any composition pipeline that starts with a CompositionBackdropBrush.

In general, the ask is to bring back global composition and get feature parity with UWP.

In other words, DirectX swapchain content should be visible to the compositor (eg. backdrop brushes).

Rationale

The SwapChainPanel isn't always used as a full-window renderer, but it's often used to compose UI components in an application (eg. a map control, some custom renderer inside a template, etc.). With the current limitation in WinUI 3, this makes it impossible to then apply effects on top of the entire UI (eg. a blur effect if a popup is opened), because those SwapChainPanel-s would then visibly be "glitched" next to the normal WinUI 3 content behind the effect.
Make the behavior consistent with UWP (this could be seen as a regression in WinUI 3)
Make the behavior more intuitive for developers ("why are these effects not working here?")

Scope

Capability	Priority
Allow composition backdrops to sample over DirectX swapchains	Must

In other words: an in-app acrylic brush should work as expected when placed above a SwapChainPanel control.

StephenLPeters commented 3 years ago

@codendone FYI

kevinguo305 commented 2 years ago

Thanks for submitting this proposal Sergio! It's in our backlog and we'll consider this for a future release.

Sergio0694 commented 2 years ago

This is great to hear! Thank you for the follow up! 😄

Sergio0694 commented 2 years ago

@kevinguo305 @codendone @marb2000 I thought I'd share an actual example of where this feature would be cool to have, especially given it's a regression from UWP (and we're telling people to move from UWP to WinUI 3). Here's a sample app for my library ComputeSharp, where I display a bunch of shaders rendered to a ComputeShaderPanel control I wrote to easily render shaders in XAML. I'm applying an in-app acrylic brush to the window content when the drawer is opened, which on UWP results in this super cool glass effect over the current shader, bringing focus for the user to the shaders list at the bottom. This is just an example, but the same would apply of course to all cases where people are just rendering whatever content they might need into their apps (eg. a custom map, a 3D object viewer, an animated background, you name it), then wanted to open an overlay and blur the app content to bring attention to the dialog that just opened, and those swap chains in the background would just "skip" the blur and kinda "peek through", which would be quite jarring to see and arguably just look glitchy. It would really be awesome if we could reach feature parity with this on WinUI 3 in the future 😊

https://user-images.githubusercontent.com/10199417/140443278-14ca5b59-ec51-45e8-8a45-e74c7d510beb.mp4

Sergio0694 commented 1 year ago

Bumping this (see https://github.com/microsoft/microsoft-ui-xaml/discussions/8638).

Sergio0694 commented 9 months ago

I've updated the title and description to be clearer and more generally applicable. Commenting here and not internally since this is a feature that everyone should be able to use, so I feel like info on this should just be shared in the open. @duncanmacmichael @bpulliam would you be able to help getting this triaged and possibly added to some roadmap, just to make sure it doesn't get lost? I think this is one of the most notable regressions of WinUI 3 compared to UWP (it's just one of the things that stopped working entirely due to the switch to the lifted compositor), and it would really be awesome to get this fixed. Bringing global composition back would allow apps to create very advanced visuals with mixed XAML and DirectX content 🙂

As an example, consider the amazing Lively Weather app:

https://github.com/microsoft/microsoft-ui-xaml/assets/10199417/338ab209-85c7-4861-a62c-e25ebdf7a965

The UI has a swapchain in the background, with XAML content overlayed above it. You can see how all widgets also have an in-app acrylic brush, which blurs the swapchain content behind them. This is currently completely impossible to do on WinUI 3.

duncanmacmichael commented 9 months ago

Thanks Sergio, really appreciate it! Great details and example. I'll bring this up in our next feature request triage and we'll go from there!

microsoft / microsoft-ui-xaml