search

gfx-rs / wgpu

A cross-platform, safe, pure-Rust graphics API.
https://wgpu.rs
Apache License 2.0
12.46k stars 914 forks source link

Window alpha transparency support #687

Closed valpackett closed 1 year ago

valpackett commented 4 years ago

Is your feature request related to a problem? Please describe. Currently, windows are hardcoded to be opaque:

config.composite_alpha_mode = hal::window::CompositeAlphaMode::OPAQUE;

so the a in wgpu::RenderPassColorAttachmentDescriptor clear_color doesn't do anything.

Describe the solution you'd like There should be support for the transparent modes. In particular, premultiplied mode should be handled, as it's the only one supported at least on my machine (using Mesa RADV driver for Radeon).

Additional context Hardcoding ::PREMULTIPLIED makes transparency work, but I actually have to multiply the colors by the alpha:

Screen_2020-05-31-14:44:07-fs8

kvark commented 4 years ago

I would be fine with exposing the composite alpha mode

valpackett commented 4 years ago

As in the whole CompositeAlphaMode choice? ah good old "let the user handle it" :)

Swapchain creation is public API so this affects webgpu-headers…

kvark commented 4 years ago

Is there an alternative to telling users what modes are available, and letting them select one? I'd be happy to consider one!

As for webgpu-headers - you are right, we'll need to expose it there as well, just like we expose the present modes. cc @Kangz

valpackett commented 4 years ago

I was thinking "pick the most common mode and handle conversions" but yeah, exposing everything to the user would be much easier.

Kangz commented 4 years ago

Can we have a brief investigation of what the compositing modes are and how they translate to the various APIs? I worry for example that on macOS the compositing mode might be a part of the CALayer so we'd be mutating that.

Also are there any guarantees that we can support all compositing modes on all platforms, or does for example Vulkan give little guarantees so we need to have queries in webgpu.h for it?

kvark commented 4 years ago

Can we have a brief investigation of what the compositing modes are and how they translate to the various APIs?

Agreed, would be great to have it!

I worry for example that on macOS the compositing mode might be a part of the CALayer so we'd be mutating that.

Pretty sure we are already mutating CALayer properties when you are re-configuring the swapchain, since we set the size and the vsync, etc

Also are there any guarantees that we can support all compositing modes on all platforms

No

atsuzaki commented 4 years ago

Any updates on this?

grovesNL commented 4 years ago

@atsuzaki I don't think anyone has been looking into this yet (unless @myfreeweb has a branch somewhere?), but please feel free to start looking into it if you'd like.

An investigation into how to handle this on each backend and/or prototyping the changes with one or more backends would be great.

valpackett commented 4 years ago

I just have a hardcoded premultiplied mode, nothing smarter, no API or anything

nyxtom commented 3 years ago

I've made a similar change with the PREMULTIPLIED mode but it doesn't seem to work in macOS. Not entirely sure what's missing. Presumably everything needed would be in place (such as color states and having a clear color with an alpha (plus the change in `swap_chain.rs`)

    let window = WindowBuilder::new()
        .with_title("Rust by Example: WGPU!")
        .with_decorations(false)
        .with_transparent(true)
        .build(&event_loop)
        .unwrap();
            color_states: &[
                wgpu::ColorStateDescriptor {
                    format: sc_desc.format,
                    color_blend: wgpu::BlendDescriptor {
                        src_factor: wgpu::BlendFactor::SrcAlpha,
                        dst_factor: wgpu::BlendFactor::OneMinusSrcAlpha,
                        operation: wgpu::BlendOperation::Add
                    },
                    alpha_blend: wgpu::BlendDescriptor {
                        src_factor: wgpu::BlendFactor::One,
                        dst_factor: wgpu::BlendFactor::OneMinusSrcAlpha,
                        operation: wgpu::BlendOperation::Add
                    },
                    //color_blend: wgpu::BlendDescriptor::REPLACE,
                    //alpha_blend: wgpu::BlendDescriptor::REPLACE,
                    write_mask: wgpu::ColorWrite::ALL
                }
            ]    
            let mut render_pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
                color_attachments: &[
                    wgpu::RenderPassColorAttachmentDescriptor {
                        attachment: &frame.view,
                        resolve_target: None,
                        ops: wgpu::Operations {
                            load: wgpu::LoadOp::Clear(wgpu::Color {
                                r: 0.005,
                                g: 0.005,
                                b: 0.01,
                                a: 0.5
                            }),
                            store: true
                        }
                    }
                ],
                depth_stencil_attachment: None
            });
    config.composite_alpha_mode = hal::window::CompositeAlphaMode::PREMULTIPLIED;
kvark commented 3 years ago

gfx-rs Metal backend needs to handle this better, I guess.

kvark commented 3 years ago

@nyxtom Metal transparency support is coming in https://github.com/gfx-rs/gfx/pull/3561

ryanw commented 3 years ago

Transparent windows don't work for me either. Using Linux and Xorg. I ran the Vulkan validation layers and see the error below. My GPU is an RTX 2080 Ti, and transparent windows work with OpenGL things. Seems a little weird that the only supported mode on my GPU is VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR.

vulkaninfo says it only supports opaque, so perhaps it's a driver issue of some kind. Maybe it'd work with XCB instead of Xlib?

supportedCompositeAlpha: count = 1
    COMPOSITE_ALPHA_OPAQUE_BIT_KHR

VUID-VkSwapchainCreateInfoKHR-compositeAlpha-01280(ERROR / SPEC): msgNum: -1341842926 - Validation Error: [ VUID-VkSwapchainCreateInfoKHR-compositeAlpha-01280 ] Object 0: handle = 0x55f648593ab8, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0xb0051a12 | vkCreateSwapchainKHR() called with a non-supported pCreateInfo->compositeAlpha (i.e. VK_COMPOSITE_ALPHA_PRE_MULTIPLIED_BIT_KHR). Supported values are: VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR . The Vulkan spec states: compositeAlpha must be one of the bits present in the supportedCompositeAlpha member of the VkSurfaceCapabilitiesKHR structure returned by vkGetPhysicalDeviceSurfaceCapabilitiesKHR for the surface (https://vulkan.lunarg.com/doc/view/1.2.162.0/linux/1.2-extensions/vkspec.html#VUID-VkSwapchainCreateInfoKHR-compositeAlpha-01280) Objects: 1 [0] 0x55f648593ab8, type: 3, name: NULL

LoganDark commented 3 years ago

Still interested in a solution to this

coderedart commented 3 years ago

just wanted to express interest too.

just like ryanw, transparency doesn't work for me either on linux/X11 with 1070Ti. Glutin's transparency example works flawlessly too. if i run vulkaninfo i'm getting supportedCompositeAlpha: count = 1 COMPOSITE_ALPHA_OPAQUE_BIT_KHR .

coderedart commented 3 years ago

just checking, but is this repo still the main one? i heard the repos were moved around. i will go make a new issue for transparency, if there's a new repo somewhere.

kvark commented 3 years ago

Oh yes, you can see the activity here, while https://github.com/gfx-rs/wgpu-rs is archived.

coderedart commented 3 years ago

can confirm. https://github.com/rust-windowing/winit/pull/2006 fixes the transparency issue for me

valpackett commented 3 years ago

Now that the HAL is right here instead of being an external dependency, it might be easier to expose CompositeAlphaMode…?

coderedart commented 2 years ago

just bumping here as its been a few months. but transparency in windows 10 is working. tested on 

Edition Windows 10 Pro
Version 21H1
OS build    19043.1348
GPU: 1070 ti

and wgpu master branch by just changing clear color to [0.0, 0.0, 0.0, 0.0] in the Load operation, and creating winit window with transparency enabled and making sure to have blend state to ALPHA_BLENDING.

image

bunny mark example

coderedart commented 2 years ago

I might have realized this too late. but is there actually a point in exposing CompositeAlphaMode ? because transparency works right now. and the mode is still hardcoded as Opaque in wgpu (using master branch). atleast on windows or linux, It seems as long as i write the Alpha channel to framebuffer, and create the window with transparency enabled (glfw / winit tested), it works normally.

valpackett commented 2 years ago

@coderedart is this Vulkan or D3D12? Have you only tested on nvidia GPUs?

If this is Vulkan, something in your stack definitely violates the spec:

VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR: The alpha component, if it exists, of the images is ignored in the compositing process. Instead, the image is treated as if it has a constant alpha of 1.0.

this clearly says that in opaque mode, there MUST NOT be any transparency.

It works correctly on Mesa — [0.0, 0.0, 0.0, 0.0] is treated as [0.0, 0.0, 0.0, 1.0] in opaque mode so I get a black background.

coderedart commented 2 years ago

I am making an egui app and used transparency on both windows and linux (1070ti) without any issues. I just cloned and ran the hello-triangle example from wgpu repo by just changing the clear color and enabling the transparency option for window, and it works perfectly.

just wanted to express interest too.

just like ryanw, transparency doesn't work for me either on linux/X11 with 1070Ti. Glutin's transparency example works flawlessly too. if i run vulkaninfo i'm getting supportedCompositeAlpha: count = 1 COMPOSITE_ALPHA_OPAQUE_BIT_KHR .

as mentioned previously, i only have opaque bit mentioned in driver, so i thought vulkan drivers don't really care about niche things like this. and wgpu source definitely sets the opaque bit.

I will check on windows with dx backend and report back. EDIT: windows 11 + 1070ti latest drivers vulkan configurator shows vulkan info with only opaque bit. wgpu hello-triangle works with transparency on vulkan backend. but dx12 just has black instead of transparency. dx11/gl backends just crash without starting.

SECOND EDIT: I had a laptop nearby with Amd A12-9720p cpu and arch installed on it. hello-triangle vulkan transparency works. RADV Carrizo is the driver name. vulkan api 1.2.195 version. the special thing to note is that it had both Opaque and Inherit bits available in the vulkaninfo.

in all these cases, i just changed the window builder to use transparency and renderpass clear color to transparent contant. didn't touch any other code, and just ran cargo run --example hello-triangle. on windows though, the whole screen was going blank for a second, so i had to separate out hello-triangle example into a normal cargo new project and it worked fine then. also, i expected dx would be the backend on windows 11, but it seems wgpu chooses vulkan

valpackett commented 2 years ago

dx12 just has black instead of transparency

Yep, so D3D12 seems to be working correctly. (@coderedart could you please confirm that you get transparency on DX12 if you change the composite_alpha_mode?)

Looking at open source WSI implementations:

Conclusion: the opaque bit actually works correctly on Wayland and macOS (MoltenVK), and the similar bit seems to work on D3D12, but many other Vulkan implementations ignore it and INCORRECTLY let transparency work with VK_COMPOSITE_ALPHA_OPAQUE_BIT_KHR. I have raised a Mesa issue about X11.

coderedart commented 2 years ago

@unrelentingtech

none of the pre/post composite modes work on windows dx12 backend. and all of them work with transparency on vulkan backend (but i do get validation errors saying only opaque bit is supported, and i'm using unsupported bit with pre or post multiplied).

the dx12 backend just crashes

[2022-02-09T01:26:28Z ERROR wgpu_hal::dx12] SwapChain creation error: 0x887A0001
[2022-02-09T01:26:28Z ERROR wgpu_core::device] surface configuration failed: swap chain creation
thread 'main' panicked at 'Error in Surface::configure: invalid surface', C:\Users\red\source\repos\wgpu\wgpu\src\backend\direct.rs:231:9
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c\/library\std\src\panicking.rs:517
   1: std::panicking::begin_panic_fmt
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c\/library\std\src\panicking.rs:460
   2: wgpu::backend::direct::Context::handle_error_fatal<enum$<wgpu_core::present::ConfigureSurfaceError> >
             at C:\Users\coder\source\repos\wgpu\wgpu\src\backend\direct.rs:231
   3: wgpu::backend::direct::impl$3::surface_configure
             at C:\Users\coder\source\repos\wgpu\wgpu\src\backend\direct.rs:938
   4: wgpu::Surface::configure
             at C:\Users\coder\source\repos\wgpu\wgpu\src\lib.rs:3181
   5: hello_triangle::run::generator$0
             at .\main.rs:78
   6: core::future::from_generator::impl$1::poll<hello_triangle::run::generator$0>
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c\library\core\src\future\mod.rs:80
   7: pollster::block_on<core::future::from_generator::GenFuture<hello_triangle::run::generator$0> >
             at C:\Users\coder\.cargo\registry\src\github.com-1ecc6299db9ec823\pollster-0.2.5\src\lib.rs:125
   8: hello_triangle::main
             at .\main.rs:145
   9: core::ops::function::FnOnce::call_once<void (*)(),tuple$<> >
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c\library\core\src\ops\function.rs:227
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
error: process didn't exit successfully: `target\debug\hello-triangle.exe` (exit code: 101)

OS:

Edition Windows 11 Pro
Version 21H2
Installed on    ‎27-‎01-‎2022
OS build    22000.493
Experience  Windows Feature Experience Pack 1000.22000.493.0

nvidia driver version: 511.23

I have no idea if there's a dx12 equivalent of vulkaninfo which i can use to check supported CompositeModes.

I hope mesa don't fix the bug xD this issue has been here for a long time, and if they fix it (especially as nvidia doesn't expose any other composite modes), there's no way to get transparency for vulkan now. I gotta go back to opengl :sob:

valpackett commented 2 years ago

there's no way to get transparency for vulkan now

No, the temporary way to get transparency for now is to just patch wgpu like I've been doing, and the real way is to make wgpu expose composite_alpha_mode to the user.

kvark commented 2 years ago

Thank you so much for investigating this and filing an upstream issue, @unrelentingtech ! Looking forward to hear from Mesa folks on it.

valpackett commented 2 years ago

@kvark in any case the "driver bug" is only that some drivers allow transparency when they shouldn't, it's not that big of a deal, just something to understand when people are saying "transparency already works".

The original issue here is still that wgpu should give us access to this setting.

kvark commented 2 years ago

Yes, I think we are aligned here. Just need a PR.

ElhamAryanpur commented 2 years ago

hey folks! How's the progress on this topic?

coderedart commented 1 year ago

I think this can be closed. we now have CompositeAlphaMode for wgpu Surface.

teoxoy commented 1 year ago

Fixed by https://github.com/gfx-rs/wgpu/pull/2836

dhardy commented 1 year ago

Two things I don't understand about the solution #2836:

  1. Portable applications which want transparency are supposed to check which of PreMultiplied, PostMultiplied and Inherit are supported, pick their preference, then output either pre-multiplied or un-multiplied textures accordingly?
  2. Portable applications need platform-specific code to know what Inherit means?

The status quo seems like a half-solution:

teoxoy commented 1 year ago

@dhardy could you open a new issue with your thoughts to improve the status quo? I'm not too familiar with this area at the moment. Thank you!