How to show RGBA frame with alpha channel in Windows?

[huangjinshe]

I have a video which is webm format, I did successfully convert it to RGBA format by ffmpeg (AV_PIX_FMT_RGBA). I can confirm it's all normal now, because I did saved the first frame of this video as a png file and I can see the alpha channel as transparent.

1. I created a window by Windows API:

_winHand = CreateWindowEx(WS_EX_LAYERED, "static", string.Empty, WS_POPUP | WS_VISIBLE, 0, 0, _width, _height, hwnd, IntPtr.Zero, IntPtr.Zero, IntPtr.Zero);

2. Then I create SDL_Window and other stuff (using SDL2):

   this->win = SDL_CreateWindowFrom((void*)window); // this is the window which I create use Windows API
   this->renderer = SDL_CreateRenderer(this->win, -1, SDL_RENDERER_ACCELERATED);
   SDL_CreateTexture(this->renderer, SDL_PIXELFORMAT_RGBA32, SDL_TEXTUREACCESS_STREAMING, this->width, this->height);

   I did use RGBA32, because RGBA8888 not showing the correct color, I don't know why. in SDL_pixels.h I saw there is a SDL_BYTEORDER == SDL_BIG_ENDIAN check, but not really know the reson behind.

3. Then I render the video frames:

while(loop)
{
  SDL_UpdateTexture(texture, NULL, frame->data[0], frame->linesize[0]);
  SDL_RenderClear(this->renderer);
  SDL_RenderCopy(this->renderer, this->texture, nullptr, nullptr);
  SDL_RenderPresent(this->renderer);
}

Now I did not see the alpha channel as a transparent. there is always show as black (alpha turning the black). no matter how, It's same.

I check a lot, and I found if I add this after create texture the background will turn to white color from black:

this->texture = SDL_CreateTexture(.....skip.....

SDL_SetTextureBlendMode(texture, SDL_BLENDMODE_BLEND);
SDL_SetRenderDrawColor(renderer, 255, 255, 255, 0);

But this is not helping.....I don't want white, I don't want black, I just want transparent. The tutorials are really less for this, really suffering.... that's why I'm asking here with these long content. apologize for that.

The last thing is some people advice use : SetLayeredWindowAttributes from windows api, but It not help, nothing change, maybe I'm setting wrong ? I don't know , I really need a official answer for this. It's really better we don't rely on that kind of API, really hope SDL already wrapping it , we just call API from SDL itself.

[slouken]

Have you tried SDL_WINDOW_TRANSPARENT in SDL3?

[huangjinshe]

Did'nt use the SDL3 yet, do we have pre-release or link for download? Is this a known issue in SDL2?

[slouken]

You can get the latest SDL3 code from GitHub: https://github.com/libsdl-org/SDL

It's not an issue, it's just not something that was ever offered in SDL2.

[huangjinshe]

OK, but it's really hard to believe SDL2 not support that, I thought using a different shape window(by png with alpha) is a common thing before.

[slouken]

No, there is a shaped window API in SDL2 that takes a separate bitmask instead of using the alpha value of the back buffer.

[sam20908]

@huangjinshe Hello! I was the one who implemented the transparent window for Windows in SDL3. If it does not work for you, can you comment on your GPU configuration as well as the SDL renderer used?

[huangjinshe]

@huangjinshe Hello! I was the one who implemented the transparent window for Windows in SDL3. If it does not work for you, can you comment on your GPU configuration as well as the SDL renderer used?

I did not try it yet, I'll let you know the result after use SDL3, thank you anyway. Could you guys just give us the link of binary version of sdl3.dll ? I think it would let us more easier to use. Beta version binary maybe, for every month before release?

[madebr]

Our ci uploads binary artifacts for (almost) all supported platforms, so check out the actions tab. e.g. https://github.com/libsdl-org/SDL/actions/runs/6727292544 and https://github.com/libsdl-org/SDL/actions/runs/6727292562

[huangjinshe]

@madebr ,Excellent, I didn't notice that.

@sam20908 , So could you tell me what I'm missing based on my code?
I did add Hint before create window:

    SDL_SetHint("SDL_WINDOW_TRANSPARENT", "1");
    this->Window = SDL_CreateWindowFrom((void*)window);

But It's not working, Is there any example for this?

[sam20908]

It's a flag you pass to SDL_CreateWindow

[huangjinshe]

It's a flag you pass to SDL_CreateWindow

I tried it too:

this->win = SDL_CreateWindow("", 200, 200, SDL_WINDOW_BORDERLESS | SDL_WINDOW_OPENGL | SDL_WINDOW_TRANSPARENT);

When I create it, it's a transparent window, but when I render the rgba frame, the background turn to black, as I said before(I pate the code before). It should turn the black alpha channel turn transparent.

And It would be really better if we could change the flag after call: SDL_CreateWindowFrom, I still want to control the window by myself.

[sam20908]

What is your GPU configuration and Windows version?

[sam20908]

Also, I note that there are lots of ways to create transparent windows, but I choose the DWM route. This means you may have to try different renderers depending on your configuration (and in my case, having an NVIDIA Optimus laptop means I need different renderers depending on which GPU is active).

The only guaranteed way to get a transparent window AFAIK is to create a layered window. But that has its own cons with being horrible in performance and others. DWM being the most balanced with pros and cons is why it was chosen here.

And It would be really better if we could change the flag after call: SDL_CreateWindowFrom, I still want to control the window by myself.

I did think of this but at that time I saw that the flag is used only when creating a window in other platforms, so I followed the trend to not differ in behaviour.

[huangjinshe]

My laptop(I did tried open it with 880M card by right click the exe to run, same result):

CPU: Intel(R) Core(TM) i7-4810MQ CPU @ 2.80GHz GPU:Nvidia 880M Windows10: 19045.3570

PC:

12th Gen Intel(R) Core(TM) i5-12400 2.50 GHz Window11(22H2)

I feel it's not a GPU problem, because I could see the transparent window first, but when I try to show avframe which format: AV_PIX_FMT_RGBA(use ffmpeg), and use SDL_PIXELFORMAT_RGBA32 texture update the image, the alpha will shown as black. but if save the avframe as png use ffmpeg, I could see the alpha turning transparent.

Flag I think we could change the window style by SetWindowLong api in windows. I don't sure could we get SDL_WINDOW to change by own.

[sam20908]

Interesting. For NVIDIA I know what worked for me is having OpenGL as the renderer, see if you can try that.

As for whether or not it's a GPU problem, the results for me were that it was inconsistent between the GPUs I have, so that was why I arrived at that conclusion. Maybe it could be a texture problem, but that also depends on the renderer.

[huangjinshe]

Did tried, same result.

For everybody understand, let me show some image: This is what the window look like after create (As you see there is nothing only the windows screen background image):

This is what the window look like after play now: (As you see the background shown as black)

This is what it look like (when I add these code):

SDL_SetTextureBlendMode(texture, SDL_BLENDMODE_BLEND);
SDL_SetRenderDrawColor(renderer, 255, 255, 255, 0);

This is the image when I save it as png by ffmpeg when I get the first frame (same frame format: AV_PIX_FMT_RGBA)

This is what I expected. obviously it could understand the alpha channel but only show it as black or white.

[sam20908]

You have to clear with 0, 0, 0, 0, that's what I used. Not sure why exactly though.

[huangjinshe]

SDL_SetRenderDrawColor(renderer, 0, 0, 0, 0); still black after I create texture and render.

I will always clear it in the loop for show video:

while(loop)
{
  SDL_UpdateTexture(texture, NULL, this->frame->data[0], this->frame->linesize[0]);
  SDL_RenderClear(this->renderer);
  SDL_RenderTexture(this->renderer, this->texture, nullptr, nullptr);
  SDL_RenderPresent(this->renderer);
}

[sam20908]

Yeah sadly I don't know what's wrong anymore :(

[slouken]

Can you post a link to a complete test, so we can take a look?

[huangjinshe]

@slouken , I took some time, save the avframe data to a file (RGBA raw data). and made a test project, read the raw data from the file and render with SDL3, here is the sample project SDL3_Alpha_Test.zip

Put this raw data to C:/ ( or change the source code first) data.zip

[slouken]

This works for me:

extern "C" {
#include "SDL3/SDL.h"
};
#include <iostream>
#include "DataControl.h"
#include <vector>
#include <chrono>
#include <thread>

int main()
{
    SDL_Window* Window = SDL_CreateWindow("", 400, 300, SDL_WINDOW_BORDERLESS | SDL_WINDOW_OPENGL | SDL_WINDOW_TRANSPARENT);

    SDL_Renderer * renderer = SDL_CreateRenderer(Window, "opengl", SDL_RENDERER_ACCELERATED);
    if (!renderer)
    {
        return -1;
    }
    int width = 1202;
    int height = 1604;
    SDL_Texture *texture = SDL_CreateTexture(renderer, SDL_PIXELFORMAT_RGBA32, SDL_TEXTUREACCESS_STREAMING, width, height);
    if (!texture)
    {
        return -1;
    }

        //SDL_SetTextureBlendMode(texture, SDL_BLENDMODE_BLEND);
        //SDL_SetRenderDrawColor(renderer, 0, 0, 0, 0);

    DataControl dc;
    uint8_t* dataPointer = nullptr;
    std::vector<uint8_t> loadedData;
    int loadedInt = 0;

    if (dc.LoadData(loadedData, "c:/data.bin")) {
        dataPointer = loadedData.data();
    }
    else {
        std::cerr << "Failed to load data." << std::endl;
        return 1;
    }

    if (dc.LoadInt(loadedInt, "c:/int.txt")) {
        std::cout << "Int loaded successfully." << std::endl;
    }
    else {
        std::cerr << "Failed to load int." << std::endl;
        return 1;
    }

    //std::thread myThread([&] {

        SDL_UpdateTexture( texture, NULL, dataPointer, loadedInt );

        SDL_RenderClear( renderer );
        SDL_RenderTexture( renderer, texture, nullptr, nullptr );
        SDL_RenderPresent( renderer );

        std::this_thread::sleep_for( std::chrono::milliseconds( 2000 ) );

    //});

    //myThread.join();
}

The key is making sure you're using the OpenGL renderer rather than Direct3D.

[huangjinshe]

@slouken , This is little weired but showing correct now.

Is this a real transparency or just let alpha channel look like transparent but not real transparent? because after I change the code:

SDL_RenderPresent( renderer );

To:

SDL_RenderPresent(renderer);
while (true)
{
    std::this_thread::sleep_for(std::chrono::milliseconds(25));
    SDL_PumpEvents();
}

And when I click the position which transparent, my mouse couldn't click the element behid the window. Like this, Could I click the file behid of the window?

[sam20908]

That requires layered windows again. A quick Google search shows potential code you can use: https://stackoverflow.com/questions/65286495/make-a-win32-window-transparent-to-messages-click-touch-keyboard

The transparency is real. There's a difference between using the alpha channel for transparency vs masking away the background from layered windows.

[huangjinshe]

That requires layered windows again. A quick Google search shows potential code you can use: https://stackoverflow.com/questions/65286495/make-a-win32-window-transparent-to-messages-click-touch-keyboard

The transparency is real. There's a difference between using the alpha channel for transparency vs masking away the background from layered windows.

I tried to use Windows API CreateWindowEx with SDL_CreateWindowFrom to create the Window by myself before, at least I could change the window settings, but now we're using SDL_CreateWindowFrom, and Renderer with opengl, but this made another issue which can't go through the transparent(alpha) area.

Any good suggestions? We have already come this far. @sam20908 Do you suggest me write some code use opengl api? It make things more complex. I use SDL just for make things more esier.... use SDL + still need to use opengl might increase learning costs. How about give me a way get the window handler from SDL3? I saw some people use this code in SDL2:

SDL_SysWMinfo wmInfo;
SDL_GetWindowWMInfo(window, &wmInfo);
HWND hwnd = wmInfo.info.win.window;

But won't work anymore. Do you think If I get the window handler, then I use API change some settings make the real transparency possible?

[sam20908]

One thing you could try is making SDL use the software renderer, but then you'd have to manually blit the surface and I have no idea if having a surface will make the hit test not work.

You can also try getting the window pointer (after creating the SDL window) and manually change the style, etc (like using SetWindowLong)

I agree that the learning cost may not be worth it. I am not sure if the other platforms' transparent window created by SDL has this hit-test feature.

I have not tried this with the hit test yet, but there is an undocumented (and unknown if it will ever be considered "stable") method to create a transparent window in Windows 10 using SetWindowCompositionAttribute (see https://github.com/selastingeorge/Win32-Acrylic-Effect for example). Again, I am not sure if the result will be different than using DWM.

[huangjinshe]

@sam20908 Could you tell me how to get the window pointer after create SDL window, I'll try this one first. SDL_GetWindowWMInfo can't work anymore.

[slouken]

SDL_GetWindowWMInfo() should work, you call it like this:

SDL_GetWindowWMInfo(window, &info, SDL_SYSWM_CURRENT_VERSION);

[sam20908]

I guess SDL3 changed that API too

[huangjinshe]

The thing not going very well. I tried use this api:

SetWindowLong(wmInfo.info.win.window, GWL_STYLE, ((WS_CHILD | WS_VISIBLE) & ~WS_POPUP));
SetWindowLong(wmInfo.info.win.window, GWL_EXSTYLE, (WS_EX_LAYERED & ~WS_EX_TRANSPARENT));

But nothing changed. then I tried put this in a new window, because I need to utilize it, also I could control the window to ignore hit test, at the end I still need to put it(SDL_Window) in some window as a customized control . So I tried to create a window and put it in another one.

So I create 2 windows:

    HWND hWnd = CreateWindowEx(WS_EX_TRANSPARENT,
        L"static",
        NULL,
        WS_BORDER | WS_VISIBLE,
        CW_USEDEFAULT, CW_USEDEFAULT,
        640, 480,
        NULL,
        NULL,
        NULL,
        NULL
    );

    HWND hWnd2 = CreateWindowEx(
        WS_EX_LAYERED,
        L"static",
        NULL,
        WS_POPUP | WS_VISIBLE,
        0, 0, 
        500, 500, // Width and height
        hWnd, // Parent window handle
        NULL,
        NULL,
        NULL
    );

After that I tried call SDL_CreateWindowFrom(hWnd2), but it always return null , turns out I need to use this way:

SDL_Window* pSampleWin = SDL_CreateWindow("",0, 0, SDL_WINDOW_HIDDEN | SDL_WINDOW_TRANSPARENT | SDL_WINDOW_BORDERLESS | SDL_WINDOW_OPENGL);

char sBuf[32];
sprintf_s<32>(sBuf, "%p", pSampleWin);

SDL_SetHint(SDL_HINT_VIDEO_WINDOW_SHARE_PIXEL_FORMAT, sBuf);
SDL_Window * Window = SDL_CreateWindowFrom(hWnd2);
SDL_SetHint(SDL_HINT_VIDEO_WINDOW_SHARE_PIXEL_FORMAT, nullptr);

SDL_DestroyWindow(pSampleWin);

Little weird, but worked. and then I need to put the SDL hosted window in the main window:

SetWindowLong(hWnd2, GWL_STYLE, ((WS_CHILD | WS_VISIBLE) & ~WS_POPUP));
SetWindowLong(hWnd2, GWL_EXSTYLE, (WS_EX_LAYERED & ~WS_EX_TRANSPARENT));

SetParent(hWnd2, hWnd);
ShowWindow(hWnd, SW_SHOW);
UpdateWindow(hWnd);

It's all normal, but the renderer start to throw exception. it wont allow me use opengl keyword :

SDL_CreateRenderer(Window, "opengl", SDL_RENDERER_ACCELERATED); // Error
SDL_CreateRenderer(Window, Null, SDL_RENDERER_ACCELERATED);

So I can't able to see the final result. without "opengl" it show up, but still with the black background

[huangjinshe]

I did tried directly use SDL_CreateWindow create window and use SDL_GetWindowWMInfo get the window handler then put it to the main window before. but the image never show up.

I'm still more perfer SDL_CreateWindowFrom, why don't you guys just use dx11 create transparent window which allow put in another window for Windows? SDL4 maybe?

I thought SDL3 would more advanced ever.... if I need to render it in Winform or WPF use opengl , dx3d render the image or video, the SDL turn to useless, but I hope we could make it better for suitable many cases, not only the black background with limitations.

The last sample project, incase someone need it. SDL3_Alpha_Test.zip

[sam20908]

I tried the same as you, but it turns out DX11 doesn't really have an "alpha channel" like OpenGL. I did lots of my own research in the past around this area as well, and I am not in the slightest surprised at the walls you have to hop through :P

Blaming the SDL is not a well-thought-out way. We are likely dealing with Windows-specific gimmicks of window handling, and SDL tries to keep the behaviour consistent across platforms. It's true SDL does its own bookkeeping even if you used SDL_CreateWindowFrom, so maybe the expert contributors (I am not an expert in Win32) can look into relaxing some "limitations" like the ones you described.

My final bits of knowledge I will share: There exists a flag to remove the background bitmap (literally removing the window's rendering surface) called WS_EX_NOREDIRECTIONBITMAP which is a Window extended style you can modify. But, the con is that in order to render anything, you have to use DirectComposition (which I do not think SDL supports on a Win32 window, but the code to use it exists). Then you can do the rendering with DirectX. Again, I did not try this myself.

I wanted to create a PR to create the necessary swap chain for DirectComposition through a flag a user can pass (maybe?), but even now I do not know how the code in SDL works that well.

[huangjinshe]

@sam20908 Thanks for the advice, WS_EX_NOREDIRECTIONBITMAP would let my window HitTest turns to false, which mean when I click the window it will directly click the element behind the window.

But for alpha transparent in some cases, I think I might give up use SDL for render it if there are no any good way for mouse come through the alpha, maybe I should directly use some element in WPF like ImageControl (if it support), or use directX runtime + Microsoft.Wpf.Interop.DirectX.D3D11Image (this is supported, because I saw this before but it required dx runtime) in WPF window to render the image or video.

At the end I hope someone could help me check my last sample and give me a good advice for how to handle SDL_CreateRenderer exception when I put the SDL window in another window as a sub window, once this worked, I think I'll still use SDL3 as my default renderer, because I still have a big hope of the GPU acceleration in SDL3, even thought it can't go through alpha, I still need to display the image which has alpha chnnal as transparent in the sub window.

[sam20908]

@huangjinshe Thanks for your patience! I also hope that maybe having a built-in GPU API would help fix things like this as a side effect.

Not all hope has been lost though, because the Windows Terminal was able to achieve transparency despite not using OpenGL (and my NVIDIA Optimus setup). This implies there exists more accurate technology within Windows that will work better, and maybe even allow clicking behind in transparent areas. AFAIK it's some tech inside UWP (but UWP is officially being "deprecated" by Microsoft, AND it's removed from SDL3). What I am trying to say is there exists a lot of Windows frameworks one can investigate, and see if they have the option to do so (and maybe integrate into SDL via a PR).

[huangjinshe]

Hi @slouken , I made alot of test, even though we use SDL_CreateWindow create window by SDL, but when use SDL_CreateRenderer(Window, "opengl", SDL_RENDERER_ACCELERATED); we can't put this window(get window handler from SDL) into other window as sub window. But if we replace opengl to NULL, the image showing but still losing alpha. This is like a dead circle. We can get it, but we can't use it.... Is there any other way to make it?

Here is the sample: SDL3_Alpha_Test.zip

[huangjinshe]

I think the main problem we always can't handle it basically because of the SDL always use whole window for rendering, how if it only render an image in memory, and we use "interop" or "interoperability" to share image memory between your C++ code and the Winform/WPF application image control in windows? I saw some people directly pointed the d3dimage as ImageConrol.Source in WPF application (not whole window, only the image, obviously image support RGBA) , and update the texture of d3dimage in the c++ code. Use this idea if we only update image, it might be more easier to show SDL texture by any window , in any position of the window.

Of course this is just an idea, currently we need to update image directly from GPU for hardware acceleration. it all need to discuss more.

[slouken]

I made a change here that allows using the direct3d11 renderer with transparent windows: https://github.com/libsdl-org/SDL/commit/adfaa65e9efec648b5f9ed568fdc3198f8da0c63 This isn't in the main SDL code yet, but you can try it out now and see if it helps you!

[sam20908]

DirectComposition may also be worth investigating for SDL.

[huangjinshe]

I did check the UI frameworks from Windows, apparently put a window to another window has a limitation from all the UI frameworks.

MFC: Implementation: Use CWnd::Create or CView. Limitation: Airspace issues; embedded windows cannot be transparent and must be on top.

WinForms: Implementation: Set the TopLevel property to false, then add to controls. Limitation: Similar airspace issues; lack of transparency for nested controls.

WPF: Implementation: Use HwndHost to embed an HWND. Limitation: Airspace issues; no transparency over HwndHost; HwndHost content is topmost within bounds.

UWP: Implementation: Use Windows.UI.Xaml.Hosting.DesktopWindowXamlSource. Limitation: Airspace constraints when hosting classic Win32 content.

WinUI 3: Implementation: Similar to UWP, use DesktopWindowXamlSource. Implementation: While improved, still has some airspace limitations.

MAUI: Implementation: Embed native views using handlers or custom renderers. Implementation: Layering and transparency can be problematic due to native view embedding.

Each framework allows window embedding with specific methods but shares a common limitation related to airspace management and transparency of hosted windows. I think we should use more modern way for rendering, not just create a window for everywhere, let developers and the final application have more flexibility, like put the render area any place in their own window.

For Windows the only choice for now something like render SDL content onto a Direct3D surface and then present that surface using different methods depending on the platform, with hardware acceleration and potentially alpha transparency.

WinForms: Render SDL content to a Direct3D surface. Use a Panel or similar control as a placeholder in the WinForms UI. Handle the rendering within this control, drawing the Direct3D content using custom painting code or a library like SharpDX or Vortice.Windows. Transparency can be achieved within the Direct3D render target.

MFC: Similar to WinForms, render SDL content to a Direct3D surface. Host the Direct3D rendering surface in an MFC view or dialog that is designed to interop with Direct3D. Transparency in the Direct3D content can be managed, but MFC itself does not support alpha transparency between HWND-based controls.

WPF: Render SDL content to a Direct3D surface. Use D3DImage to display this content in WPF with hardware acceleration. Alpha transparency is supported by D3DImage.

WinUI: Render SDL content to a Direct3D surface. Use SwapChainPanel to display the rendered content with hardware acceleration. Alpha transparency within the displayed content should be supported if properly managed in the Direct3D pipeline.

MAUI: MAUI supports interfacing with platform-specific graphics APIs. For Windows, use a similar approach as WinUI with a SwapChainPanel. For other platforms, need to use their respective native graphics APIs. Transparency handling would depend on the specific API and compositing setup.

For other Platforms, like Linux, Mac....etc, use OpenGL?

[sam20908]

@huangjinshe AFAIK if SDL uses DirectComposition (integrates with DX11-12 I think, minimal work and uses different swap chain and change some parameters), then it could be a good thing to look at.

Windows has a nasty habit of deprecating frameworks and re-inventing new ones. I would not bet on SDL officially supporting any of them besides raw Win32. An interesting solution would be to expose enough internal information through some means (maybe through properties?) and let the user render however they like.

[huangjinshe]

I made a change here that allows using the direct3d11 renderer with transparent windows: adfaa65 This isn't in the main SDL code yet, but you can try it out now and see if it helps you!

It finally show the image with transparent background in sdl window(but the mouse can't throuhg the alpha), also I can't put it to another window by use: SetParent(sdlWindowHandler, hWnd); the sdl window would disappeared. now like I'm using "opengl" hint.

[slouken]

We have greatly improved shaped window support in SDL 3.0, can you try it there?