smourier / VCamSample

Windows 11 Virtual Camera sample code.
MIT License
32 stars 7 forks source link

GPU & CPU #2

Closed My-zhai closed 6 months ago

My-zhai commented 7 months ago

In Win11, the system camera uses a GPU 屏幕截图 2024-04-19 145515

OBS uses CPU Screenshot 2024-04-19 14-56-16

modify 【_renderTarget->Clear(D2D1::ColorF(0, 255, 255, 0.5));】rgba->a 0.5 屏幕截图 2024-04-19 145651 Comparing two images, setting 【 rgba ->a 0.5 】 did not take effect in the CPU channel

smourier commented 7 months ago

The fact that you're on CPU vs GPU can depend on the process which started the intermediate Frame Server process. Also in your code, D2D1 ColorF uses float values between 0 and 1 (note it's not a problem it will be clamped to 1)

As for the transparency issue, yes there was an inconsistency the code between CPU & GPU rendering. This has been fixed with this commit https://github.com/smourier/VCamSample/commit/6b5dcca87b21e49ca49c7a5459e6f5c795468d7f

Thanks for reporting this.

My-zhai commented 7 months ago

Thank you very much for your answer. I have made the necessary modifications. I would like to create a completely transparent camera, which means the background color of the camera is completely transparent. This is my first time using C++to write code, but I don't know what to do. Can you give me some suggestions.

smourier commented 7 months ago

If you mean transparent like seeing part of the screen and other windows, the camera is a surface you draw on, so the only way I can think would be to capture the screen for example using the WinRT graphics capture API (https://learn.microsoft.com/en-us/uwp/api/windows.graphics.capture).

There's an example of this here in C++ https://github.com/robmikh/Win32CaptureSample But this is nothing easy for a start.

My-zhai commented 7 months ago

I'm sorry for taking so long to reply to you. I've tried capturing a transparent window before, but it's not the desired effect. According to your notes, I will modify [MediaStream:: Initialize] to 1, which is the RGB32 mode, The video format captured in OBS, XRGB, is still the same as before. This virtual camera is not transparent. Screenshot 2024-04-21 04-22-39

When using the capture card mode in TikTok Live Studio, the captured video will be in RGB32 mode. I don't know if this video is in XRGB format, but it creates a magical phenomenon where the virtual camera becomes transparent

微信截图_20240421041707

After comparing with other transparent cameras, I speculate that if the video format is changed to ARGB, a completely transparent camera should be obtained

smourier commented 7 months ago

The black screen is expected. As for the other, I'm not sure what happens but it's possible this capture card mode captures all windows on the screen using windows.graphics.capture or another technology (hooks, etc.) and merges them together.

My-zhai commented 7 months ago

TikTok Live Studio captured a transparent virtual camera from the capture card mode, and the background behind it is an image, not a Windows desktop. Therefore, using rgb32 mode, it is entirely possible to achieve virtual camera transparency 微信截图_20240421143437

The following image shows the collection mode of TikTok Live Studio and OBS 微信截图_20240421144345

微信截图_20240421144603

smourier commented 7 months ago

Is there a question here?