microsoft / Windows.UI.Composition-Win32-Samples

Windows.UI.Composition Win32 Samples
MIT License
464 stars 186 forks source link

Screen capture in combination with remote desktop software (Parsec) locks the mouse to the remote window #137

Open NickThissen opened 1 week ago

NickThissen commented 1 week ago

I'm using the Windows.Graphics.Capture api's as in your example to capture a specific window of my application. One of my users wants to use my application remotely via remote desktop software, specifically Parsec. He noticed that his mouse gets locked / stuck to the remote desktop window whenever the screen capture is enabled. In other words, after enabling screen capture on the remote desktop (the app is actively listening for frames, and is displaying the capture border), then on his local PC, the mouse cursor cannot leave the Parsec window which is displaying the remote desktop. As soon as I stop my app from capturing the screen, the mouse can move outside of the window and interact with the local PC again.

I guess there is some kind of interference going on here where Parsec is capturing the screen (to send back to his local PC) and my application is also capturing the screen (to capture the graphics in my application). Is there anything on the 'screen capture app side' that can prevent this mouse locking behavior? It seems equally likely that the remote desktop software (Parsec) can or should do something to mitigate, but I'm looking for all angles for a solution.

To note: the screen capture itself still works perfectly fine. My app is still capturing the content just fine, and Parsec is still streaming just fine too. It's just the mouse that gets stuck in the window which prevents my user from using it.

I thought it may be related to the 'capture mouse cursor' setting, but it happens the same whether the mouse cursor capture is turned on or off.

NickThissen commented 1 week ago

This is info I got from Parsec. Is it a known issue? Is there nothing I can do in the screen capture to resolve it? Thanks.

It's a conflict between DXGI Desktop Duplication and Windows Graphics Capture. WGC is changing the cursor mode and embedding it into DXGI's capture. This is a Windows bug

robmikh commented 1 week ago

I'll have to test this for myself... but we haven't had this reported to us before.

It's true that WGC forces the software cursor, and that causes the cursor to be drawn in frames returned by DDA... but that shouldn't affect your ability to move the mouse...

We've been working to remove the software cursor behavior from WGC but that isn't ready to ship yet.

Thanks for reporting this!

NickThissen commented 1 week ago

Perhaps "lock the mouse" is too vague. What I mean is: