[Bug] Cursor misalignement in combined desktop and incorrect graphical capture of the desktop windows

[NergalStudio]

My computer is configured as follows:

• Tested with DesktopPlus v 1.8.4
• Windows 10, DesktopPlus has admin access, I used CreateElevatedTask.bat and EnableUIAccess.bat
• 2 gpus and 6 monitors, 3 monitors each

I have activated in performance: single desktop mirroring

With graphics capture and combined desktop I'm viewing all 6 monitors

With desktop duplication I'm viewing only the 3 bottom monitors

If i select each monitor independently the cursor is visible in the 3 bottom monitors but invisible in the 3 top monitors but is aligned with the index pointer.

It seems that the mouse cursor is 1920px to the right and 1080px down, as is not detecting the correct size of the combined desktop?

I tested also beta, newui and newui-browser >> all have the same cursor offset

I'm sharing a capture of the desktop with of how i see the controler pointer and mouse cursor position in the Index display

Thank you , this program is amazing

[elvissteinjr]

NewUI branch builds create log files in the application directory (DesktopPlus.log & DesktopPlusUI.log). They contain the desktop layout/GPUs as the application sees them, which would be very useful here.

Desktops across multiple GPUs isn't very well supported (not unfixable, but somewhat involved and pretty low priorty), but at least cursor inputs are supposed to be still correct, so this needs fixing. I do have a hunch about it maybe being related with the clear vertical separation of GPU 1 and GPU 2 desktops, but as said, logs would help me reconstruct accurately this on my end.

[NergalStudio]

Here they go, i hope they are useful.

Thank you

DesktopPlusUI.log DesktopPlus.log

[elvissteinjr]

Hi, I've managed to reproduce and fix the issue. Kind of feel bad about how it probably took less time than you doing this detailed write-up, though.

It essentially boiled down to the desktop offsets not being applied for combined desktop Graphics Capture, which is only an issue if there are desktops at negative coordinates. Turned out unrelated to the multiple GPUs at least, as the display enumeration thankfully appears to work fine.

Before I go and close this up, though, I do notice a handful of system transitions being logged on your end. These happen whenever the Desktop Duplication capture is dropped for various reasons (display changes, exclusive fullscreen switching, some display arrangement adjustments). Not much can be done except handling them but there's a concerning amount there. Were you doing anything related or did these resets come out of nowhere? The resets are relatively quick but I fear they might induce load spikes if they were to keep happening in the background.

Also, I see that Desktop+ is running with cross-GPU copy active, at least in some parts of the logs. When not using Desktop Duplication this won't change much (capture is paused if no overlay using it is active), but otherwise it incurs a performance penalty. Is the Index not connected to the 3090? Cross-GPU copy should only activate when the first detected desktop is not on the GPU SteamVR reports to use. Unless this was just something during testing, then disregard this.

Thanks for the report. If you need me to provide a new build, I'm looking to get one out soon but was also hoping to resolve someone else's issue first.

[NergalStudio]

Hello

I managed to compile the project with visual studio 2022, i tested master and new-ui branches and with graphics capture enabled combined desktop is working perfectly.

The mouse position in the first 3 screens works OK, the other 3 (2nd GPU) don't capture anything and the mouse position is incorrect. But if i select a window maximized in one of the 3 screens that don't work, it shows OK.

It seems they have the same kind of bug as the combined desktop capture.

The Index is connected to the 3090, GPU cross copy is off , where do you turn on and off this option?

I did not manage to activate the logs in my compiled version I attach the link to a video with the test

https://drive.google.com/file/d/1cJt3MITvDVNedhpMvNmQFlZV6tJLLVLg/view?usp=sharing

Thank you

[elvissteinjr]

Thanks for finding all those multi-GPU issues. A separate function doing display enumeration to get the monitor handle for a desktop ID didn't enumerate outputs correctly when there were desktops on multiple GPUs. Yet another oversight of mine. The inputs being off is a side-effect from Graphics Capture threads not notifying the application when this function fails. This is now treated as a lost capture so it won't produce mouse inputs anymore. Though it shouldn't get there in your case anyways.

Cross-GPU Copy is an automatic state (active when GPU of the first desktop is different from the VR headset connected GPU). If it's off, you don't have to worry about it, that's actually good. Cross-GPU Copy is only useful for laptop hybrid graphics cases. It copies the Desktop Duplication texture over to the VR GPU before sending it to SteamVR. Desktop Duplication itself as it's currently implemented in Desktop+ still has to run on a single GPU for all desktops, so this doesn't help when they're spread across multiple.

Logs not working for you would be because there's no detailed logging on the master branch (only fatal errors). NewUI has them and is supposed to replace master sooner than later, so I didn't bother backporting that stuff.

[NergalStudio]

Hello

I just tested your last code, and it works!!!

https://drive.google.com/file/d/1ZmaeJ-JKPA0xig0Ni4cOskij-BvpdFgV/view?usp=sharing

At first i notice some windows from my last test with the newui, steam was restoring the config with his cloud , i disable it and delete the old config.

I copied the new compiled files and voila it works...

I notice some quirks with the mouse when you move a window outside desktop boundaries, I show at the end of the video how i move the steamvr window outside the desktop and I'm unable to move it again with the controllers , but with the mouse I'm able to move it inside the desktop again.

I like a lot the new ui its easier to use and configure.

And outside the dashboard in the play area the new keyboard works with two hands!! like with the steam keyboard.

It seems that all the graphical capture issues are fixed except the cursor movement with controllers compared with the mouse.

Thanks a lot

[elvissteinjr]

At first i notice some windows from my last test with the newui, steam was restoring the config with his cloud , i disable it and delete the old config.

Steam Cloud is a blessing and curse in a way. There is an option to reset to default settings within the settings UI if you need it, though.

For the mouse cursor I couldn't quite reproduce what's shown in the video. I assume the seemingly off pointer isn't part of the complaint, since that's extrapolated movement by the laser pointer during held down input and doesn't take the overlay that happens to be right behind and showing what would be there into account (didn't sound like it was, but I wanted to point that out).

For the dragging case I'd blame the Laser Pointer Override setting, which is meant to detect external movement of the cursor and the disable laser pointer mouse movement until the overlay is clicked again (in the stable branch this was only active for the HMD pointer). Trying to point outside the valid cursor space/outside any desktop area triggers this as well since Windows denies it.

Not being able to drag this particular window seems odd however, since the SteamVR Status window is draggable from any non-interactive window space in my experience.

But that's just as an aside, I do see some usability issues there so I adjusted the behavior of the Laser Pointer Override a bit: In general, it won't be in effect while the left mouse button is pressed by the laser pointer, so it won't trigger while dragging anymore. Additionally it now only triggers while pointing inside one of the desktop areas. On top of that I also added an attempt to not have the cursor get caught at sticky display corners all the time, but it doesn't seem 100% avoidable (Windows sometimes catches it despite being quite far from a corner).

At the end of the day there's still the option to just disable that feature entirely, to be fair.

Admittedly, combined desktop isn't the most commonly used config since it's just a bit unwieldy, so it hasn't gotten much attention. Doubly so for the Graphics Capture variant, which seems to do funny things in the dead display space at times (displaying window content there as seen in your footage, but I've also had my wallpaper extend where it shouldn't). I get that capture as-as though and it's not that harmful, maybe even useful somehow, so I'm inclined to keep it that way.

Thanks again. Let me know if I missed something.