More than one cursor for multiple clients ?

[shodanx2]

Hi,

I used windows sunshine as a shared computer for me and my friend

But we have an issue, only one of us can have the cursor in the screen at a time Or else it quickly flickers between the two positions

There is software such as eithermouse and mousemux, that can colour the cursors, so multiple users can use one screen at once and keep track of which cursor is whose

Here is what that looks like

https://github.com/user-attachments/assets/5d553efd-505e-4527-b2e9-a9c579dab187

I suggested this to the sunshine them

they closed and locked the issue without further explanation

https://github.com/LizardByte/Sunshine/issues/3044

That would be a great upgrade for us

I think what would be needed is to spawn separate virtual mouse instances per client

That was software like mousemux can tell the cursors are different mouses

[ClassicOldSong]

TL;DR: It can't be done right now. It might be possible in the future if remote USB passthrough could be implemented, but that would be a huge amount of work.

Physically, there could be only one mouse pointer for each PC, and computer would only expect and display only one mouse pointer at a time since mouse pointers are drawn by your graphics card separately from your screen content. That's why you still could see only one cursor even you connected 2 physical mice.

What Sunshine does here is, it uses Windows API to inject mouse events to the system, so there's no way for Windows to tell if the events are coming from separate mouse hardwares. Things like mousemux could only draw separate cursors on the desktop, hiding the actual hardware pointer on the screen, but (in theory, unless it hooks even lower parts of Windows) can't actually do clicking/dragging with different mice simultaneously. And it relies on physical device input to distinguish movements from different users.

Currently you could use VirtualHere to pass USB mouse to your computer remotely, and I think it should work with mousemux.

[shodanx2]

Hi ! I am curious if the mouse event could be duplicated as mouse "fake mice" in a manner similar to the game controllers ?

In our case I think it might be "fake touchscreen" I'm not sure ?

But I think there is another thing that could greatly improve usability while staying inside how things work at the moment.

The MAIN problem is that when there are two cursor, if one cursor clicks while the other moves, it will click at the location of the other cursor. I think click and drag events have coordinates. I think that whenever one client clicks or drag, for that duration, the other client mouse events should be entirely ignored so as to avoid clicks and drags in the wrong location.

Having properly supported multi mouse support would be great polish, but eliminating the misclick is where the major improvement over the current state of things is.

Also, VirtualHere is proprietary software and that would make integration very difficult. But there is open source USB over IP that might work great here.

https://github.com/cezanne/usbip-win

HOWEVER, in my case, where the moonlight screen is just a shared computer displayed on an extra monitor, I wouldn't want my whole mouse being captured by the moonlight client. Also mouse does relative mouse events but in my case, I use it as a virtual touchscreen. And that virtual touchscreen is happening in the client, not a physical usb touchscreen that I could redirect over IP.

So other than the above, more important misclick mitigation. I think the "long term" solution is emulating virtual devices like touchscreen or mouse, one of each per client, so that the host OS can tell them apart.

But also I recognise that our use of sunshine/moonlight as a shared remote computer is very novel. I don't know if services like discord can do "multi user desktop sharing" in this way either....

Anyway, we know of the current limitations and manage to work around it but it is a drag explaining the quirks of this methods whenever new people joins us to it.

[ClassicOldSong]

USBIP-Win is unstable and still not working with many devices, and requires turning on test mode, which is not acceptable in many cases and will more likely to cause security issues.

Separating emulated devices isn't possible at this moment either as it requires kernel mode drivers which will face the same situation as usbip-win, and kernel driver signing costs real money and that's why seldom have open source kernel drivers available for use without turning on test mode.

For the clicking at wrong position problem, it requires some modifications on the client, which I think is already done with the Android client, but for other platforms there's no guarantee and no ETA for that as I don't have time for them yet.

VirtualHere is proprietary but the effort they put into it making it compatible with so many devices is huge, it's understandable for it to not be free.

I understand that there're lots of cases where a shared desktop is used for escaping multi seat licenses for some paid softwares, but personally I won't encourage users to do this with open source projects that people donate their own free time to.

[shodanx2]

"a shared desktop is used for escaping multi seat licenses"

Thanks this does explain the apparent hostility toward such features that the sunshine people appear to have.

But in my case this is just a shared computer we have open while we play co-op games. We use sunshine because it doesn't have that bad feel of sluggishness that vnc and rdp have.

Still usb over ip wouldn't work without capturing the whole mouse and we use touchscreen mode so having sunshine server differentiating between clients is really the only way.

Also come to think of it. I believe those who do the multi-seat, don't have this issue at all, since they are each in their own independent desktop. My issue only arises because we use the same mouse on literally the same screen. Sometimes, I just "wave" at my friend with the cursor.

Oh well, maybe in the far future it will have this capability...

[ClassicOldSong]

Any well designed co-op games should not rely on having multiple mouse cursors? And I don't think there're games designed like this...

[shodanx2]

No, we do not play the games in that computer. We each play some game on the main computer and we have this moonlight window on the side monitor which is some old computer in my closet

Anyone on our chat can bring up some webpage or anything else, really just a shared general purpose computer

Here is what that looks like

This is really convenient to move, at any time I can just click on that window and it just works No capturing the cursor, no alt-tabbing. I can even just wave my cursor around without clicking it and the other can see me waving.

[ClassicOldSong]

It is really a weird use case though...