search

moonlight-stream / moonlight-ios

GameStream client for iOS/tvOS
GNU General Public License v3.0
1k stars 248 forks source link

[visionOS] Right click (via magic trackpad or mouse) does not work #613

Open mddfe opened 3 months ago

mddfe commented 3 months ago

When using the apple vision pro, the iPad version of Moonlight will not recognize a right click via touchpad (a two finger press) or mouse. There are some iPad apps that do recognize these actions (splashtop, in particular), so it seems to be something finicky with the way the vision sends these actions to the app (maybe touchesEnded not reporting peakTouchCount as two?). This behavior happens regardless of the citrix mouse settings.

Device details

Server PC details (please complete the following information)

edit: According to Steven Troughton-Smith: 'since iOS 17, UIApplicationSupportsIndirectInputEvents defaults to yes, which changes a couple things about how mouse events are routed'. Looks like UIApplicationSupportsIndirectInputEvents was already set to 'true' anyway so I doubt that's the issue.

edit2: I've found the issue. in StreamView.m:

            if (@available(iOS 14.0, *)) {
                if ([GCMouse current] != nil) {
                    // We'll handle this with GCMouse. Do nothing here.
                    return YES;
                }
            }

needs to be removed/commented out to properly function on VisionOS (two places).

vrxgaminginc commented 2 months ago

@mddfe I tried this and it does indeed work. However, there's a slight difference between magic trackpad and visionos' finger pinch and move for the mouse. For the most part, it works fine. However, there is some jumpiness when using the trackpad vs finger movement. I am also using a synergy clone (software kvm) to share my PCs mouse with my mac. Vision os' pinch to move the mouse allows me to control the mouse on the mac and pc. However, the trackpad only works with the PC and not the mac. Any idea what the difference is between the trackpad and visionos' pinch to move action? Or how I can simulate vision os' pinch to move in StreamView? Thanks!

Edit: ok. the difference between trackpad and visionos' finger pinch is absolute vs relative when sending the data over the network. I changed the trackpad to send deltas rather than absolute screen values and it worked. it's likely because synergy considers the mac to be off the screen coordinates (like multiple monitors), so moonlight/sunshine wasn't able to accomodate that properly when working with absolute screen coordinates since it doesn't know about it.

ghost commented 1 month ago

@vrxgaminginc do you know if there was any progress on getting this to work?

RikuKunMS2 commented 1 week ago

I've updated a fork of the Vision OS version that supports right clicking with the magic trackpad here:

https://github.com/RikuKunMS2/moonlight-ios-vision/tree/visionos