Windows multi-touch support

[Marocco2]

Is your feature request related to a problem? Please describe.

Current implementation on moonlight android involve specific hardcoded touch/gesture inputs for touchscreens (3 fingers for popping up Android keyboard eg.) but it will overwrite native Windows capabilities for touchscreens (scrolling, long press, pinch to zoom etc.)

Describe the solution you'd like

Add an option for preferring native multi-touch for supposed hosts (Windows)

Describe alternatives you've considered

Moonlight-qt is a perfect example for multi-touch support and I'd like to have the same implementation here on moonlight android

Screenshots

No response

[cgutman]

Yep, this is actually already implemented but disabled. The problem preventing us from shipping it on Android is that full native multi-touch interferes with the ability to launch the virtual keyboard using the 3-finger tap combo. We need to implement some kind of overlay or something to allow activation of the keyboard before we can turn it on. We should also probably add some type of zoom feature since touch targets can be very tiny on a phone/tablet screen.

This wasn't an issue with the PC client because it never had virtual keyboard support.

[Marocco2]

This wasn't an issue with the PC client because it never had virtual keyboard support.

I'm curious: are there cases where android virtual keyboard got the upper hand over Windows one? On my use case, Windows touch keyboard would pop up even on UAC prompts (on moonlight-qt) and it has keybindings with Win/Ctrl/Shift. So, perhaps, it's for Linux? I really don't know

[tianpengwan]

This paragraph was translated. I think it is necessary to support multiple touch screens for operation on mobile phones. On the contrary, there are more keyboard calls, and I haven't found this feature after playing for half a month. If this affects the function, it would be more than worth the loss. My suggestion is to change the number of fingers exhaled, such as 5 fingers exhaling without affecting its function, or give users a chance to choose 这段话使用了翻译。我认为手机上操作支持多个触屏是有必要的，反而呼出键盘有些多于，玩了半个月没发现有这功能如果说这影响了功能，这未免得不偿失。我的建议是更改呼出的手指数量，比如5指呼出不会影响其功能，或者给用户一个选择的机会

[tianpengwan]

太难受了哇哥 【PC版原神串流触屏-哔哩哔哩】 https://b23.tv/gBTZ9wz

[tianpengwan]

Yep, this is actually already implemented but disabled. The problem preventing us from shipping it on Android is that full native multi-touch interferes with the ability to launch the virtual keyboard using the 3-finger tap combo. We need to implement some kind of overlay or something to allow activation of the keyboard before we can turn it on. We should also probably add some type of zoom feature since touch targets can be very tiny on a phone/tablet screen.

This wasn't an issue with the PC client because it never had virtual keyboard support.

hihihi

[cxw620]

Yep, this is actually already implemented but disabled. The problem preventing us from shipping it on Android is that full native multi-touch interferes with the ability to launch the virtual keyboard using the 3-finger tap combo. We need to implement some kind of overlay or something to allow activation of the keyboard before we can turn it on. We should also probably add some type of zoom feature since touch targets can be very tiny on a phone/tablet screen.

This wasn't an issue with the PC client because it never had virtual keyboard support.

Does it mean that multi-touch support has been already implemented, but has not been released yet?

[Marocco2]

Yep, it’s in the code. You can compile it yourself and enable it

[ZirrorDmage]

Now I understand why we use long press to achieve right click instead of double finger click，maybe we can use another why to adapt these needs, such as floating pop-up window. Altough multi-touch game is not common on PC, but I think cut off this function is a loss. RD client is a good teacher for us to learn from, two input mode can fit different demands.

[ocelot233]

Yep, this is actually already implemented but disabled. The problem preventing us from shipping it on Android is that full native multi-touch interferes with the ability to launch the virtual keyboard using the 3-finger tap combo. We need to implement some kind of overlay or something to allow activation of the keyboard before we can turn it on. We should also probably add some type of zoom feature since touch targets can be very tiny on a phone/tablet screen.

This wasn't an issue with the PC client because it never had virtual keyboard support.

@cgutman I think that the three-finger gesture for bringing up the keyboard is not a good design choice, as seen in my XiaoMi Pad 5 Pro. It takes over all three-finger gestures: a long press with three takes a region screenshot, swiping down with three takes a screenshot, and swiping up with three opens the task manager. On its system, the keyboard invocation in your software becomes ineffective. Instead, sacrificing touch functionality to achieve this feature is unwise. Some games, such as Genshin Impact, require multi-touch gestures to play properly when touch UI is enabled through launch parameters.

I suggest taking inspiration from the design of Microsoft RDP. Consider adding a toggle button and/or a separate button for bringing up the keyboard, and/or include some settings that allow users to choose from various gesture-based operations.

I used machine translation. Below is the original text. 我认为三指呼出键盘不是一个好的设计，就像我的XiaoMi pad 5 pro，它占用了所有的三指操作逻辑：三指长按是区域截屏，三指向下划动是截屏，三指向上划动是任务管理，您软件中的呼出键盘在它的系统上就会失去作用。相反，为了实现这个功能牺牲触控的功能是不明智的。有些游戏，如genshin impact，可以通过启动参数启用触摸ui，需要多指触控才能正常游玩。我建议您参考Microsoft RDP的设计，添加一个切换按钮和/或独立的呼出键盘按钮，和/或添加一些设置项使用户可以自己选择几种操作逻辑中的一种。

[TrueZhuangJia]

want this feature on iOS moonlight pls

[Moeweb647252]

Yep, it’s in the code. You can compile it yourself and enable it

I think adding a configure entry of touch screen in settings is feasible

[nhac-lly]

Why not add this as option? I mean I never used 3 fingers gesture to use Android keyboard but built in Windows keyboard instead.

[samkitty]

We already have the option to hold start button for mouse mode, why not hold select button for keyboard? Do people use Moonlight without a controller that often? Wouldn't they also have the controller overlay (and thus a select button on screen which they can tap and hold)? I feel not having native touch support is one of Moonlight's biggest downsides and is the only reason I still use SuperDisplay. The fact this feature is so within reach but not enabled is quite frustrating. It can surely be added as an option in settings?

[Marocco2]

We already have the option to hold start button for mouse mode, why not hold select button for keyboard? Do people use Moonlight without a controller that often? Wouldn't they also have the controller overlay (and thus a select button on screen which they can tap and hold)? I feel not having native touch support is one of Moonlight's biggest downsides and is the only reason I still use SuperDisplay. The fact this feature is so within reach but not enabled is quite frustrating. It can surely be added as an option in settings?

Yes, it can surely be an option and it would be quite easy to make a PR. I've made my own build for it as I don't use Android keyboard at all, so I don't need an option for it

[Marocco2]

You only need to remove / and / before compiling your own Moonlight build

https://github.com/moonlight-stream/moonlight-android/blob/6551076613fd4bf511aa5a0fcf56080cef576651/app/src/main/java/com/limelight/Game.java#L2050-L2057

[samkitty]

You only need to remove / and / before compiling your own Moonlight build

Thanks for the info! I have since moved to a Chinese fork of the Moonlight client which includes native touch support as well as a handful of other features such as a full windows keyboard overlay and the ability to switch between touch/trackpad mode on the fly by using the back gesture on your phone to access a hidden menu.

Search for user 阿西西的日常 if interested.

[tianpengwan]

谢谢，我找到了~o(〃'▽'〃)o

---Original--- From: @.> Date: Tue, Mar 19, 2024 08:26 AM To: @.>; Cc: @.**@.>; Subject: Re: [moonlight-stream/moonlight-android] Windows multi-touch support (Issue #1271)

You only need to remove / and / before compiling your own Moonlight build

Thanks for the info! I have since moved to a Chinese fork of the Moonlight client which includes native touch support as well as a handful of other features such as a full windows keyboard overlay and the ability to switch between touch/trackpad mode on the fly by using the back gesture on your phone to access a hidden menu.

Search for user 阿西西的日常 if interested. IMG-20240317-WA0003.jpg (view on web)

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[TrueZhuangJia]

want this feature on iOS moonlight pls

Just created a fork myself to implement multi-touch on iOS. https://github.com/TrueZhuangJia/moonlight-ios-NativeMultiTouchPassthrough/tree/Moonlight-iOS-ZWM

[MuhamadRifkii]

I made a fork of moonlight for android with a few added feature like gameback menu https://github.com/moonlight-stream/moonlight-android/pull/1171 Perfomance overlay toggler https://github.com/moonlight-stream/moonlight-android/pull/1219 And custom resolution https://github.com/moonlight-stream/moonlight-android/pull/1349

I did not write the original code, just applying it for personal use.

https://github.com/MuhamadRifkii/moonlight-android

[protodevnan0]

I made a fork of moonlight for android with a few added feature like gameback menu #1171 Perfomance overlay toggler #1219 And custom resolution #1349

I did not write the original code, just applying it for personal use.

https://github.com/MuhamadRifkii/moonlight-android

No words can describe how much I love You for this.