Feature: Better touchscreen support (Multitouch)

[Felix14-v2]

Description Now there are 3 platforms that support touch input and are able to run Java Edition — Android/iOS via Pocket Java Launcher and Windows 10 installed on tablets. The number of such devices capable of running JE is growing, but so far the Java Edition does not have touch control support.

Solution MidnightControls could implement several types of touch control of the game by adding the necessary buttons (or joysticks) with actions in the UI. And, possible, some additional controls.

Alternatives Bedrock-ify is a good base to request this feature, but I think that we need it outside bedrock-styled segment. Despite the fact that most touch-screen devices run on Android, this feature would be in demand on touch-screen laptops or tablets with Windows, whose owners don't necessarily want to "bedrockize" their Java Minecraft. In addition, the mod developer stated that he was not ready to work on touch controls in this issue, so, here I am.

Additional

My current Pojav controls settings


[Motschen]

The basics of this feature were already implemented in LambdaControls, but never really finished. I'll see what I can do about this in the future.

[Adzetko]

Although the readme says midnightcontrols has an experimental and bugged touchscreen support, is there actually any way to enable it?

[Motschen]

The Touchscreen support was commented out in later versions of LambdaControls, I'll uncomment it soon

[Motschen]

Added in v1.2.0 :)

[Poopooracoocoo]

I thought I might add just how buggy it is: https://github.com/LambdAurora/LambdaControls/issues/42 https://github.com/LambdAurora/LambdaControls/issues/94 https://github.com/LambdAurora/LambdaControls/issues/86 says

The issue that I encountered with touchscreen is, well, on desktop it considers it a single pointer: the mouse, so you can't use multiple fingers at once, or you would need to access some very low-level, potentially breaking libraries to hack into the system to just make it work. So yeah, not worth it imo.

This explains why the touch controls in LambdaControls and MidnightControls are so poor and is why Lambda ultimately dropped it. I don't know why Minecraft doesn't handle touch events properly. It seems like something that only Mojang can fix. I don't know why though and it'd be great if someone could clarify that. I just read a bit about when Chromium added support for touch on Windows but I didn't understand it.

[Felix14-v2]

Yeah, now it is *a bit* buggy... Tests with Pojav: https://youtu.be/JEEhm4H-Cs8

[Motschen]

It likely clashes a bit with Pojavlauncher's built-in mouse emulation, I'm going to add an option to disable MidnightControls' mouse emulation in the next version. As for the multitouch input on desktop OS's Lambda mentioned in the issues, I think the situation has improved a bit, at least on Linux using wayland (it works pretty well on Steam Deck after all I know).

[Motschen]

Also, Mojang have added preliminary touch input support in a recent update (1.18 I think), maybe I could take advantage of that.

[Adzetko]

About the official "touchscreen mode" in Minecraft, I never understood what it did apart from toggling a boolean? I know this isn't the point of this issue, but is it supposed to do something?

[Felix14-v2]

It likely clashes a bit with Pojavlauncher's built-in mouse emulation

Maybe. Previously, lambdaControls had problems working on it, but these problems were similar both on a tablet (win10) and on a phone with LambdaControls. I will soon test the latest MidnightControls on my tablet.

[Adzetko]

Honestly, I know I'm in an edge case, but I'd be okay if I just have a touchscreen ui without the controls part, so it doesn't have to support mulitouch if that's a bummer to support, as when using Steam Link, I use the app's on screen display for movements. I believe it's the same for pojav? Anyway, I think pojav/moonlight/steam link/mobile setup users, at least, want their clicks to be more touch-friendly (on my Steam Link app, without this mod, I tried setting tap as right click and hold as left click, but it's not as simple as that, I get many interactions wrong). Also, would it be considered as cheating when playing online if you'd hit at what you're tapping and not at the center of your screen? Like with the circular overlay of the bedrock edition? It's practical when you quickly need precision (creeper jumpscares anyone?)

[Felix14-v2]

About the official "touchscreen mode" in Minecraft, I never understood what it did apart from toggling a boolean?

It changes the logic of moving items through the inventory. Instead of...

• press the item slot
• release the slot
• move your mouse
• press the new slot
• release the new slot ... you get
• press the item
• move the mouse
• release the item

Not so convenient, actually. Especially on the touch screen. Ironically, this mode was called "touchscreen mode".

[Poopooracoocoo]

The touch controls remain terrible on Windows by the way. Spinning death. I couldn't press escape to free my cursor but I was able to just tap on close window. 😆

[Felix14-v2]

Finally I was able to test it on my tablet. Unfortunately, it doesn't work properly at all. https://youtu.be/tYRyEEzx3Es

[WesleyVanNeck]

@Motschen this might fix the touch issues https://github.com/Tungstend/TouchInjector

[nimfq]

" Вероятно, это немного противоречит встроенной эмуляции мыши Pojavlauncher, я собираюсь добавить опцию отключения эмуляции мыши MidnightControls в следующей версии. Что касается мультитач-ввода на Lambda настольной ОС, упомянутой в выпусках, я думаю, что ситуация немного улучшилась, по крайней мере, в Linux, использующем wayland (в конце концов, он довольно хорошо работает на Steam Deck)." Does this mean that this function should work in Linux?

[Felix14-v2]

Today I dreamed that I was playing Java Edition with a touchscreen... Just a fun fact :)

[plasmatank]

@Motschen this might fix the touch issues https://github.com/Tungstend/TouchInjector

Seems doesn't work either, installed it on a vanilla minecraft due to poor instructions, in the end the game still just rapidly spinning

[Motschen]

Today I dreamed that I was playing Java Edition with a touchscreen... Just a fun fact :)

Your dreams have now become reality, with MidnightControls 1.9.0! Even though multitouch is still not supported [yet ;) ], the touchscreen is now actually usable (tested on my Steam Deck in Desktop mode)

[Felix14-v2]

Just found this interesting project for Minecraft 1.8-1.12: https://github.com/Aang23/MC-TouchControls

It relies on an external framework to achieve multitouch. Maybe it is possible to use the same approach?