search

libretro / RetroArch

Cross-platform, sophisticated frontend for the libretro API. Licensed GPLv3.
http://www.libretro.com
GNU General Public License v3.0
10.14k stars 1.82k forks source link

Multitouch not working on overlays #9223

Open bobbybudnick opened 5 years ago

bobbybudnick commented 5 years ago

Description

I see from this commit that multitouch should be working within retroarch: https://github.com/libretro/RetroArch/pull/7483 I cannot get it to work when playing a game with the touchscreen and an overlay. My screen is 5-points touch and I can swipe scroll and pinch and zoom in Chromium as a testing example. This problem heavily affects usability when gaming with a touch screen.

Expected behavior

When moving Mario with the virtual D-pad with my left thumb I should be able to touch the "B" button with my right thumb to jump at the same time. This is how gaming on smartphones works.

Actual behavior

When moving Mario with the virtual D-pad nothing happens when I press the "B" button or any other button.

Version/Commit

You can find this information under Information/System Information

Environment information

bobbybudnick commented 5 years ago

After looking at this some more, I can see a potential problem. The commit makes changes to "switch_input.c" but this seems to be for the Nintendo Switch only.

If you look in "udev_input.c" on line 1055 and 1059 there are comments saying multi-touch unsupported.

The questions now is: what is involved in changing this to support multitouch? I don't mind if the changes are not good enough to include but how would we get started on this? It doesn't look too complicated, right?

hizzlekizzle commented 5 years ago

it has to be handled per input driver, AFAIK, so someone would just need to implement it for a driver and send a PR.

bobbybudnick commented 5 years ago

I looked into changing the code itself and it seems like it would be really tedious to change something and then have to wait for the whole of Retroarch to compile. Even researching how to avoid that may take days. I kind of operate on trial and error so it could take the better part of the rest of my life to do that.

I came up with a script-based solution that should add multitouch support to nearly any X11 application including Retroarch. I have tested it and it works. Mario can jump while running with the left thumb on the virtual d-pad and the right thumb on the virtual "B" button.

I have also added the script to the next release of the software for my Mobile Internet Devices. I will add a link below for both repositories. Unfortunately I think I may be getting too old because Super Mario World is very hard with the touchscreen controls. Either that or the script lacks in responsiveness some, but it is better than nothing.

Not to toot my own horn too much but I think I may be the first person to play a multi-touch game on GNU/Linux. I will go on thinking that at least until someone tells me I am wrong.

Linux Generic Multitouch Script: https://github.com/bobbybudnick/SFS-LINUX-GENERIC-MULTITOUCH

Mobile Internet Device: https://github.com/bobbybudnick/SFS-Mobile-Internet-Device/wiki

HelloMrMeseeks commented 2 years ago

I looked into changing the code itself and it seems like it would be really tedious to change something and then have to wait for the whole of Retroarch to compile. Even researching how to avoid that may take days. I kind of operate on trial and error so it could take the better part of the rest of my life to do that.

I came up with a script-based solution that should add multitouch support to nearly any X11 application including Retroarch. I have tested it and it works. Mario can jump while running with the left thumb on the virtual d-pad and the right thumb on the virtual "B" button.

I have also added the script to the next release of the software for my Mobile Internet Devices. I will add a link below for both repositories. Unfortunately I think I may be getting too old because Super Mario World is very hard with the touchscreen controls. Either that or the script lacks in responsiveness some, but it is better than nothing.

Not to toot my own horn too much but I think I may be the first person to play a multi-touch game on GNU/Linux. I will go on thinking that at least until someone tells me I am wrong.

Linux Generic Multitouch Script: https://github.com/bobbybudnick/SFS-LINUX-GENERIC-MULTITOUCH

Mobile Internet Device: https://github.com/bobbybudnick/SFS-Mobile-Internet-Device/wiki

@bobbybudnick

I tried your script. At first i has issues with sudo evtest throwing an error:

evtest: No such file or directory

So I ran evtest, and my output was:

$ evtest No device specified, trying to scan all of /dev/input/event* Not running as root, no devices may be available. Available devices: /dev/input/event0: Logitech M570 /dev/input/event1: ST LIS3LV02DL Accelerometer /dev/input/event2: Bluetooth 3.0 Wireless Keyboard /dev/input/event3: seeed-tp /dev/input/event5: gpio_keys /dev/input/event7: vc4 /dev/input/event8: vc4 Select the device event number [0-8]:

My touch panel is:

/dev/input/event3: seeed-tp

So with my VERY BASIC knowledge of scripting in general I opened the LINUX_GENERIC_MULTITOUCH file and found:

sudo evtest /dev/input/event13 >

Obviously I needed too change that. SO I did.

Issue #1: Upon Reboot my touch display ID changed, which needed a script change from 3, to now 0.

Then I ran into another issue:

bash: xdotool: command not found

Again my BASIC knowledge told me that xdotool needed installing. So I installed it.

Issue#2:

Upon running RetroArch, pressing one button on the overlay WORKS, but when I press TWO butttons, the mouse cursor jumps all over the screen and the second input I can see is being placed somewhere other than where my other finger is.

It appears that the input needs calibration? Touch is normally well calibrated.

My knowledge stops here.

Also, when running RetroArch, I still see my mouse & can see that its basically just using the input as if I were using a mouse.

Any ideas?

bobbybudnick commented 2 years ago

Sorry it took so long to notice and redevelop but the Linux Generic Multitouch utility has now seen an update.

It was generally working in the old state for screens that had a supported resolution. It is now edited to work with the 5" Waveshare screen at 720x1280 vertical.

There is also a hybrid script which is more portable by not being dependent on exact touchscreen parameters. It is more simple but should send a second button press when tapping anywhere on the screen after holding down a button first. This second button press can be fairly easily edited.

The problem is that the touch coordinates are not the same as the mouse coordinates in all cases. So the touch coordinates need to be converted by "scaling" their value by the actual size of the screen which is how the proposed script works. Of course a script task could just listen for a touch and hold and subsequent tap anywhere on the screen no matter the coordinates which is how the hybrid script works.

The cursor should follow your finger and act generally like a mouse. Without script intervention the cursor should stay with the first touch but subsequent touches are still being picked up by the multitouch layer.

I noticed that Retroarch Android multitouch support was now implemented nearly a decade ago. It would be nice to get official support on this problem.

HelloMrMeseeks commented 2 years ago

Thank you for eventually replying. I am a patient Meseeks but after a while I began to think I had been forgotten or that even YOU didn't care about this problem any more.

Anyway..

You mentioned 720x1280 vertical. However, I have my screen rotated to 1280x720. A more "normal" approach. I am running retroarch without emulationstation and have simply "borrowed" the cores and created my own playlists as RetroArch within Raspbian can't download it's own cores and cannot find it own games without external help (thus the reason emulationstation was created).

However on Windows there is a Playlist Buddy that allows you to specify cores and their locations along with games and their locations. Quite handy.

Additionally doing this, you don't have to worry about rotating the screen as it's already rotated.

I will try this script again when I have some time. 

Thanks again!

MrMeseeks -- Sent with Tutanota, enjoy secure & ad-free emails.

Jun 22, 2022, 07:36 by @.***:

Sorry it took so long to notice and redevelop but the Linux Generic Multitouch utility has now seen an update.

It was generally working in the old state for screens that had a supported resolution. It is now edited to work with the 5" Waveshare screen at 720x1280 vertical.

There is also a hybrid script which is more portable by not being dependent on exact touchscreen parameters. It is more simple but should send a second button press when tapping anywhere on the screen after holding down a button first. This second button press can be fairly easily edited.

The problem is that the touch coordinates are not the same as the mouse coordinates in all cases. So the touch coordinates need to be converted by "scaling" their value by the actual size of the screen which is how the proposed script works. Of course a script task could just listen for a touch and hold and subsequent tap anywhere on the screen no matter the coordinates which is how the hybrid script works.

The cursor should follow your finger and act generally like a mouse. Without script intervention the cursor should stay with the first touch but subsequent touches are still being picked up by the multitouch layer.

I noticed that Retroarch Android multitouch support was now implemented nearly a decade ago. It would be nice to get official support on this problem.

— Reply to this email directly, > view it on GitHub https://github.com/libretro/RetroArch/issues/9223#issuecomment-1163187787> , or > unsubscribe https://github.com/notifications/unsubscribe-auth/AZAPRHKV5JI6VKDYV4TIBLTVQMQIXANCNFSM4IHVI7YA> . You are receiving this because you commented.> Message ID: > <libretro/RetroArch/issues/9223/1163187787> @> github> .> com>