simultaneous button presses don't produce the expected behavior

[Googer]

Heh, I've noticed this too but had always just passed it off as my own incompetence and / or the controllers in question being very finicky about simultaneous button presses (for instance I pretty much can't ever do a back-attack in Golden Axe or SoR on the Genesis / MD or throw a weapon in SoR since these are all multi-button tasks in each respective game). Glad to know it's probably not just me! :p

[theRetroGuy]

I'm having the same problem on my Retroarch android. I was playing the PS version of tekken 3 and I've noticed that grab didn't work. Went into the practice mode and I was able to find that simultaneous button press just doesn't work. (In practice mode you can see what buttons you've pressed) It only works about one out of twenty times. It seems that the input window for the simultaneous press is very short. When using epsxe to play tekken 3 the grab works fine. So it seems to be a problem of retroarch input. This is the hardware I'm using.

LG Vu III (F300) Dual Shock 3

[inactive123]

Are you using a real gamepad or just touch?

[theRetroGuy]

I'm using a real gamepad. The sony dual shock 3. The opener of this thread also seems to be using real gamepads.

[theRetroGuy]

I recommend using the PS1 version of Tekken 3 for testing. You can check your input in the practice mode.

[MegaMissingno]

Same problem here, trying Final Burn Alpha with controller plugged into USB. Can't grab, overhead, taunt, or do EX moves in Street Fighter III: Third Strike, and can't roll, blowback, max, etc in King of Fighters.

[rsn8887]

Could this be related to this commit? 0ba45d457ef62e82f79e97d2be76e3cc56b47fea

[SwedishGojira]

Also suffering from this problem. Using Xiaomi Bluetooth game controller on my M8S Android box. No problem using the same controller in Windows or Linux version of retroarch, or other native Android apps.

[rsn8887]

I confirm this issue using Nvidia Shield Portable with Kitkat. The bug is present in all cores and makes most emulators pretty much unusable on Android, especially with fighting games. The bug prevents the system from recognizing two button presses at the "same time," Golden Axe Genesis A+B is a good test. I can certainly hold jump and then press attack and that gives the expected "slash while jumping" action, but the overhead slash that happens on simultaneous down press of A and B does not work.

Maybe the Android polling code has a slight delay between recognizing button 1 and button 2 "down" action, maybe there is one frame lag or something.

The code is here: https://github.com/libretro/RetroArch/blob/master/input/drivers/android_input.c

I think the bug is probably in the functions android_input_poll_event_type_key, android_input_poll, or android_input_poll_input

But I cannot see anything wrong immediately, unless it is a matter of delay caused by the processing of all the different types of events. In the older versions, only input events where processed in the pollAll loop, now it is a bunch of events with different types.

[Brianvgplayer]

I have this issue with most cores, but simultaneous inputs work on mGBA, FCEUmm, Nestopia, and Gambatte. I tested with 8bitdo Zero and a logitech keyboard on a Samsung Galaxy Core Pro.

TMNT III behavior is actually correct. Despite what the manual says, just pressing A+B doesn't do the move. The move is done by pressing jump while attacking and then pressing A+B. I was also confused by TMNT II until I tried the cart on my toaster NES. The A+B slash only works after attacking or landing from a jump. The odd thing is that the JP version of TMNT The Arcade Game has the input as just A+B. Final Fight GBA is a similar oddity. Unlike every other version of Final Fight, the special move can only be done by holding one button and pressing the other.

I tested Double Dragon II, River City Ransom, and Mighty Final Fight and they all recognize simultaneous button presses. I also tried the GB Double Dragon, King of Fighters EX2 GBA, and River City Ransom EX GBA. I was able to do the jump in Double Dragon and RCR and summon strikers in KoF EX2.

I also noticed that simultaneous inputs don't work with some of the overlays, though they work with most of the flat overlays.

[Brianvgplayer]

Anyway, my point is that some cores do recognize simultaneous button presses and that TMNT III works fine in Retroarch. I heard that it isn't a core issue, but I thought it was important to mention that the inputs do work on some cores.

[Brianvgplayer]

I'm not trying to argue about it being core related. I just thought it would be helpful to mention that some cores do recognize simultaneous inputs. I want this issue to be solved too, especially with the Genesis Plus EX core, which seems to be the only emulator on Android that properly emulates Sega CD.

[Brianvgplayer]

Thanks for the info. I also had the same problems with FBA and SNES 9x. I had to do the hold and press method for Final Fight and only got Batman's cape swipe 1 out of 10 tries (or something along those lines) in Batman Returns SNES. Couldn't do the backwards slash in GA1 Genesis or arcade or the backwards jump attack in Golden Axe II using the Genesis Plus GX core for the Genesis games either. Thanks for the input for TMNTIII too. It's much easier pulling that move off now and the instruction manual doesn't make it clear how to do it.

[Brianvgplayer]

I noticed something really odd when playing Tekken 2 PSX in practice mode on RetroArch. When buttons are pressed simultaneously, it registers another button before registering the buttons pressed.

[Brianvgplayer]

On android. It's like the latter. I can't do throws or some other moves consistently. I used the 8bitdo zero, but I also tried the touch controls and it seems the only way to do simultaneous presses consistently with the touch buttons is pressing between the buttons on the flat overlays. The touch control behavior seems to be consistent will all cores. My device also supports multi-touch.

The odd thing is that I can do strikers in KOF99 and other simultaneous moves consistently in some 2D games with the same core (PCSX Rearmed), though it's definitely not core related.

edit: Looks that that odd behavior happens on the Windows emulators and on PS1 when the buttons are pressed right after each other. So it's basically how Tekken acts when the buttons aren't pressed at the same time.

[Brianvgplayer]

Looks like the input issue is still there as of the April 7th nightly. Still not able to do simultaneous button presses in Golden Axe. I noticed that some outside emulators like classicboy and MAME4droid lack the button press issue.

I know it's not the cores causing the issue, but I'm still under the impression that some cores are affected differently. I'm able to do simultaneous button presses consistently in some NES, GBC, and GBA cores and input response seemed better than some outside emulators.

[blackman91]

Has anyone tried changing the joypad driver to hid? Or messing with the input settings too see if it makes any difference? Man this bug is major I want to use retroarch on Android as my only app for emulators but I am forced to use standalone android emus because of this bug.

[Brianvgplayer]

The controller doesn't work at all in Retroarch when I switch to hid. I get the same behavior with Moga Hero Power and 8bitdo Zero. The former is definitely a hid controller.

I read on the forums that simultaneous inputs work on overlays, but that is not my experience. I only got them working with the in between presses on the flat overlays, but simulateous presses still don't work with the buttons themselves. My Android does support multitouch. It's limited, but I was still able to to simultaenous presses with two buttons in other emulators.

[blackman91]

Apparently only external controllers have this issue, touch input works perfectly for some reason. Sucks for users with tablets with builtin controllers like me.

[Brianvgplayer]

No, it's definitely the touch input that's not working for me, as well as external input. I tested multiple times in Retroarch. I tried multiple overlays, including the flat overlays. Only the flat ones work, but only when pressed between the buttons, not with the buttons themsevles. From what I read, it sounds like the experience is different on different devices. As I said, my device has limited multitouch, so that might be part of it, but multiple inputs on the touch overlays do work in other emulators like MAME4Droid and a few of the .emu emulators.

[andres-asm]

The issue is as follows:

On every frame the input state is cleared, so if the inputs aren't performed exactly at the same frame it won't register (unless the game has a longer windows for multiple inputs)

The only reason it works on some overlays is that some overlays have combo buttons. Most of the comments here are wild theories and misinformation, the issue is well known and documented, but noone has really looked for a fix.

[blackman91]

Really? Because I have tested almost all gamepad overlays on retroarch 1.3.2 and the simultaneous presses work perfectly without using the combo buttons, a lot of people have reported the same in the forums: http://libretro.com/forums/showthread.php?t=5116&page=2 I am surprised noone has looked for a fix for such a major bug that affects all external controllers, it makes a lot of games unplayable.

[Brianvgplayer]

I'm sorry if I came off the wrong way, but that was a bit harsh to mention "wild theories and misinformation". Some of my posts were in reply to what was said on the Retroarch forum, which I would be replying to if I got a confirmation email. The first post in this topic didn't make it clear that you knew the input of TMNT III wasn't just A+B or that the cause of the problem was known. Posts on the forum do make it clear some devices are affected differently. One poster said the problem was fixed when he switched to a different android UI (forgot exactly what it was called).

[blackman91]

Use a gmail account to register, outlook and others do not work.

[Brianvgplayer]

Will do. Thanks.

[meepingsnesroms]

How would you push them frame perfect even on a console?

[andres-asm]

@Brianvgplayer didn't mean to be harsh either sorry, but yeah there are some wild theories, and more precisely, this issue is about physical gamepads. Anyway, I hope I can get maister to look at this the next time he shows up, he had some theory. This issue actually bugs me a lot, and not being able to fix it bugs me even more, and then I get lots of notifications about it and I get bugged a lot more... that's it

[blackman91]

Hoping maister can find a solution. Does anyone know what version of retroarch doesn't have this bug?

[meepingsnesroms]

Both Mame4droid and MD.emu use passive input handling in java with onKey() or onKeyDown() and onKeyUp(),where as retroarch polls an event queue.

Mame input code MD.emu input code

Did retroarch ever do this(possibly before the android ui was replaced with glui),because if it did that is probably the last version that worked.

I looked at the input code for retroarch and it is reading from an event queue and alot of work is being done for every keypress,if that work takes 3/4 of a frame if you push 2 keys at the same time they will be put in the queue at the same time but it takes 3/4 of a frame for each so one will be registered on the next frame unless you push them half way through a frame.

This explains why some devices aren't affected(there faster) or why kernel parameters(as stated in the forum) would affect it.

[blackman91]

That is good info, so why would the simultaneous button input work using the gamepad overlays? Does retroarch handle that input differently?

[meepingsnesroms]

I don't know why but most likely it is because the overlay code is triggered after all the events are processed.Compared to the button input that |= the button in as soon as it gets the event.(I haven't looked at the overlay handler that much so don't hold me to this.)

So with the the button input the game could read the pressed buttons before all the events are finished but the touch input updates all the buttons at once and is dependent on there being no events in the queue(to have the newest touch data).

My advice is to log all the changes by the events and apply them all at once when the queue is empty since the very nature of a queue is that 2 things cant have the same spot even if they happened at the same time.

Also here are some optimizations to the key input.

while (AInputQueue_hasEvents(android_app->inputQueue)) //the second performs both functions { while (AInputQueue_getEvent(android_app->inputQueue, &event) >= 0) { ... } }

In android_input_poll_event_type_key() the volume keys will act as volume keys even when mapped and if they cant be mapped you should check before putting them in the keymap.

[andres-asm]

I don't really understand the input code but as far as I understand the whole button handling happens at L986 of android_input.c, which leads to

android_input_poll_event_type_key There is a comment in that function:

`/* some controllers send both the up and down events at once

• when the button is released for "special" buttons, like menu buttons
• work around that by only using down events for meta keys (which get
• cleared every poll anyway) */`

The keypress or release is set here:

   switch (action)
   {
      case AKEY_EVENT_ACTION_UP:
         BIT_CLEAR(buf, keycode);
         break;
      case AKEY_EVENT_ACTION_DOWN:
         BIT_SET(buf, keycode);
         break;
   }

But my guess is that the buffer is empty every time. Sadly debugging on android is a PITA, it should be easy enough to print the values that android_keyboard_state_get(port) returns.

[andres-asm]

debugged a bit but I don't really understand what's going on. if I hold a button, for example B

BIT_SET(buf, keycode_for_b); is called endlessly, shouldn't be the case, that event should have been handled by then and it shouldn't keep setting that bit.

If I then press and hold another button, for example... Y then it calls BIT_SET(buf, keycode_for_y) once, and stops spamming BIT_SET(buf, keycode_for_b).

If I press both buttons once at the same time it would seem it sets the two bits on a single frame but I'm not sure of it I didn't log the frame number. I think it may be the case, maybe some variable is resetting on every call or something.

[meepingsnesroms]

The constant repeating is probably because you need to call finishEvent regardless of predispatched status.

     if (!predispatched) //this if should be removed
        AInputQueue_finishEvent(android_app->inputQueue, event,handled);

You can remove the predispatching altogether,all it does is send the keys to the active ime(that retroarch doesn't even use).

[andres-asm]

I thought so too, and I added the finish event line, that's the first thing I tried, but it caused segfaults so I reverted that

On Mon, Apr 25, 2016 at 10:56 AM, meepingsnesroms notifications@github.com wrote:

The constant repeating is probably because you need to call finishEvent regardless of predispatched status.

 if (!predispatched) //this if should be removed
    AInputQueue_finishEvent(android_app->inputQueue, event,handled);

You can remove the predispatching altogether,all it does is send the keys to the active ime(that retroarch doesn't even use).

— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/libretro/RetroArch/issues/2043#issuecomment-214414824

[meepingsnesroms]

Can you give me the details on the segfaults.(I am trying to compile the source myself to try to work on it but haven't gotten it compiling yet.(I don't need help,just time))

[andres-asm]

actually we have a staging branch that you can toy with to test stuff, just send a PR to that branch, I can merge and a few minutes (or several minutes depending on the buildbot workload) a build will be uploaded. IIRC this was the build with the segfault http://bot.libretro.com/nightly/android/2016-04-24-RetroArch_devel.apk

[meepingsnesroms]

I installed that build and the issue is not fixed but there where no segfaults.(tested with golden axe for genesis)

[andres-asm]

maybe it wasn't that one then idk, I don't keep builds per-commit so it could have been overwritten

[diablodiab]

(I originally posted this in another thread, so apologies if I repeat what some of you already mention earlier)

I think the problem occurs because multiple buttons need to be registered within a single frame. When looking at runlook_iterate in runloop.c, I'm guessing that button presses are registered by comparing changes to the input states in the current frame compared to the input state in the previous frame (cmd.state[0] and cmd.state[1]). So in order to register multiple buttons pressed at once, they would have to both have changed state during that single frame.

I think the problem might occur because input events are handled faster on Android so they don't have time to queue up and be handled as a bundle but are instead handled individually in separate frames.

I tried experimenting with timers in android_input_poll in android_input.c to try to force input events to only process them every 100ms or so, so there would be time for them to queue up before they are handled, but it didn't work out the way I had hoped.

Any thoughts on this?

[rsn8887]

That sounds like how it should be. As long as the runloop checks the button state every frame and does not skip a frame. It would help to log the button bits together with the current frame number once every frame to check the exact timing.

I think fr500 must be onto something though because he mentioned the inconsistent spamming of the bit_set function that does not happen when two buttons are pressed simultaneously.

[diablodiab]

Well, if the runloop checks for button presses by comparing the button state changes between frames, wouldn't you have to press both buttons within the time of one frame change in order to have it register as a simultaneous button press?

Eg. at 60 fps, you would have to press both buttons within 17ms for both of them to register in the same loop?

[Googer]

Actually on average I think it'd have to be within 8 ms of each other (half a frame) even, since, in the worst case possible case, you can have one button press just before the poll loop and one just after (i.e., <<1 ms of each other) and be registered by RA as occurring on separate frames instead. AFAIK, none of this explains why this is only an Android issue though - don't all the input cores work in pretty much the same manner?

[diablodiab]

I've found it difficult to get a good overview of the input drivers, but from what I can tell, the Android input driver is implemented mainly as a universal keyboard driver and doesn't really follow the structure of the other drivers.

This also made it tricky when I was extending the Android driver with keyboard functionality, because the normal keyboard structures were already in use but had been twisted into handling gamepads.

For example:

• the gamepad polling should be placed in android_joypad.c
• gamepad button states should be maintained in android_joypad.c but right now they are maintained in keyboard_event_android.c in a variable that would normally be used for keeping track of keyboard state, but which has now been changed into an array of 8 keyboard states to keep track of gamepad states instead

And I can go on :)

The right step would probably be to do a complete overhaul of the Android input driver so that it respects the same structure that the other input drivers do. I worked a bit on doing this when I was implementing the keyboard functionality, but it would be such a large change that I was worried I might break a lot of stuff so I dropped it.

TLDR: The Android input driver is very different from the other input drivers.

[andres-asm]

Yeah @twinaphex has mentioned that there's a need to rewrite that driver (actually several) but as you said it's... well difficult.

Actually there is one thing on the android input driver that I like better than in any other driver, and that is that gamepads are bound on a button press.

Example, connect DS4, does nothing, press X, it gets bound to port 1, connect another, does nothing, press TRIANGLE, gets bound to port 2.

Sadly as you said, it's a big project. Remember there wasn't native gamepad support on android till relatively recently, and any changes we do should be tested on a broad variety of devices of all ages, sizes and running different android versions...

And on top of that, you have to go through JNI... It's really a pita.

I think @Googer's reasoning could be accurate, I think all JNI code is called in another thread? (I might be wrong), maybe the eventqueue is filling/emptying faster than we're polling?

Anyway

[meepingsnesroms]

I successfully removed all predispatch code but the bug was not fixed. (the segfault was likely a use after free because the event was freed then the ime used the old pointer given by the predispatch) I got everything compiling so now i can try to fix it.

[diablodiab]

If I remember correctly there was a problem with keyboard input being registered twice when I didn't check for !predispatch. You can check for this by starting a game in ScummVM that uses keyboard input, eg. the original version of Kings Quest 1.

Could you check if this bug has been reintroduced after you removed the predispatch code?

[meepingsnesroms]

I did not see it,I tested it with dosbox dos terminal and usb otg keyboard.

[diablodiab]

I've implemented a proof of concept solution/workaround in my fork: https://github.com/diablodiab/RetroArch/commit/769c12717449da4d0308237fe340851ec0b89048

You can download a compiled .apk here: https://www.dropbox.com/s/41t3lbjfmk2onfo/retroarch-release.apk?dl=0

The basic idea is this: Button input is registered into a temporary input state which is flushed to the normal input state every ~67ms. This provides a small window for pressing more than one button which are then sent together to the normal input state within a single frame.

I've tested it in Golden Axe and it does seem to work.

I'm sure it can be implemented in a better way, so I'm referring to it as a proof of concept of the thoughts i had on queueing inputs and flushing them to the input state at a certain frequency rather than immediately.

What do you think of this approach?

[meepingsnesroms]

I was just working on doing this! It is perfectly fine,we might just be polling too fast for android to keep up.