Characters produced do not match swipes entered

simon-dima commented 3 months ago

Context

Flickboard functions normally most of the time but sporadically enters a state in which the characters sent by the keyboard do not match the swipes and taps entered.

Steps to reproduce

I have not yet determined specific starting or ending conditions for the anomalous state; it occurs and disappears seemingly at random.

Here is a screen recording of an instance of the bug: https://github.com/user-attachments/assets/8441dd9c-7521-4ac1-8d24-470981271877

Expected behavior

The characters produced should match the taps and swipes entered. The entered text in the example starts with "He named the nameless hills and dells".

Actual behavior

The characters produced do not match the taps and swipes entered, producing jumbled text. The produced text in the example starts with "hhe ne. the ne.ees h=lss al".

Android version

Android 14 (LineageOS 21)

Device

Fairphone 3

Other details

While in the bugged state, the keyboard logic seems to follow a state machine similar to this: 0: Upon a tap, emit the character tapped and proceed to 1. Upon a swipe, emit the corresponding character and proceed to 0. 1: Upon a tap, emit the last character tapped and proceed to 1. Upon a swipe, emit the character which is in the corresponding direction from the last key tapped (or no character if the last key typed does not have a character in that position), then proceed to 0.

I have observed this bug while entering text into multiple different apps.

Acknowledgements

[X] I have written a short but informative title.
[X] I have searched the existing issues and this is a new one, NOT a duplicate or related to another open issue.
[X] I have filled out all of the requested information in this form.

nightkr commented 3 months ago

Hey, this definitely seems bad! Sadly I haven't seen anything resembling this myself, it's hard to debug without being able to repro it..

Would you mind turning on the "Highlight taken actions" and recording a new video the next time you manage to repro the issue? It would be helpful in narrowing down where things are going wrong.

simon-dima commented 3 months ago

Of course, here it is! https://github.com/user-attachments/assets/3ea0f72d-6ee3-4053-8176-59e4169903f7 It is probably relevant that the highlighted actions reflect the characters produced, and the Android pointer location overlay and Flickboard pointer trace match the position of my fingers.

Please let me know if there is any more debug information I can gather!

nightkr commented 3 months ago

Ah! So it is getting registered as the "correct" gesture, just to the wrong key.

Which version of FB are you using? Are you using an official build?
Please post a screenshot of your workarounds page in the settings.

simon-dima commented 3 months ago

I am using version 0.1.7, installed from the IzzyOnDroid repository! However this bug is not exclusive to this version, I have seen it ever since installing FB for the first time in the spring. My workarounds settings are all default as far as I can tell.

nightkr commented 2 months ago

Okay, so I can see a few possibilities here that could theoretically cause it to enter a mode like this:

The finger-up event gets eaten by the jump threshold workaround threshold (unlikely since you don't use it, and even then I can't repro it on my device)
The finger-up event gets eaten somehow by the hold timeout (this would mean doing a swipe but holding the finger for a second or so before releasing)
The finger-up event gets eaten by hiding the keyboard (so holding down a key while pressing the "hide keyboard button"

However, I can't seem to repro it w/ any of these methods… There might also be some device specific screwery?

nightkr / flickboard