bug(web): Keyboard completely frozen

[ermshiperete]

I managed to get in a state where the Keyman keyboard doesn't react to any taps. Predictive text still works and I can select suggestions there, but the keyboard itself ignores everything. Switching to a different app doesn't help, it's still frozen there.

Typing in the Keyman app though still works.

Keyman 17.0.301-beta Android 12

https://github.com/keymanapp/keyman/assets/181336/55f16f6f-9bd3-4d7e-88d0-024b2945ccd5

[darcywong00]

Do you have steps for repro?

[ermshiperete]

Unfortunately not.

[mcdurdin]

I have also experienced this twice now. Can we enable this in the beta please and then I will try and dig around in the webview next time it happens?

if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.KITKAT) {
    WebView.setWebContentsDebuggingEnabled(true);
}

As this is new in 17.0, my guess is that we are falling in the gap in a gestures promise somewhere, @jahorton what do you think?

[jahorton]

As this is new in 17.0, my guess is that we are falling in the gap in a gestures promise somewhere, @jahorton what do you think?

Sounds like it. And, this is after the rapid-typing fixes have landed - that was in 17.0.301 - so it's definitely worth looking at.

[jahorton]

Also, is this after having recently typed a lot, or does it happen pretty quickly into use when it does occur? Any particular gesture type used recently?

I think I've noticed something kind of similar before... after using breakpoints while investigating something else. I always figured it was due to that, though, until seeing this issue.

[jahorton]

Also, out of curiosity... had there been any recent error reports displayed by the keyboard? Sentry does have a few errors associated with 17.0.301-beta, all of them related to #11165 (when reported under 'web').

Granted, the handling on that should prevent the error from blocking future input, but... it's still best to check, I figure.

[ermshiperete]

Also, is this after having recently typed a lot, or does it happen pretty quickly into use when it does occur? Any particular gesture type used recently?

I didn't type particularly much, but I don't remember particulars since I was using my phone and not specifically testing things. If I used a gesture then it was South.

Breakpoints were definitely not involved.

I don't remember seeing any error displayed by the keyboard prior to this freeze. And a search on Sentry with device:RMX2086 doesn't show anything on that day either.

[jahorton]

I have also experienced this twice now. Can we enable this in the beta please and then I will try and dig around in the webview next time it happens?

if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.KITKAT) {
    WebView.setWebContentsDebuggingEnabled(true);
}

On this:

Investigation avenue 1: check keyman.osk.vkbd.gestureEngine.touchEngine.eventDispatcher.queue whenever this occurs. When the engine is at a "resting" state, having processed all events, the .queue should always be length zero.

I've been inspecting this section as a potential cause and don't see it being viable - the logic looks rock-solid. That said, it's best to verify the assumptions, and this one is "easy enough" to validate.

Investigation avenue 2: keyman.osk.vkbd.gestureEngine._activeSources and ._activeGestures should be empty in the same scenarios. If there is something in either, that'll be quite informative.

Investigation avenue 3: keyman.osk.vkbd.gestureEngine.currentSelector - getting a nice JSON.stringify() of that would be useful.

[mcdurdin]

Yes, I don't recall any specific gesture used, nor any specific sequence of events. Will wait for the beta build which has debug enabled and work from that for next time it happens. It's rare, so hopefully I will be close to a debugger

[jahorton]

Possible relation to #11221 - part of longpresses involves blocking other input while the subkey menu is viewed. If something were to go wrong with that mechanism, there's a chance it could trigger this issue. It'd certainly help to have insight into the underlying, low-level trigger, naturally - and I expect my previous comment's instructions should be able to give solid leads.

[jahorton]

We haven't seen this since the recent rapid-typing fixes, so we're going ahead and closing this. We can always open a new issue if we get new reports.

[jahorton]

So, fun story: something I did with #11306 actually just shed light on the underlying cause!

First things first: it requires an error handling due to other reasons during a specific point in the gesture lifecycle. It requires that a JS exception trigger during gesture-model spin-up for a newly-started touchpath.

In this case, because this does happen synchronously during the process, it prevents the touch engine's order-preserving event queue from processing further. Refer to https://github.com/keymanapp/keyman/pull/11306#issuecomment-2084119061. When the 'flick-start' model's startup crashed due to a JS error, this resulted in a locked keyboard.