Android soft keyboard stuck in show/hide loop whenever text input is possible

[chris-r-williams]

For example: in City of Secrets, the keyboard will constantly show hide during the title screen and first room, but when you go north and are presented with two clickable options instead of text input, the keyboard remains hidden. This makes games near impossible to play.

My guess is that there's some JavaScript changing the cursor position, which focuses the input, and triggers a keyboard show event. Then it loses focus somehow and triggers a keyboard hide event. This guess is based on Android Studio debug logs like these:

08-06 12:43:29.631 1605-1605/? I/LatinIME: Starting input. Cursor position = -1,-1 08-06 12:43:29.667 1605-1605/? I/LatinIME: Starting input. Cursor position = 0,0

[erkyrath]

This has been noted before (see also https://github.com/erkyrath/glkote/issues/7 , https://github.com/erkyrath/glkote/issues/25 ). I would love some help on figuring out what to do about it. I do not have Android devices to test on.

[chris-r-williams]

Android Studio provides an excellent emulator, if you'e interested in debugging, yourself. I will try to look into it myself, as time permits.

[WakeRealityDev]

I'm looking into the issue today with the intention of finding the problem code. Posting here to coordinate efforts / duplicate attempts.

[chris-r-williams]

I got it set up for debugging, but haven't had time to start, yet. Will probably have time this weekend if you haven't found it yet.

FWIW I was going to start by investigating all the focus and blur code in glkote.js

[WakeRealityDev]

Ok, thank you. I posted a discussion topic specific to the issue and test case of City of Secrets: http://www.intfiction.org/forum/viewtopic.php?f=38&t=22938

[WakeRealityDev]

I think the problem is more fundamental than just focus issues. A normal Chrome web page that you use, like the one we are currently reading on GitHub - doesn't get re-scaled and re-sized just because the keyboard pops up. Normally a vertical scroll bar is there to account for the vertical space, like you scrolled the page up or down - the page content isn't changing at all when the Android soft keyboard appears and disappears.

On a 10" tablet, if the keyboard is off, the Purple/Yellow/White opening image of City of Secrets is nearly the full screen height on Android Chrome. However, when I activate the keyboard - instead of a vertical scroll bar appearing - it reacts by doing a major repaint / redraw / graphics scale of the entire screen to now where the Purple/Yellow opening image of City of Secrets is only about 4" of my 10" Android tablet.

No interaction with Glulx is taking place here, the VM is still waiting on the Glk char INPUT for the "PRESS ANY KEY TO BEGIN" prompt.

The story Dead Cities by John Ingold demonstrates this resize issue even more, and it won't allow me to type at all into the main Glk TextBuffer window because it insists on resizing the Glk windows every time the keyboard pops up/disappears.

Also note that with a hard keyboard: USB, Bluetooth, or the emualtor one - this problem goes away - as GlkOte isn't thinking the window resized each time the keyboard is unplugged and re-plugged (imagine a USB keyboard you remove and insert, that's kind of what the soft keyboard is doing when you hide and show it).

[curiousdannii]

You wrote at the intfiction.org forum:

So far, I think a key factor is that when the keyboard pops up it resizes the window / HTML5 canvas area, and GlkOte is scaling the entire Glk window down so it fits on the window above the keyboard.. that's triggering these calls to accept_inputset()/accept_inputcancel() in the repaint... then the keyboard goes away, causing the window size to increase, another repaint, and it's looping. That's a key behavior change from desktop Chrome - when you use desktop Chrome - going in and out with the keyboard (changing to another app) is not going to cause the Chrome browser window to resize and a major repaint of the entire output.

I think accept_inputset is the major cause of the problem. Now the <input> element is never detached from the DOM (in line input mode), but it is forcibly appended even if it is already in the right position in the DOM. My first recommendation of what to try would be to make the append call conditional on whether the element was already present. Perhaps by not setting win.inputel until the end of the function so that it can be tested for null.

Note also that the <input> is absolutely positioned, and possibly that is also a problem. I see a comment before the absolute positioning saying /* ### opera absolute positioning failure? */, but from the commit when it was introduced, it doesn't mean that the code is because of Opera, but that Opera might not handle it well. Probably that's no longer an issue now that Opera uses Blink. The irony that you changed this Opera code one month before Blink was announced!

Do you remember @erkyrath why it needs to be absolutely positioned? Is it to do with character/key input? If so I would recommend that we completely separate the two keyboard input types in code. I think we need to completely redo key input with more modern events. (What those are I'm not sure yet.) Seeing as char input doesn't work in Mobile Chrome anyway it would be safe to test whether commenting those lines out fixes this issue.

In char input the <input> element is detached. If this is all because of the cursel, maybe it's too clever for its own good?

[WakeRealityDev]

It could be what we are seeing is this:

1. User puts focus on input field, Soft keyboard is activated, which triggers a HTML5 canvas / screen window resize to smaller in GlkOte
2. The input is removed from the DOM - so by the time the keyboard pops up, the input it was there for is now gone, so the keyboard automatically closes by Android. Now the canvas is back to full size and GlkOte is doing another resize/scale change to larger.
3. The new input is added to the DOM. This triggers the keyboard to popup again.

[curiousdannii]

Yep, it's in a race condition, question is why. My guess is the append calls.

[erkyrath]

The input line is absolutely positioned, I think, because I didn't want the page size to change when the <input> tag was being deleted and replaced.

The absolute positioning is used regardless of whether we're waiting for line or character input. There's no difference in how the DOM is set up for the two events, except that it's legal to print stuff during character input. (That's what line 1113 is for.)

My first recommendation of what to try would be to make the append call conditional on whether the element was already present.

If stuff has been printed, then the <input> needs to be added (if not present) or moved to the bottom (if present). You're talking about the narrower case where it's present but does not need to be moved, because it's already at the bottom. Detecting this should be possible; I'll take a look this weekend.

[curiousdannii]

If stuff has been printed, then the needs to be added (if not present) or moved to the bottom (if present). You're talking about the narrower case where it's present but does not need to be moved, because it's already at the bottom. Detecting this should be possible; I'll take a look this weekend.

Right, but text can only be added for character input, and for char input the <input> doesn't have to be in the DOM where it is because the player's input will be gone as soon as it's entered. Making it fixed instead of absolute might work. (Although I think I remember that fixed didn't work in some mobile browsers.) I presume your desire is to have the browser not scroll to a silly part of the page? If it was positioned offscreen (perhaps with the negative margin trick) could that work?

In order to fix the character input for mobile browsers I think we'll need to switch to the input or beforeinput event, so a complete overhaul could be worth it.

[erkyrath]

Am I right in thinking that in the CoS case, the game is accepting character input repeatedly without printing to the same window? I think covering that would handle a large chunk of the real-world problem.

[WakeRealityDev]

@erkyrath - tracing it with RemGlk, I only see one generation that is waiting for a "press any key" prompt. In contrast, same author has a nearly identical layout at the opening of the story Best of Three: http://ifdb.tads.org/viewgame?id=t1egxcvjz5pcm0xq - and it does a hyperfast Glk timer to do the same prompt. I can see it on Quixe on my desktop Chrome blinking the cursor at ludicrous speed.

[curiousdannii]

The problem is present with line input too. That's the main thing which needs fixing.

[curiousdannii]

I've submitted a pull request which fixes the show/hide loop in CoS. If we can get that merged (and then into Quixe) we can see what other issues remain.

[WakeRealityDev]

Confirm @curiousdannii fix works on my testing. When I touch to retract the keyboard it no longer reopens by itself. Thank you.

[WakeRealityDev]

I need some help, confirmation: it seems I might have overlooked a more serious problem that is happening with City of Secrets on Android with this opening screen; Chrome browser / Android 7 (LG tablet, and Google emulator). Does the soft keyboard even work on the "PRESS ANY KEY TO BEGIN" prompt? I've been mostly testing with both soft and hard keyboard at the same time - and I actually tried the soft keyboard spacebar. Android gives audio feedback at the soft key press, and I see the HTML input field advances one space - but the story itself does not continue! Soft keyboard backspace even seems to work, removing the space added after "BEGIN". Pressing USB/Bluetooth/Emulator hard keyboard spacebar works fine.

This is with or without @curiousdannii fix from 14 hours ago.

[curiousdannii]

It would probably be best to post that as a separate issue.

[WakeRealityDev]

I am outright begging someone takes the time to confirm the issue. As I said 6 days ago: "I need some help, confirmation" - it was a plea for HELP!

[nebocamin]

executive summary: show/hide loop is gone.

I can confirm, that the "endless open-close-keyboard loop" is gone after applying that patch. But there are some other issues too, I am investigating. The scroll behaivour is different on mobile systems. On Firefox OS (I know, its dead now) the patched Version is working perfectly. Keyboard stays activated and doesn't move. On Chromium and Firefox for Android, its sometimes "flashing". If the text is longer than the visible area it gets hidden and visible again and the user has to scroll a bit to find her last position. I made, because that would be my goal, a "progressive web app" so the web page is pinned to the homescreen and behaving like an app. The patched version is working without the show/hide loop. But there is another issue. the cursor is on the last line of the full screen, means the activated keyboard will be over the text. scrolling seems fine, but the cursor position stays under the keyboard. manually deactivating the keyboard shows it. i can show you my test cases and can hopefully help so sort the issues out.