Improve support for keyboard input on mobile browsers

[curiousdannii]

Mobile browsers (with virtual keyboards) don't support keypress events, and for keydown events set the code to 229 for most keys. But, there is pretty good support through the input event - at least it supports all actual textual input keys plus the backspace key (one of the few keys actually returned in keydown events). It would be good if Emscripten (and then SDL) supported the input event with little/no manual work by the user.

(I just discovered something new: the input event doesn't seem to fire if Chrome dev tools have breakpoints in the event handler. Makes it very hard to test and debug these events! This did not used to be the case.)

What this might involve:

1. Creating an input element, as the input event won't fire on document or window. This should be a <textarea> not an <input> because of this other unfixed Chrome bug.
2. The input event needs to be handled differently, as it has different properties, or you can access what was typed via the textarea's .value. When I've used it (not in an Emscripten project) I also had to empty it each time. And if the character was space, blur then refocus, as otherwise it thinks there are still spaces and backspace doesn't work.
3. Adding something like emscripten_set_keydown_callback_on_thread and then using it in SDL. I don't know if it could be directly used or if it would need to be mapped into something more like a keydown event.
4. Maybe also ignore keydown events which have a code of 229 and don't send them to the wasm? That's what I've previously done, but it may not be necessary.

[sbc100]

Do you know if SDL2 has the same issues? I general I would expect SDL2 support to be more advanced than the SDL1/js code.

[curiousdannii]

I was referring to SDL2, I've never tried SDL1.

[sbc100]

Adding @Daft-Freak.

[curiousdannii]

Well I've got my personal solution working, which can be seen here: https://curiousdannii.github.io/infocom-frotz/zorkzero.html

<textarea id="textinput" autocapitalize="off" rows="1"></textarea>

#textinput {
    position: absolute;
    left: -10000px;
}

const textinput_elem = document.getElementById('textinput')
Module.canvas = document.getElementById('canvas')

// Mobile input event handlers
Module.canvas.addEventListener('touchstart', ev => {
    textinput_elem.focus()
})
textinput_elem.addEventListener('input', ev => {
    const char = ev.data
    if (char) {
        const char_keycode = char.codePointAt(0)
        const upper_key = char.toUpperCase()
        const upper_keycode = upper_key.codePointAt(0)
        const down_up_options = {
            code: 'Key' + upper_key,
            key: char,
            keyCode: upper_keycode,
            which: upper_keycode,
        }
        window.dispatchEvent(new KeyboardEvent('keydown', down_up_options))
        window.dispatchEvent(new KeyboardEvent('keypress', {
            charCode: char_keycode,
            code: 'Key' + upper_key,
            key: char,
            keyCode: char_keycode,
            which: char_keycode,
        }))
        window.dispatchEvent(new KeyboardEvent('keyup', down_up_options))
    }

    // To fully reset we have to clear the value then blur and refocus, otherwise Android will keep trying to do its IME magic, which we don't want.
    textinput_elem.value = ''
    textinput_elem.blur()
    textinput_elem.focus()

    ev.preventDefault()
    ev.stopPropagation()
})
textinput_elem.addEventListener('keydown', ev => {
    if (ev.which === 8) {
        const options = {
            code: 'Backspace',
            key: 'Backspace',
            keyCode: 8,
            which: 8,
        }
        window.dispatchEvent(new KeyboardEvent('keydown', options))
        window.dispatchEvent(new KeyboardEvent('keyup', options))
        ev.preventDefault()
        ev.stopPropagation()
    }
    if (ev.which === 13) {
        const options = {
            charCode: 13,
            code: 'Enter',
            key: 'Enter',
            keyCode: 13,
            which: 13,
        }
        window.dispatchEvent(new KeyboardEvent('keydown', options))
        window.dispatchEvent(new KeyboardEvent('keypress', options))
        window.dispatchEvent(new KeyboardEvent('keyup', options))
        ev.preventDefault()
        ev.stopPropagation()
    }
    if (ev.which === 229) {
        ev.preventDefault()
        ev.stopPropagation()
    }
})
textinput_elem.addEventListener('keyup', ev => {
    if (ev.which === 8 || ev.which === 13 || ev.which === 229) {
        ev.preventDefault()
        ev.stopPropagation()
    }
})

Basically when an input event comes in, I generate fake keydown, keypress, and keyup events, and send them to the window. keydown events for backspace and enter do contain the correct which value, however their other properties were not complete enough for my SDL-using app to understand them (the event handler does send 14 properties through to the wasm, so some users may find what it does adequate). So I also ended up making fake keydown and keyup events too.

[ericoporto]

@sbc100 I think some support has to be added here in Emscripten (even if only for the upcoming virtual keyboard API), otherwise I don't see how this could work once the canvas gets in Fullscreen.

@curiousdannii have you tested your game linked above on chrome in iOS (the American chrome that is still Safari).

[curiousdannii]

Not specifically in iOS Chrome, just Safari. Though there's currently a bug that the keyboard won't stay up.