[BUG] When typing with OSK into gnome-terminal, no input is possible if a non-ASCII keyboard is selected

[mike-fabian]

See this video:

https://github.com/mike-fabian/ibus-typing-booster/assets/2330175/f12238dc-985e-454b-bcd5-83078acda056

[mike-fabian]

This is on Gnome Wayland.

The left gnome-terminal shows the debug log of ibus-typing-booster:

tail -F ~/.local/share/ibus-typing-booster/debug.log

The right gnome-terminal has input focus.

ibus-typing-booster has not been selected by the user, it has been automatically started by the on-screen-keyboard OSK.

When one selects English (US, Euro on 5) keyboard layout (it does not matter whether on the cog-wheel on the OSK or in the Gnome panel) and they presses the a key on the OSK, an a appears in the right gnome-terminal.

In the left gnome-terminal one can see the debug output:

2023-10-02 01:13:22,727 hunspell_table.py line 5833 _handle_hotkeys DEBUG: KeyEvent object: 'val=97 code=30 state=0x40000000 name=“a” unicode=“a” msymbol=“a” shift=False lock=False control=
False super=False hyper=False meta=False mod1=False mod2=False mod3=False mod4=False mod5=False button1=False button2=False button3=False button4=False button5=False release=True modifier=T
rue'

I.e. the key value is 97 and the key code is 30, the key state is 0x40000000, the unicode value is a.

When a non-ASCII layout like Russian or Hindi (Bolnagri) is selected, nothing appears in the right gnome-terminal when the a key is pressed on the OSK and the debug output in the left gnome-terminal shows:

2023-10-02 01:15:01,812 hunspell_table.py line 5833 _handle_hotkeys DEBUG: KeyEvent object: 'val=97 code=0 state=0x00000000 name=“a” unicode=“a” msymbol=“a” shift=False lock=False control=F
alse super=False hyper=False meta=False mod1=False mod2=False mod3=False mod4=False mod5=False button1=False button2=False button3=False button4=False button5=False release=False modifier=F
alse'

I.e. the key value is still 97 but now the key code is 0 and the key state is 0x0000000, the unicode value is still a.

The problem is the key code 0. ibus-typing-booster currently passes the key to the gnome-terminal with one of these two ways:

• return False
• forward_key_event(key.val, key.code, key.state)

Both these ways fail when the key code is incorrectly 0.

This is not a problem of ibus-typing-booster because even when ibus-typing-booster is removed from the system and therefore surely not running, the same problem occurs: When selecting a non-ASCII keyboard layout, no input is possible with OSK into the gnome-terminal.

ibus-typing-booster does not cause this problem, but the debug output of ibus-typing-booster shows what is happening.

[mike-fabian]

ibus-typing-booster has already a workaround for a similar issue, see: https://github.com/mike-fabian/ibus-typing-booster/issues/397

But this previous workaround was only helping for keys which caused a commit.

And typing a a into gnome-terminal while OSK is active on Wayland does not cause a commit because ibus-typing-booster can detect on Wayland that the OSK is shown and then disables most of is features to avoid possibly showing passwords in completion buttons when something is typed at a password prompt in the terminal. When most features are disabled like this, the keys are just passed through to the application either using return False or forward_key_event(key.val, key.code, key.state) and this fails.

[mike-fabian]

It is possible to improve the workaround in ibus-typing-booster to handle this situation better at least in most cases by applying a similar workaround also in the cases where no commit happens so far but the key is just passed through.

The special helper function _return_false() in ibus-typing-booster can be improved to do it like this:

• prefer a commit if possible
• if committing is not possible, but forward_key_event() is possible, use forward_key_event() and try to fix key.code in case it is incorrectly set to 0 by OSK (unless unit testing, then the key.code does not matter)
• if committing is not possible and forward_key_event() is is not possible either, fall back to the worst option “return False”.

This will “fix” the problem in most situations because when something like a is typed on the OSK, this can be committed as a Unicode string. And sometimes, if committing directly is not possible, the second step, i.e. trying to fix key.code and then use forward_key_event() with the fixed key.code helps.

There are situations when all this does not help, a commit might not be possible, and trying to fix the key.code might not work either. But for many cases this workaround helps so it is better than nothing.

[mike-fabian]

This is not a problem of ibus-typing-booster because even when ibus-typing-booster is removed from the system and therefore surely not running, the same problem occurs: When selecting a non-ASCII keyboard layout, no input is possible with OSK into the gnome-terminal.

This is not true actually when ibus-typing-booster was not running, it was already working at least sometimes.

Carlos told me

With ibus-typing-booster uninstalled, GNOME Shell does already prefer commit over key emulation, for all state=none|shift keys, so that case should already work.

and indeed it works sometimes.

But quite often it suddenly stops working for me in that situation, I cannot yet find a way to reliably reproduce it. When it suddenly stops working, the only way I could get it working again was by restarting the gnome session.