Dead key inputs not working properly on Ubuntu

LinqLover commented 2 months ago

Ubuntu 20 or 22 (tested on both), Qwertz keyboard layout, trying to enter different characters into a Workspace using dead keys:

^ (circumflex) + ^ (circumflex): ^ (as expected)
^ (circumflex) + Space: expected ^, got ̂ (space + U+0302 COMBINING CIRCUMFLEX ACCENT, rendered as question mark in Squeak)
´ (acute accent) + ´ ´ (acute accent): expected ´, got ' (single apostroph, depending on your editor preferences, this character might be duplicated automatically)
´ (acute accent) + Space: expected ', got ́ (space + U+0301 COMBINING ACUTE ACCENT, rendered as question mark in Squeak)
` (grave accent) + ` (grave accent): ` (as expected)
` (grave accent) + Space: expected ', got ̀ (space + U+0300 COMBINING GRAVE ACCENT, rendered as question mark in Squeak)

When typing the "mixed dead keys" (ending in Space) into a keyboard exerciser morph, an error is raised because the combining accent keycodes are not known by EventSensor class>>#virtualKeyTable.

The expected inputs are what most other applications such as the gnome desktop environment or Chromium on the same machine behave like.

I don't know enough about this domain to even say whether this should be fixed in the image or the VM. Probably other keyboard layouts are affected as well.

LinqLover commented 2 months ago

Duplicate: https://lists.squeakfoundation.org/archives/list/beginners@lists.squeakfoundation.org/message/CT2DDV4EUDMZOFXN3GAJF3VDUHSQ5MID/

LinqLover commented 2 months ago

BTW, the right enter key (numeric keypad) is also not detected in Squeak under X11. The VM does not emit any event when I hit it.

LinqLover commented 2 months ago

Hi all,

here is a quick update of what I have tried and found out so far:

Technically, handling simple character inputs such as á or ^ is feasible with pretty low effort:

However, the fact that we are receiving first a regular keystroke and then the combining character makes it impossible to handle dead/composition key inputs properly in general. For example, the user might type ^ + Backslash (which should result in no new character) or ` + Home (which should result in a single grave character without shifting the cursor). It might be possible to achieve that by using a second, more fine-grained command history, but that would clearly be a dirty hack.
I wonder how other applications handle this. I have found surprisingly few information about this feature in general so I could not even research other implementations, simply because I lack the right vocabulary. This blog post was the only real resource I found: https://blog.gtk.org/2021/02/18/gtk-happenings/#:~:text=depth%20and%20structure.-,Better%20Input,-GtkIMContextSimple%20is%20the

As an attempt, this new dead key behavior seems to be an alternative input method that is part of the Gnome desktop environment (DE) or its GTK (Gnome Toolkit). Sorry, I'm all new to headful Linux. :-)

We have seen this issue at least for Qwertz keyboards (German) using Gnome/Ubuntu 22 and ABNT2 keyboards (Portuguese) using Gnome (unknown Linux distribution).
Despite further information about idiomatic implementations, I do not know whether the events we get from the VM are correct in the first place, but I tend to doubt it: First, even the first (compose) keyboard input is only sent after pressing the second key, which essentially undermines the idea of the new input method, and second, the order of keyboard events is impractical (see 1.).
Currently, I see two possible ways to proceed:
1. Work with what we get by detecting pairs of \<any key> followed by \<combining character> directly in the EventSensor>>#fetchMoreEvents, effectively converting them into a single regular keyboard event for the entire image.
  
  Caveats: Does not really embrace the new concept. There might be no guarantee that the VM provides both events without an intermediary EventTypeNone.
2. Study the implementation in the VM and the documentation of GTK/X11 (whatever is responsible for that).

For now, I am just dropping these notes here 1) because I will have to suspend this work and resume it later and 2) because I (quite desperately) hope that someone on the list has a better understanding of these input method semantics and could share it with me or point me to some resources or best practices for handling dead keys in Linux.

(/cc @marceltaeumel @dtlewis290)

dtlewis290 commented 2 months ago

When looking at the keyboard events on X11, the /usr/bin/xev utility can be helpful. This will show you the actual X11 keyboard events before they are delivered to the VM. In the VM, the X11 events are converted to Squeak events before being delivered to the image. If your keystrokes show up in /usr/bin/xev then it's not really a dead key, but might be something this is not mapped in the VM.

LinqLover commented 2 months ago

Thank you for the pointer!

When I run xev -event keyboard and type ^ and then a, this is what I get:

KeyPress event, serial 28, synthetic NO, window 0x4400001,
    root 0x50f, subw 0x0, time 179242899, (1044,435), root:(1094,554),
    state 0x10, keycode 49 (keysym 0xfe52, dead_circumflex), same_screen YES,
    XLookupString gives 1 bytes: (5e) "^"
    XmbLookupString gives 0 bytes: 
    XFilterEvent returns: True

KeyRelease event, serial 28, synthetic NO, window 0x4400001,
    root 0x50f, subw 0x0, time 179243010, (1044,435), root:(1094,554),
    state 0x10, keycode 49 (keysym 0xfe52, dead_circumflex), same_screen YES,
    XLookupString gives 1 bytes: (5e) "^"
    XFilterEvent returns: False

KeyPress event, serial 28, synthetic NO, window 0x4400001,
    root 0x50f, subw 0x0, time 179246355, (1044,435), root:(1094,554),
    state 0x10, keycode 38 (keysym 0x61, a), same_screen YES,
    XLookupString gives 1 bytes: (61) "a"
    XmbLookupString gives 1 bytes: (61) "a"
    XFilterEvent returns: True

KeyPress event, serial 28, synthetic NO, window 0x4400001,
    root 0x50f, subw 0x0, time 179246355, (1044,435), root:(1094,554),
    state 0x10, keycode 0 (keysym 0xe2, acircumflex), same_screen YES,
    XLookupString gives 0 bytes: 
    XmbLookupString gives 2 bytes: (c3 a2) "â"
    XFilterEvent returns: False

KeyRelease event, serial 28, synthetic NO, window 0x4400001,
    root 0x50f, subw 0x0, time 179246450, (1044,435), root:(1094,554),
    state 0x10, keycode 38 (keysym 0x61, a), same_screen YES,
    XLookupString gives 1 bytes: (61) "a"
    XFilterEvent returns: False

Notably, the first two events appear immediately, that is before I press the second key. In contrast, in Squeak no event will be received from EventSensor before I press the second key.

It also seems as if the XFilterEvent information is relevant to discover characters that should not be printed into an editor directly but only used for manual character combination or the "dead key preview" feature that I referenced above. We do not have an equivalent concept to that in Squeak.

As a vague theory, the VM might be too clever and try to split up combined characters again. Or it mixes up the order of some things. I will have a look into the sources on another day. If someone can point me to a dead key-aware reference implementation, this still would be helpful. :-)

codefrau commented 2 months ago

Related, although not sure how relevant: take a look at the ImmX11 class and plugin.

codefrau commented 2 months ago

Oh, also, in SqueakJS I pass the first character to the image, and send backspace before the composed character: https://github.com/codefrau/SqueakJS/blob/main/squeak.js#L766

KenDickey commented 2 months ago

I would recommend evtest on Linux. It shows all input devices and lets you select them.

Note keyboard code in platforms/unix/vm-display-fbdev which makes use of /dev/input/event* i.e. sqUnixFBDevKeyboard.c sqUnixEvdevKeycodeMap.c

If you update keyboard input, it would be nice if framebuffer display worked the same.

squeak-smalltalk / squeak-object-memory

Dead key inputs not working properly on Ubuntu #120