Keyboard Event issues on Windows

[marceltaeumel]

VM: 201608171728 OS: Windows 10 1607 Keyboard: German

Here is how I observed the keyboard events in Squeak5.1rc1 (after filing in the attached files):

KeyboardEvent-printOn.st.txt KeyboardEvent-printKeyStringOn.st.txt

| filter |
filter := PluggableEventFilter on: [:evt :target |
    Transcript showln: evt printString.
    evt].

Preferences disable: #duplicateControlAndAltKeys.
Preferences disable: #swapControlAndAltKeys.
Preferences disable: #duplicateAllControlAndAltKeys.

ActiveHand addKeyboardCaptureFilter: filter.

"After observation finished."
ActiveHand removeKeyboardCaptureFilter: filter.
Preferences enable: #duplicateAllControlAndAltKeys.

I observed the following bugs:

• There are way too many keyDown events when holding, for example, the letter 'x' key. I would expect a single keyDown, multiple keystroke, and after letting it go, a single keyUp. Since whe have keystroke events in the first place, there is no need for multiple keyDown (or a repeat field as in SDL).
• If I add the modifier key after pressing the key to be modified, there will be no keystrokes at all and not even the correct keydown event (e.g. [up] + [ctrl] instead of [ctrl] + [up])
• The values for keyDown and keyUp for key characters 65 < c > 90 are wrong. For example, pressing '#' gives keyDown/Up value 192, while the keystroke is good with 35. On German keyboard.
• There are no keystroke events generated for CTRL+[0..9].
• keystroke events using the CTRL modifier are wrong for [a..zA..z] while the keyDown events are good. CTRL+C yields 67 for keyDown but 3 for keystroke, which is ENTER. Adding the SHIFT modifier to CTRL shows the same bug.
• On German keyboards, the [AltGr] modifier is mapped to a mixture of [CTRL] 17 and [CMD] 18 for keyUp/Down. The resulting keyCharacter for keystroke events is good.
• The keystroke for [CTRL]+[backspace] shows up as [CTRL]+[delete]. keyUp/Down are good.

[StephanEggermont]

Do the function keys do the right thing?

[marceltaeumel]

In Windows, F1 to F12 did never arrive in Squeak. F2 opens the VM context menu. Or what do you mean with "function keys"?

[StephanEggermont]

Yes, that was exactly what I meant

[marceltaeumel]

Then there is Alt+F4 (resp. CMD+F4 from the perspective of a user in the image), which wants to quit the session. I think that the other combinations are free. Would be nice to think about adding function key up/down/stroke to the image. We should open another issue for that as a feature request. :smile:

[codefrau]

If you take into account the discussion in #43, what are the actual bugs that need fixing right now? We need a new issue for redesign/enhancements.

[marceltaeumel]

There is nothing new broken on Windows. I would rather keep this issue here to discuss the simplification of key-up, key-down, and key-stroke.

[nicolas-cellier-aka-nice]

Note about Ctrl+A/Z: AFAIU, it's what the console app were expected in the ages, so our OS still carry this tribute to legacy...

See how we have to workaround on linux: https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/abf05549bd377106f27600a293d79f75a59e705e/platforms/unix/vm-display-X11/sqUnixX11.c#L2065

Who is turning the keysym into 1 to: 26 charCode? https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/abf05549bd377106f27600a293d79f75a59e705e/platforms/unix/vm-display-X11/sqUnixX11.c#L2049 See https://linux.die.net/man/3/xlookupstring

I also mentionned this in issue #396

So we might want to do something similar on Windows? (and Mac OS?) For Win32, see https://docs.microsoft.com/en-us/windows/win32/learnwin32/keyboard-input

MSDN page above explicitely explains that CTRL+A/Z are translated to ASCII control character in the VM_CHAR event (as I suggested in preliminary comment, for legacy console app). It also does not recommend handling CTRL+... or ALT+... thru WM_CHAR event, this is not the idiomatic handling on this platform. And the doc is right - see why below.

The equivalent of linux keysym is the virtual key code (VK_*) passed thru wParam in WM_KEYDOWN event. https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/d58235ca011b9a8f8344814b66d9f9d6428c963f/platforms/win32/vm/sqWin32Window.c#L1262

The problem is that we effectively get the keycode in case of WM_KEYDOWN event, but not in case of VM_CHAR event: msg->wParam holds an UTF-16 Unicode character the doc says...

So, IMO, we should better generate an extra OpenSmalltalk EventKeyChar event in response to WM_KEYDOWN event when we detect a CTRL down status, and filter out the event in case of VM_CHAR with CRTL down...

Note that Pharo VM uses MapVirtualKey with MAPVK_VSC_TO_VK to handle ctrl+char https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/d58235ca011b9a8f8344814b66d9f9d6428c963f/platforms/win32/vm/sqWin32Window.c#L1319 https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-mapvirtualkeya The description of msg->lParam comes from https://docs.microsoft.com/fr-fr/windows/win32/inputdev/about-keyboard-input. MapVirtualKey expects a scan code, which is low level keyboard code and will return a VK code that fortunately matches ASCII for A to Z keys and other printable ASCII... It's not the way that MSDN (and I) recommended above, but probably work (I'm not in a position to understand if there are other side effects, but we could just experiment it...)

I forgot to say this: WM_CHAR events are generated by the call to TranslateMessage https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/d58235ca011b9a8f8344814b66d9f9d6428c963f/platforms/win32/vm/sqWin32Window.c#L1666 https://docs.microsoft.com/fr-fr/windows/win32/api/winuser/nf-winuser-translatemessage

[LinqLover]

Another bug:

SHIFT seems to be ignored in the VM when CTRL is pressed. SHIFT+1 yields $! but CTRL+SHIFT+1 yields $1.

See http://lists.squeakfoundation.org/pipermail/squeak-dev/2021-October/216736.html.