Scandinavian keyboard layout can't handle input tilde (~)

[ixrock]

To type ~ with Finnish/Swedish keyboard I have to press hotkey alt+^:

After that it shows something similar to ~ but right after when I press any other character (except Enter) it will erase "pre-tilde" symbol. Press Enter "converts" to normal ~ and nothing more (without real Enter).

I checked how this works with vanilla JS and it seems something wrong at browser-level:

Is it fixable? Wdyt?

Related bugs:

https://github.com/xtermjs/xterm.js/issues/174

Details

• Browser and browser version: Chrome (latest)
• OS version: MacOS X (latest)
• xterm.js version: 3.14.0

Steps to reproduce

1. Add support of Finnish keyboard in your OS
2. Try to use ~ from keyboard

[Tyriar]

VS Code issue: https://github.com/microsoft/vscode/issues/68262

[ixrock]

@Tyriar how xterm related to vscode?

[Tyriar]

vscode uses xterm, that's the issue that's been reported over there.

[gdlx]

Terminus has the same issue: https://github.com/Eugeny/terminus/issues/768 This isn't limited to Scandinavian keyboards. At least French and Spanish are affected too (any layout with the AltGr key and 2 steps accents feature I presume).

[ixrock]

It's still possible to catch ~ with InputEvent:

[Tyriar]

I'm not sure if this ever worked if InputEvent fixes the problem, sounds like it might be the same root cause as https://github.com/xtermjs/xterm.js/issues/469 and needs us to move input handling over to input instead of the key events.

[ixrock]

@Tyriar Using only InputEvent for sure not enough. Its evt.data just contains current value-string for input/textarea and nothing about pressed keys, etc. I just thought it might help to fix this issue combined with normal KeyboardEvent.

[goulartdev]

I'm having the same issue in Fluent Terminal. I can't type ~, it shows Þ instead. My keyboard is PT-BR (with the AltGr key), but if I change the layout to US, I can type ~ using the ' key.

[insertt]

Until this issue will be fixed I suggest using workaround from here

[GitSquared]

I'm not sure if this exactly the same issue, but I find that typing any diacritics (like ö ầ etc), not just the tilde, on xterm doesn't work at all (prints like o a, without any diacritic applied). Tested with french keyboard and Linux, latest xTerm.js on Hyper and/or eDEX-UI.

[Tyriar]

@GitSquared I think this might be related to your $LANG variable?

[GitSquared]

@Tyriar My $LANG is set to en_US.UTF-8 - but typing diacritics does work in other terminal emulators (Konsole), and switching to, for instance, fr_FR.UTF-8 does not resolve the issue.

Also, I'm unsure how $LANG affects typing input? Is this some xterm.js-specific behavior?

[jerch]

@Tyriar Is this only an issue under Mac or with Mac keyboards (MBP)? I have no problems with PC keyboards with german keyboard layout - all third level shifts work as intented in all browsers (tested with demo in chrome + FF).

Some ideas whats going wrong (in fact it looks like 2 separate issues on the lower level):

• original poster reported to press Alt + ^: Alt has a special meaning by default, we map it to the Meta key for PC keyboards (as it lacks a dedicated Meta key). This assumption might be wrong for some keyboard types (Mac?). For Mac keyboards we have an macOptionIsMeta setting, we might have to check if this needs a slightly different Alt+XY key handling in Keyboard.ts as well if set.
• fluentTerminal and AltGr issue: This looks like the browser engine reports the wrong key/keyCode (not applying keyboard layout correctly?). When AltGr is held down normally the reported entry should shift to the third character for the second key, if browser do wrong here I fear there is not much we can do in the first place. But atm this is all speculation and needs proper bug testing to catch all edge cases. If InputEvent still reports the right key value we might get away with a combined KeyboardEvent + InputEvent processing. If this does not work - a last resort might be to deliver our own (customizable) keyboard layout maps.

Key handling always has been a pain in the a** in browsers, maybe we can learn something from the monaco guys here?

Things mentioned in this issue are hard to fix as long as we cannot repro it and dont know the exact circumstances this bug shows up (OS + browser version). Any volunteers to (partly) address this?

To test if the browser reports the right thing you can use https://w3c.github.io/uievents/tools/key-event-viewer.html.

[Tyriar]

Is this only an issue under Mac or with Mac keyboards (MBP)?

@jerch you might be right OP and the 2 linked vscode issues are all on macos.

[laarmen]

FYI, a string of closed/duplicate issues on the VSCode repository lead me to this issue, and I'm on Windows. Using a French AZERTY layout, the tilde is on the "number" row, on the "2" key, which has as main char "é", shift char "2" and third char "~", which normally has a dead key behaviour. Thus, to type in a ~ character, the normal sequence would be AltGr+é, Space. However, when doing so the result is é~.

I'm not entirely certain it is the exact same bug, but I'm assuming it stems from the same root causes.

[jerch]

@laarmen It seems we first have to identify the problematic layouts, AZERTY seems to be one of them. Our problem is that none of us can dig into that, we only use english/german keyboard layouts.

So if you dont mind - could you also check if the other third shifts are wrong as well (I assume so)? And whether https://w3c.github.io/uievents/tools/key-event-viewer.html reports the right thing - if so its prolly a problem on our side. Please also report us your OS version and installed language / keyboard layout. (Not sure yet, I think the browser tries to apply the OS keyboard layout setting for the event codes, thus this might also be a source of trouble depending on the configuration).

Edit: Here is a list of common latin keyboard layouts https://en.wikipedia.org/wiki/List_of_Latin-script_keyboard_layouts (and links to other layouts in the footer). And the ISO spec for keyboards https://en.wikipedia.org/wiki/ISO/IEC_9995.

[laarmen]

OK, so I tried to reproduce on a Docker xterm.js instance, and couldn't reproduce (Firefox and Chrome both). I'm on Windows 10, and my bug happens in the VSCode terminal (1.38.1). 🤷‍♂

[jerch]

@laarmen Oh thats weird, Im pretty sure vscode runs exactly the same code for key input unaltered.

@Tyriar FYI. Maybe an electron bug not correctly applying OS keyboard layout?

[Tyriar]

@rebornix have you seen any reports around this for monaco?

[ssh-peppe]

For me on macOS, Finnish keyboard, with Safari and Chrome a tilde + space results in ~ and tilde + n results in ñ as it should. But on Firefox a tilde + space results in ~~ and tilde + n results in ~ññ.

Interestingly this behavior difference can be seen (but a bit differently for some reason) on the https://xtermjs.org/ home page example terminal. If you enter tilde + space there on Safari you get nothing at all. Same for tilde + n. But with Firefox while tilde + space gets you nothing a tilde + n gets you ñ. Not sure why it's working differently there.

Debug logging shows the following when typing tilde + space on Firefox:

xterm.js: sending data "~" Array [ 126 ]
xterm.js: sending data "~" Array [ 126 ]
xterm.js: parsing data ~~

For tilde + n it shows:

xterm.js: sending data "~" Array [ 126 ]
xterm.js: sending data "ñ" Array [ 241 ]
xterm.js: sending data "ñ" Array [ 241 ]
xterm.js: parsing data ~
xterm.js: parsing data ññ

Note that there are differences in browsers with regards to what is reported from e.g. compositionend vs. input vs. keydown vs. keypress events.

I should add that nothing is being sent until you hit space or n. That is, the initial tilde that starts the character compose does not result in any callbacks firing, whch is correct.

[ssh-peppe]

The sequence of keyboard events recording by attaching a custom keyboard handler using the Terminal API, for tilde + n is on Windows Firefox (71) with Finnish keyboard is the following:

keydown Control { target: textarea.xterm-helper-textarea, key: "Control", charCode: 0, keyCode: 17 }
keydown { target: textarea.xterm-helper-textarea, key: "AltGraph", charCode: 0, keyCode: 18 }
keydown { target: textarea.xterm-helper-textarea, key: "Dead", charCode: 0, keyCode: 160 }
keyup { target: textarea.xterm-helper-textarea, key: "Dead", charCode: 0, keyCode: 160 }
keyup { target: textarea.xterm-helper-textarea, key: "Control", charCode: 0, keyCode: 17 }
keyup { target: textarea.xterm-helper-textarea, key: "AltGraph", charCode: 0, keyCode: 18 }
keydown { target: textarea.xterm-helper-textarea, key: "ñ", charCode: 0, keyCode: 78 }
keyup { target: textarea.xterm-helper-textarea, key: "n", charCode: 0, keyCode: 78 }

On macOS with Firefox (71) with Finnish keyboard the sequence is:

keydown Alt { target: textarea.xterm-helper-textarea, key: "Alt", charCode: 0, keyCode: 18 }
keydown Alt { target: textarea.xterm-helper-textarea, key: "Dead", charCode: 0, keyCode: 160 }
keyup Alt { target: textarea.xterm-helper-textarea, key: "~", charCode: 0, keyCode: 160 }
keyup { target: textarea.xterm-helper-textarea, key: "Alt", charCode: 0, keyCode: 18 }
keydown { target: textarea.xterm-helper-textarea, key: "ñ", charCode: 0, keyCode: 78 }
keyup { target: textarea.xterm-helper-textarea, key: "n", charCode: 0, keyCode: 78 }

[ssh-peppe]

Perhaps this is a different issue so I'll file a separate ticket.

Filed issue #2661

[tcardonne]

Hello,

I'm having the same issue on VSCode's integrated terminal with a french layout. I'm using Windows 10 2004. When I want to input the ~ character, it prints me é~ (Alt right + é then space). Alt right + é (twice), it prints me éé~. I can reproduce the issue on xtermjs.org.

I tried to gather some logs/infos to help you. If I understand correctly, you can't reproduce the issue on your side, so tell me if you need anything else.

VSCode debug, attached onkeydown and oninput to console.log :

(the é appears on AltGraph keyboard event).

Keyboard event viewer :

[Tyriar]

@tcardonne I believe this never worked and is happening because we're listening using keydown/key/keyup events as opposed to the input event, this is the same reason emoji support isn't great https://github.com/xtermjs/xterm.js/issues/469. This change might end up being relatively easy to move over but since it's a pretty fundamental change to how input works it will need a lot of validation and tests.

[tcardonne]

@Tyriar I understand.

It's weird, I have the issue on my desktop at home but I can't reproduce it with my work laptop (with the same keyboard attached to it). I've tested on xtermjs.org website and on vscode. The only difference appears to be the Windows version (2004 on personal computer, 1903 on work laptop).

Here is the keyboard event viewer table : (notice the Dead key)

This might be related to a difference in OS settings, but I can't find any difference. Do you have any idea on settings that might affect this behavior?

[tcardonne]

Ok, I feel stupid, I didn't noticed the NumLock on my first comment. I have a small keyboard without numpad and function keys, so I didn't noticed.

However, even with num lock, it should work, so the underlying issue (use of the input event) is still present, I guess.

Nevertheless, I'm happy to have found the cause of my issue. Hope it helps!

[Zincr0]

I see that the https://github.com/microsoft/vscode/issues/98533 was marked as duplicated for this, but https://xtermjs.org/ can type any áéíóú characters whitout any problems while the integrated vscode terminal cannot...

[ricpelo]

I see that the microsoft/vscode#98533 was marked as duplicated for this, but https://xtermjs.org/ can type any áéíóú characters whitout any problems while the integrated vscode terminal cannot...

Disagree. I can't type any áéíóú characters in the https://xtermjs.org/ demo with my Spanish keyboard, neither.

[Zincr0]

I see that the microsoft/vscode#98533 was marked as duplicated for this, but https://xtermjs.org/ can type any áéíóú characters whitout any problems while the integrated vscode terminal cannot...

Disagree. I can't type any áéíóú characters in the https://xtermjs.org/ demo with my Spanish keyboard, neither.

Maybe is a SO or a browser thing? i'm using firefox on kde Neon here

[ricpelo]

I see that the microsoft/vscode#98533 was marked as duplicated for this, but https://xtermjs.org/ can type any áéíóú characters whitout any problems while the integrated vscode terminal cannot...

Disagree. I can't type any áéíóú characters in the https://xtermjs.org/ demo with my Spanish keyboard, neither.

Maybe is a SO or a browser thing? i'm using firefox on kde Neon here

You're right: I can reproduce the issue using Google Chrome but the áéíóú characters works right on Firefox. :open_mouth:

[jerch]

@ricpelo This smells like an issue with "dead" keys behaving differently in FF, Chrome and Safari. Could you try to record the events with https://w3c.github.io/uievents/tools/key-event-viewer.html in FF and Chrome and post the results here?

[ricpelo]

@jerch sure. Here I go:

Chrome:

Firefox:

[jerch]

@ricpelo Thx alot. The problem comes from this:

While FF already gives the dead key accumulated character on final keydown in key, chrome just returns the raw key. I am not sure who is to blame here, imho the keyboard stuff in keydown is not well specced at all. It also shows that input gets the chars right in both cases. Thats quite a strong argument to move to input (we already figured that above).

But input has difficulties to correctly catch meta keys like Strg, Alt, AltGr (left/right option on OSX), esp. Shift in combination, but we need those in a terminal to promote any Strg+X and such. I guess we need a more complex event handler, that basically relies on input but does another decoration/translation of the char from meta key states states of "enclosing" keydown / keyup events. Sounds nasty, but should be doable. Still the testing is hard to do (no good way to automate this without physical interaction).

[ricpelo]

Any news about this issue? Is there a proposed pull request or something similar?

[skneko]

I see that the microsoft/vscode#98533 was marked as duplicated for this, but https://xtermjs.org/ can type any áéíóú characters whitout any problems while the integrated vscode terminal cannot...

Agree. That website works perfectly with my Spanish keyboard, but in VSCode I can't input tilde nor letters with diacritics.

[ojacques]

🥳 The workaround mentioned earlier https://github.com/xtermjs/xterm.js/issues/2151#issuecomment-515116994 by @insertt worked for me (French keyboard). No more é~ in VS Code terminal with my French layout keyboard!

• Create / install the layout without dead keys per the instructions above (here is the one I built for me: French Keyboard layout for VSCode.zip)
• Reboot (otherwise, Win 10 did not took fully into account)
• In Win 10 / Language settings, add the new keyboard layout

(My PC is configured for US, but I use a French Keyboard)

[alexandrehpiva]

As @Tyriar set that issue as a duplicate of this one, I am posting here that nothing solves for me.

I tried to re-install VSCode, change terminal to use ZSH shell, change the VSCode language and added configuration "terminal.integrated.env.linux" to use UTF-8.

I personally don't think this is the same problem of that issue because I am using Ubuntu and when I type (in VSCode terminal) a special char alone and press space it works ex.: "^~`'", but when I type a special char like "^" + "a" only show "a" instead of "â". Same with "áâãéẽíĩóú". But in normal Ubuntu terminal everything works ok.

Does anyone have any more ideas?

[jerch]

@alexandrehpiva Can repro that under ubuntu with chrome and QWERTZ (german layout), where ^ is a dead key. I think dead key composition never worked correctly for chrome under linux, while it works with firefox.

Can you confirm - is ^ a dead key in your keyboard layout? (dead key - not producing a symbol on first press, instead adding some modification to a following key, often used to add ^, ~, ¨ or other accents to a letter)

[Eugeny]

@Tyriar bad news, found out that some Windows machines just don't produce the "Dead" keydown no matter what. I'm investigating and I think we should reopen this for the time being.

[Eugeny]

Looks like it's producing a bogus version of a third level shift with AltGraph instead of alt and missing modifier flags - here's the broken version:

and a normal third level shift:

[Tyriar]

😦

[fperez]

I am seeing the same problem (which also impacts JupyterLab, where we use xterm.js) and wanted to mention that there's an easy way even for developers on US keyboards to reproduce it, which may help track it down: I'm on a Mac, and if you install the "US International" layout:

that will give you dead keys for accents in western languages (I use it as I have to type regularly in English, Spanish and French + coding and my keyboards are US-layout). With this layout active, to type say cd ~ I will type cd, then tilde, then space. The space keystroke makes the tilde dead-key become the normal tilde (instead of say ~ + n, which will become ñ with that layout.).

As of right now, testing this on the xtermjs.org terminal with this layout does not work (nor do other accents).

I hope this helps the dev team perhaps track it down better - we'd greatly benefit from a fix to this on the JupyterLab team... LMK if I can provide further testing/feedback (no fixing offered - I don't know this codebase/JS well enough to help, sorry!!). Otherwise thanks for the fabulous tool!

[alexandrehpiva]

Can you confirm - is ^ a dead key in your keyboard layout? (dead key - not producing a symbol on first press, instead adding some modification to a following key, often used to add ^, ~, ¨ or other accents to a letter)

No, it isn't. My keyboard is pt-BR Brazilian Portuguese. Really, my problem explanation is like the issue https://github.com/microsoft/vscode/issues/105051

[fperez]

@alexandrehpiva do you still get the problem on VSCode? I just tested it on my current setup (VSCode 1.59.1 on MacOS Big Sur, with US International layout) and it's now fine. Just checking to see if with current versions of VSCode it still happens, to help narrow down things.

[alexandrehpiva]

@alexandrehpiva do you still get the problem on VSCode? I just tested it on my current setup (VSCode 1.59.1 on MacOS Big Sur, with US International layout) and it's now fine. Just checking to see if with current versions of VSCode it still happens, to help narrow down things.

Yes, I just update to 1.60.0, and still same problem. I'm unable to use accented characters with Ubuntu bash inside VSCode. I tried to install and use Oh My ZSH too and still same. The mad thing is that work's perfectly in terminal outside VSCode. Anyone knows how to change configuration for this (if exists) in Ubuntu 20.04 LTS?

[alexandrehpiva]

I recorded an example:

[alexandrehpiva]

I just discovered a strange thing: If I press the accent key 3 times and then press the letter, it works and I get: "~ã"

[thiagobraga]

I just discovered a strange thing: If I press the accent key 3 times and then press the letter, it works and I get: "~ã"

I realized the same thing today. If I press the tilde key followed by the "a" key, it is displayed a in the terminal and not ã. But if I press the tilde three times and then the "a" key, ~ã is produced. I imagine the issue should not be closed yet. I'm on Ubuntu 20.04.3 and on Terminator and Gnome Terminal the accent works perfectly.

[devphobia]

Ubuntu 20.04 using GNOME and having the same issues. The integrated terminal doesn't seem to allow me to input characters like [é, ó, ú, ô, etc...] while all my other terminals work just fine. Even installed Atom to see if the bug persisted on other code editors but it seems it's related to VSCode and VSCode only. Any ideas on how to get rid of this?

[mauriciosantosds]

Same problem here with vscode integrated terminal in Ubuntu 20.04 GNOME.