Non Qwerty\En US Input

[Kethku]

Right now keyboard layouts are handled by manually listing out bindings given key presses and modifiers. This is very broken.

Luckily there is movement here which should fix this issue. https://github.com/rust-windowing/winit/pull/1788

Once this lands and looks stable enough. I'm going to pull in some version of winit which uses it, and rewrite our keyboard input handling to use that. Most of the hard work was already done thanks to @j4qfrost, so now its a waiting game. Looks like its moving very quickly though so I'm hopeful we will have motion on this soon.

For now I'm going to comb through our issues looking for any that have to do with keyboard input handling and close them as this should fix them or at least make a fix possible.

[pickfire]

Note that IME support is different from keyboard layout, IME uses an external program unlike using a different layout/variant.

[Kethku]

Agreed. However IME support in sdl2 is not good either. I think at this point supporting multiple windowing systems is not the way to go, so I'm going to be moving to switch to winit only once the upstream changes land. At that point, we can work with them to get IME support working for real on the relevant platforms.

[Kethku]

https://github.com/Kethku/neovide/pull/506 a PR has been contributed targetting the new winit api. It currently only works on windows, but once upstream support for linux and mac lands, we should be in very good shape.

[Kethku]

https://github.com/rust-windowing/winit/pull/1890 looks like an upstream macos PR has been created. Very excited

[p00f]

I've set my window manager to swap escape and caps lock but it doesn't work in neovide (they are not swapped), will this fix my issue too 👉👈 ?

[pickfire]

I've set my window manager to swap escape and caps lock but it doesn't work in neovide (they are not swapped), will this fix my issue too point_rightpoint_left ?

I did the same but IIRC I don't have this issue, I only have an issue with dvorak keys when used with other combination keys, I even swap my left ctrl and left alt but I think I don't have an issue with that either. Maybe I did but I don't recall.

[Kethku]

I believe this would fix both of your problems @p00f @pickfire. At the very least it would unblock addressing those issues.

[Kethku]

https://github.com/rust-windowing/winit/pull/1788 PR one of 3 has merged! Shouldn't be long now!!!

[Pebor]

12 days ago But hype!

[Kethku]

Note, we need mac and linux support to merge first

[zoomlogo]

I am using uncap, but it doesn't work.. Is this related to this issue also?

[Kethku]

@PyGamer0 seems possible.

[Kethku]

Linux: https://github.com/rust-windowing/winit/pull/1932 Mac: https://github.com/rust-windowing/winit/pull/1890

There are now PRs for both mac and linux implementations. Making good progress.

[ewen-lbh]

For those that need a workaround while this is getting fixed, you can input any character into the current text buffer by typing the following (from Normal mode):

:r!echo 'YOUR CHAR(S)'

Aliasing echo to sth. like e might make the process slightly less annoying

I just can't get enough of these cursor animations :sweat_smile:

[p00f]

I've set my window manager to swap escape and caps lock but it doesn't work in neovide (they are not swapped), will this fix my issue too point_rightpoint_left ?

this seems to have been fixed on the opengl branch now, nice

edit: caps lock works like escape (nice) but when caps lock is enabled using esc, the letters in neovide are still small

[zoomlogo]

I am using uncap, but it doesn't work.. Is this related to this issue also?

this is fixed on opengl branch

[TobiasDev]

Hello! Will this merge (once a new release comes), also solve the issue with having a Swedish keyboard? Currently when I try to do several different keys it gives me a different character than I expect. E.g. [ turns into something different than it should.

[Kethku]

It should

[fredizzimo]

I saw that rust-windowing/winit#1788 is now merged, so I compiled the latest from source on Windows. However, the input still appears to be broken at least with the Swedish layout.

Regular characters like åäö works fine, and so do most special characters. I did not find any broken ones in combination with shift, but alt-gr combinations have a lot of problems.

For example alt-gr+2 should produce @, but it just acts as ESC. On the other hand alt-gr+3 correctly produces £ for example, I don't really see any pattern, just that it's consistent.

[fredizzimo]

Ah, sorry, I did not read everything properly, it needs to be integrated into neovide too

[Kethku]

Yeah we need to do integration work on our side first. Theres an open PR which attempts this, but I think there will be some more fiddling on top of that.

[Kethku]

https://github.com/Kethku/neovide/issues/691 tagging here. I think it will be necessary to switch to scancodes for linux machines while we wait for the upstream PRs to merge.

Alternatively I may try to make a fork which pulls both the linux and mac versions of the keyboard changes

[Kethku]

https://github.com/Kethku/neovide/pull/736

This integrates the new keyboard api by depending on my merged forks of the upstream keyboard pr branches.

This brings us up to the current level of capability. I would appreciate if folks here could try this PR and let me know:

1. If keyboard handling is different than you would expect
2. What OS you are using, and what keyboard format do you have set
3. What keys exactly did you press
4. What happened in the editor
5. What did you expect to happen

This way I can iterate on the PR a bit. I think the new PRs will let us do something that works how people expect, but it will probably need some fiddling

[TobiasDev]

I can't wait for a new build with this! I love Neovide, but since my default keyboard is a Swedish one I can't use it for everything yet. :D I still do when I code though since I use the US layout for that. But once this is integrated I will be able to use it for all! :)

Thanks for all the good work!

[last-partizan]

I tried that branch, and I found no issues (Archlinux Gnome/Wayland, standard 105 keys keyboard layout).

(I still cannot use it daily becouse of rendering issues in #723, but at first glance all keys/combinations working as expected)

[cljoly]

I’ve tried the branch keyboard-rework and it seems to work.

1. The keyboard handling is as I would expect
2. I’m using Arch Linux with wayland (sway). The layout is what sway calls French (BEPO), which is a dvorak-like layout for French.
3. As you can imagine given the keyboard layout, I have a lot of things remapped. All the below worked flawlessly:
   • directional keys: h, k… are remapped to r, s…
   • w is remapped to é
   • some custom shortcut (in normal mode) like à, ê or <space>à
   • typing letters with diacritics like é, ç… in insert mode
4. & 5. I expected the editor to type the letter I would get in my terminal or web browser and it is what happened!

This is very promising, thanks a lot for your work on this branch!

[danielrparks]

1. If I press a key with the shift key held, and then release the shift key, neovide doesn't register the shift key release, and every character letter that I type afterwards will be interpreted as capitalized.
2. Arch Linux with Sway (Wayland). Plain dvorak. The bug also happens to me on standard QWERTY.
3. ihello world<S-1> the special k... wait why is everything stuck in caps<ESC>ohow do I exit<ESC><S-;>q<S-1><Return>
4. Actual:
   
5. Expected:
   (exits)

Some additional notes:

• Pressing the shift key alone once is sufficient to trigger the bug.
• Special characters and number keys are not affected; only letters.

[cljoly]

@danielrparks I have the same bug as you, except that it happens for all modifier and that I can’t reproduce it as consistently (pressing a modifier rarely triggers it).

[Kethku]

Ok that's a valuable hint. I have some ideas

[Kethku]

@cljoly, @danielrparks I pushed a small change that I think should address the modifiers issue. Can you try again?

[danielrparks]

That's fixed the shift-stuck bug, but now I found a new one.

I use auto-pairs to automatically insert closing symbols. In this build, auto-pairs only works on symbols that are not shifted, like ``, [], ''. It does not automatically complete pairs for (), {}, "", <>. On standard terminal neovim, it works as expected.

[ccakes]

I’ve tried the branch keyboard-rework and it seems to work.

:+1: Extra data point, works here too. Arch, sway, US layout (altgr-intl variant). Thanks for the awesome work!

[ouuan]

Agreed. However IME support in sdl2 is not good either. I think at this point supporting multiple windowing systems is not the way to go, so I'm going to be moving to switch to winit only once the upstream changes land. At that point, we can work with them to get IME support working for real on the relevant platforms.

Just want to make sure, is the progress of IME support also being tracked here? I think they are two different issues. The keyboard layout problem seems to be nearly fixed, but I don't know the status of IME support. Perhaps it will be better if we can track the progress of IME support in a different issue. I'm using the US keyboard layout but sometimes use fcitx5 to input Chinese. I subscribed to this issue, but I actually don't care about the progress of the keyboard layout problem.

[last-partizan]

After #723 is fixed, i'm tried to use this branch again, and found another bug:

When i press % it moves between matching braces, this is works.

But, in raw neovim it also moves between opening/closing html tags, this part is now broken.

[ITHackerstein]

Hi, I tried out the keyboard-rework branch, and I've had some interesting results trying out a bunch of key combinations. I noticed that when I maximize the Neovide window using Win+UpArrow I get some weird character sequences. Here are some examples for some key combinations:

• AltGr Shift è (should output {) --> <S-D-{>
• AltGr Shift + (should output {) --> <S-D-}>
• Tab (should output a tab character) --> <D-Tab>
• Alt Tab (should switch to another window) --> <M-D-Tab> and switches to normal mode.

If I instead don't maximize the window or I maximize it using the title bar button I get the correct keys.

OS: Ubuntu 20.04 Keyboard layout: QWERTY Italian

[danielrparks]

In the same vein as what @last-partizan noticed, using % to move between begin/end environment statements in vimtex is also not working. Instead, it tries to move between the curly braces.

For example, instead of moving from |\begin{enumerate} (pipe represents the cursor) to \end{enumerate|}, it moves from |\begin{enumerate} to \begin{enumerate|} (it moves to the curly brace on the same line).

[fredizzimo]

I tested it on Windows with the Swedish keyboard layout, and the dead keys don't seem to work correctly. But it's a big improvement from the previous version, since everything else seems to work.

More specifically dead keys followed by a space or an unsupported character don't work. For example

¨ followed by e correctly outputs ë ¨ followed by <space> incorrectly outputs <space>, but it should output ¨ (notice that the space should not be output in this case) ¨ followed by d incorrectly outputs just d, it should be ¨d ¨ followed by another ¨ incorrectly outputs nothing, it should be ¨¨

All the dead keys ¨^~´` behave the same way

Especially the second case with space is very important since that's the only way to ouptut common symbols like `~^, althought I suspect that some people who don't know better might rely on the last case and press the dead key twice and then delete the extra character.

[cljoly]

That's fixed the shift-stuck bug, but now I found a new one.

I use auto-pairs to automatically insert closing symbols. In this build, auto-pairs only works on symbols that are not shifted, like ``, [], ''. It does not automatically complete pairs for (), {}, "", <>. On standard terminal neovim, it works as expected.

Same for me on those two points.

OS: Arch Linux with Sway (wayland)

[cljoly]

I tested it on Windows with the Swedish keyboard layout, and the dead keys don't seem to work correctly.

@fredizzimo I have the same behavior on Sway (wayland) with Arch Linux.

[Kethku]

@fredizzimo @cljoly I suspected this would be an issue with dead keys. The problem is that I don't know what the correct behavior should be for handling both bindings in normal mode and text input in input mode. I think the PRs upstream give us tools which should make the problem tractable, but I don't know enough about common keyboard patterns with non qwerty keyboards to be confident in what changes are necessary.

A couple questions (I'll have more):

1. What should happen you configure a keybinding to a deadkey? Is that something anybody does? Should it be a no-op?
2. Should normal mode keyboard handling act differently from insert mode? We could pipe the current mode into the window system, but I would need to know what people would expect the difference to be.
3. Its not clear to me how modifiers should get sent to neovim. My understanding of keyboard layouts is that sometimes a modifier is "consumed" in order to modify the inserted text for a given key press. I don't know if I have the ability to tell whether a particular modifier has been used up in this way given the api I have from winit. Is it ok to still pass the modifier even if it has modified the associated key? Maybe I'm completely confused haha

Answers to these questions would go a long way toward helping me know what the correct implementation of this stuff should be. I'm very much not an expert in this area, so fixing this would require some time and commitment to help me understand. Alternatively I would be happy to assist somebody if they want to dive in. Either way works for me

[fredizzimo]

@Kethku,

Yes, dead keys can be a bit hard to understand until you have actually used them. But the behaviour we are looking for here is quite simple. Normally the terminal handles the dead keys, and Vim will only see the end result, so that's what should happen here too.

So to answer your questions,

1. Keybindings to deadkeys are perfectly normal, the default binds ^ (go to first non blank on the line), and the backtick ` for marks. To use them you would have to press the the dead key followed by space. So to go to the start of the line I would press shift+^ and then <space>. I don't know how common it is, but it's also possible to map things like nmap ë ihello, which would enter insert mode and type hello if I press ¨ followed by e.

   It's also possible to map the other way around, and I guess many people do that for example map ö ^, so that they can press the dedicated ö key instead.

   Personally I have a programmable QMK keyboard, so I work around the problem by having a keymap that sends the space combination by default, with the actual dead keys are on another layer.

   2. All VIM modes should behave exactly the same way, since the dead key processing is the first thing that should be done.
   3. For dead keys, the modifier should be handled separately for the actual dead key and the following letter. So I can for example press altgr+~ and then shift+a for Ã. I'm also fairly sure that you should pass the modifiers that were held down at the moment the second key was pressed to Vim.

A final note. Most keyboard layouts have simple dead keys with just two key presses, but some layouts have chainable dead keys, where you can press two or more dead keys in any order followed by the key to modify. I don't know how popular those are, and if they need to be supported, but the principle for supporting them is the same, wait for the final modified character and pass that to Vim along with the modifiers. But it can go even beyond that, for example, I know that the Vietnamese keyboard layout has a slightly more complex variant of the dead keys, and that the Japanese and Chinese input methods are completely different.

[cljoly]

I don't know how common it is, but it's also possible to map things like nmap ë ihello, which would enter insert mode and type hello if I press ¨ followed by e.

I have a few mappings on things with dead-keys, it works quite well in the terminal. I’m not saying it is hugely popular though.

I'm also fairly sure that you should pass the modifiers that were held down at the moment the second key was pressed to Vim.

That’s the behavior I have with Alacritty.

[Kethku]

Ok I've made another stab at better dead key handling: https://github.com/Kethku/neovide/tree/keyboard-deadkey-handling

Can you try this and let me know if it works how you would expect? I don't have time now, but I will explain the logic later if that would help. The important logic is here: https://github.com/Kethku/neovide/blob/keyboard-deadkey-handling/src/window/window_wrapper/keyboard_manager.rs#L155-L170

[Kethku]

@danielrparks I think the keyboard-deadkey-handling branch may address your shift key-auto pair problem. Could you try again on that branch?

[danielrparks]

Everything seems to be working properly for me now, including auto-pairs and using % to move between html tags / latex environment begin & end. Note that I don't use dead keys, but I do use the compose key. Input using the compose key worked as expected, including custom mappings from ~/.XCompose, and I was even able to map compose key combinations to neovim commands. Thanks for putting so much effort into making sure neovide works for everyone!

[fredizzimo]

Thank you @Kethku,

The dead keys seem to work perfectly now, at least for the cases I can think of.

[clason]

Here's my (non-working) setup:

• macOS Big Sur (Intel)
• German keyboard layout

If I type option-8 (which is { in my layout), I just get dropped out of insert mode. The relevant log events are

TRACE [neovide::channel_utils] ui_command Keyboard("<M-8>")
TRACE [neovide::redraw_scheduler] Next frame queued
TRACE [neovide::bridge::ui_commands] Keyboard Input Sent: <M-8>

[cljoly]

Thanks @Kethku, in every way I could think of, dead keys work for me as well, in particular the mappings on dead keys.

[Kethku]

@clason what happens in terminal vim for your use case. Alt in base vim escapes from insert mode. Its possible that your use case will need some setting for langmap :(

[clason]

Well, obviously in terminals it works as expected, or I would never have started using neovim :)

(To be honest, that's the first time I heard that alt is not a modifier key but escapes insert mode on US keyboards...)

But, yes, input handling needs to be language (or layout) specific, I'm afraid. Since the base keys work now (which was not the case until recently), I had assumed that the plumbing for that was already in?