Keyboard layout "U.S. International PC' is not handled correclty

[ecerulm]

I have "U.S. International PC" as Keyboard Input Source which means that to produce ä I need to press " followed by a and to produce an actual double quote I have press " followed by space.

To go to a mark like '> I can press '<space>> in vim inside iTerm (terminal emulator) and works fine. When I try the same thing in MacVim it doesn't work. MacVim doesn't seem to recognize this while in normal / operator pending mode,

So '<space>> seems to be interpreted (not sure) as :

1. jump to mark called <space>
2. then indent.

but it it should be interpreted as "jump to end of last visual selection"

If I type just '>seems to be interpreted as nothing. But the right sequence to type would be '<space>>.

Anyway MacVIm doesn seem to be usable with U.S International PC layout, while vim in iTerm works fine.

[eirnym]

If you open on-screen keyboard in OS X, you'll see these two symbols (` and ') are not the same as others. They are used to enter sybmols like á without requirement of holding a button or using an option key. Because of this handling, OS X generates different input sequence to an application while using these keys.

Terminal's input is a text-based, when MacVim (GUI) not in Insert or Replace modes, the input is keypress-based, and it swallowed by OS X.

Try to change a layout to US and check if '> works there. To insert symbols from extended alphabet you can press and hold a button and then select required from a pop-up selection.

[ecerulm]

Changing to US instead of US International work but it's not really an option for many (for those of us that write in several languages like spanish, english and swedish and don't want to switch to different layout every time).

I wonder if it will be possible to handle the input an at different level letting macOS handle the dead key like iTerm does.

[eirnym]

MacVim uses NSTextView as a base and doesn't handle keyboard input as PTY does (e.g. in iTerm/Terminal). To rewrite this core components it would be too much effort to be involved than use compatible keyboard layout.

PS: I use Polish layout (Polski pro) for most of my input and completely fine with it. PPS: Looked on a few more European keyboard layouts and saw that most usable keys are accessable directly or with modifiers like Shift or Option

[eirnym]

@ecerulm Does it still an issue for you?

[ecerulm]

well, yes, I stopped using macvim (and VimR) because of this. I know I can use Swedish Pro , as you say most of the keys are accessible via modifiers. But I choose long time ago to go the International Dead keys way, I can't go back anymore. I just use vim in the terminal.

I understand that maybe dead keys layout are not that common and maybe you want to close this as WONTFIX.

[eirnym]

I think it's unsolvable right now because we use NSTextView as for a render and key handling. But it could be changed in the future, who knows.

[bighi]

I just found this issue. I am using MacVim and it makes me sad. I don't know what NSTextView is, but every other Mac app handles dead keys perfectly, MacVim is the only one with a problem. So it makes me guess it's MacVim doing something wrong, not the text thing provided by Apple.

I could help debug it if needed. But I would relaly like to see this issue fixed. It's the only thing preventing me from using MacVim (and many others, since everyone on my company alerted me not to try).

[eirnym]

@bighi You're welcome to help.

AFAIK not every single application has fixed such issues and not many applications in macOS are really using ALT/Meta/Option keybindings by default at all. For example, there's a problem for Visual Studio Code (and other Electron-based apps), IntelliJ Idea (and other java apps) where the problem is quite the opposite. Currently I doesn't see any such bindings (Option-letter) in the recent versions of IntelliJ products.

macOS has both option keys behave as AltGr in Windows and Linux (and there you have Alt and AltGr separated), so you have to catch these keys in a special way and then put them into event queue for Vim.

PS: I personally using Polski Pro without any headache. All keybindings with alt I use, I've re-binded and I'd like to keep current behavior for myself when possible.

[bighi]

Sure, I'm happy to help however I can.

But maybe there's a misunderstanding. OP was not talking about ALT/Meta/Option at all.

It's just that for example, while using US International, pressing ' and then a results in á in every other app, except MacVim. And there are many other combinations.

In the US International layout, some keys become dead keys. When pressed, they do nothing until you press a second key. And then one single character is inserted, despite the fact that the user pressed two keys. And some keys like ' and " are dead keys.

The problem is that in MacVim, dead keys doesn't work. They insert a character as soon as the first key is pressed. So in MacVim I can't write áor ò. And single quotes like ' behave in unexpected ways.

[eirnym]

I have MacVim 161 and macOS 10.15 and typed Option-u-u to have ü (¨+ u). if first or second key is bonded to a specific action, you'll never get the right result. This is how Vim core works, it doesn't know what'll be next and it's hard to predict it.

[eirnym]

AFAIK usually editors doesn't have key-bindings for letters or alt-letters in macOS

[bighi]

This is not how Vim core works. Or maybe there's still a misunderstanding (on your part or mine).

In vim (terminal vim in Mac and Linux, and also gVim in Linux) I can use all the keybindings bound to ' (like jumping to marks) just fine, even though it's a dead key in this keyboard layout.

Just to make sure we're on the same page, we're not talking about Option-u. With dead keys, it would be " + u to insert ü.

I can even use ' as a binding in the insert mode, without losing the option to write á. That's because the ' is only actually inserted if I press ' and then <space>. Before I press space (or any other second key), the quote character is shown in a smaller font (or some other visual hint), to indicate it's waiting for the second keypress.

Except in MacVim.

I hope I could explain it better this time.

[eirnym]

first, you have to be sure, that this symbols are present in the font you type or exception ranges to type in another font

with Alt there's many hacks here and in many other lines in this file (mostly) to threat this as alt, not as special key

[bighi]

No special font at all. Works on every font, on every app except MacVim.

Also, telling me to change my layout and learn new key combinations to write after decades using US International just because of MacVim, is not a real solution is it? It's almost Apple's "you're holding it wrong".

There's something wrong MacVim is doing, but I don't know enough about the code to find out myself. If I had any indication of what I could do to help find out what is wrong, I would help. I am offering my time to help you debug this in any way I can, but I can't go into the code and look for the solution, unfortunately.

One example of wrong behavior, if it helps:

Try this using US International keyboard layout:

1) Create a new buffer. 2) Write this sentence: one two ümlaut "four" five 3) At the beginning of the line, try to move the cursor to the ü by pressing f " u in sequence.

Expected behavior: It does nothing when pressing ". After pressing u it would move the cursor to the beginning of the ümlaut word. Works on terminal Vim, also on gVim in Linux.

Actual behavior on MacVim: After pressing " it moves to the "four". The " didn't work as a dead key and didn't wait for the next keypress.

I've seen similar problems in a Linux app years ago. The app was capturing every single keypress, instead of waiting for the OS to alert of a new character. I have no idea if that's the case here, sorry.

But almost every person that write in a language other than English uses dead keys and are probably facing similar problems.

[eirnym]

with default font handling NSTextView changes font for each letter and symbol and there's many exceptions like you provided, like emoji, etc.

MacVim handles symbols rendering by it's own and don't send a symbol to Vim core processing before it is not "finished". NSTextView sends to a backend ¨, then u, then backend connect two symbols together, change fonts behind the scenes, etc. MacVim has no such processing with an exception for emoji and ligatures for rendering.

If we speak about ¨ (option-u + space) symbol-alike to make ü withoutoptionsequence, it's not a"symbol as you wrote above. So if I typef,"anduI'll getf"u, but neverüas I type withoption-u + u` in Safari where I write this comment.

When you enable mode rendering, MacVim writes, that I do IM INSERT, not INSERT for many such combinations. This processing is done by cooperative work between MacVim GUI and Vim Core. This way GVim works as well to insert such symbols. Terminal.app doesn't show this, because the symbol is processed before Vim core, and it's a problem to have some option/meta/alt combinations in Vim (terminal) in macOS.

[eirnym]

BTW, I type Polish every day and use to type option-l to type ł, for example. I have no English keyboard at all and macOS do it's spelling depending on a language I type. I use Polish Pro layout as it resembles qwerty for English with an addition with Polish letters.

The same I've seen in many other keyboard layouts. Probably, there some layouts have such problems that some character input requires type such combinations, but I haven't seen them. If you'll tell me which language you want to type, I'll check layouts for this specific language.

[bighi]

I believe there's still a misunderstanding.

I'm not talking about ¨, I'm talking about the actual " symbol, like the one you would use to define a string in lots of programming languages. In US International, you use "+u to write ü, like I've been saying from the beginning.

Please, don't be condescending about it. I know what key I'm typing and what symbols I'm inserting. Forget option+key at all, please.

I've been saying from the beginning that I'm using the US International layout. I've described the bug with expected behavior and actual behavior, to see if it helps. This is not a font thing, not a regular Mac thing, and isn't a problem with any other app I've tried on Mac OS before.

But I am unable to look at the code and fix it myself. I don't know where or how to start debugging it. But as I said, I offer my time to help however I can, if there's interest in fixing this.

Edit: When I change the keyboard layout to brazilian portuguese (just to test this bug), the dead key problem still happens, so it seems to be a problem with dead keys in Macvim, independent of the keyboard layout (maybe).

[eirnym]

I see. Keyboard preview shows these keys as additional keys to input other letters, like ¨, so we can speak about option-u and other such things as well (they . when I type using this specific layout f"u, I have fü at the end with seeing f""in between (I have a plugin for quote pairing installed). To insert really f"uI need to type f" u (but I can't input ¨ as a character using insert mode)

As I've pointed above, you should start looking into MMBackend.[ch] files and make breakpoints and debugging there. Probably, you could take this information from an active layout for better processing, but I don't know how.

[bighi]

Sorry, I wasn't clear on the bug description part. I meant to type f " u in normal mode. To find the character ü. What ends up happening is that MacVim finds " and then when you press u you undo the last change (instead of complementing "), which is not what the user wants.

[eirnym]

I see, it's hard and I don't think it's supported in GVim in Windows/Linux as well.

[bighi]

Now I'm pretty sure I'm being trolled or something. I don't understand why this propensity to call people liars for nothing. Just say you won't fix a bug that don't affect you and leave it at that. But dead keys are supported everywhere and have been for a long time.

So I'm just going to leave this here and move on: I recorded myself typing f " u in gVim in Linux. Also myself typing it in MacVim. Sorry about the quality, it's not easy to film and type at the same time. The video shows it working on gVim, so yes it is indeed supported.

Finding the ü working in gVim (under Linux) with dead keys: https://imgur.com/gallery/UM4qUSI

Finding the ü NOT working in MacVim (Mac OS) with dead keys: https://imgur.com/a/qQ7lF5w

So I'm just doing what everyone else that depends on dead keys does: moving on from MacVim. Good luck on calling people liars.

[eirnym]

@bighi I never told here anybody is a liar, I meant to share only my experience and what I've tested just before writing. I'm sorry, I haven't tested on a recent version version of GVim. Earlier it was definitely a problem with any non-US keyboard.

I've written that input in macOS has it's own difficulties and it's not very obvious how to handle it properly and where to start looking if anybody want to find out how to fix them.

[ychin]

Sorry all, I just read this thread as I have been busy and late to jump in.

@eirnym, this is a valid and solvable issue. I appreciate your help, but I think it's reasonable to submit a issue for this to be addressed and be discussed over.

This is not an Option/Alt key issue, but more how MacVim handles OS input methods. This is a somewhat similar issue to #837, where normal mode doesn't quite handle input methods properly.

@bighi / @ecerulm, can you try disabling the "Drawed Marked Text Inline" option and see if it works for you (also see https://github.com/macvim-dev/macvim/issues/837#issue-403037750)? I will need to investigate more and this has been on the back burner, but my feeling is we can just always disable that while in normal mode, but there could be other knock-ons from it as well, so I need to give it a little more thought about when to enable/disable marked texts and how to make it properly work with MacVim.

[ychin]

It's the only thing preventing me from using MacVim (and many others, since everyone on my company alerted me not to try).

@bighi Just out of curiosity, what's the rationale behind "many others" who alerted you not to try MacVim? It's fine if they don't like it but I would like to know the reason so they can be fixed.

[ecerulm]

@ychin I tried disabling the "Drawed Marked Text Inline" options and it did work!

So I use the following example text:

This text contain the swedish letter ö and the swedish letter ä but also " (the double quote symbol)

When I have Drawsed Marked Text Inline enabled if I press 0f"a it will jump to the " which is incorrect since in U.S International PC keyboard layout 0f"a means 0fä When I have Drawed Marked Text Inline disabled if I press 0f"a it will jump to ä in the text which is what is expected.

[ychin]

@ecerulm Cool. Good to know. I'll try to see if I can squeeze in a fix for this next update, but as I said I want to think a little more what the desired behavior is as magic options that you need to know to set is never fun.

[ecerulm]

@ychin when @bighi said

It's the only thing preventing me from using MacVim (and many others, since everyone on my company alerted me not to try)

I think he mean that just this issue with the dead keys is preventing him and many other people in his company to use MacVim, not that there are other many other issues.

Also to be fair, I should say that in the past I tried all the GUI options for mac and none worked like I wanted. For example, I reported this very same issue in dec 2017 to the VimR project (macOS GUI for neovim) and it still not fixed there yet.

It's good that @bighi resurrected this thread. Maybe I can finally jump from using vim in the terminal and go the GUI way.

[bighi]

Just came here to say that I tried that and it really did work. All my problems with dead keys seem to be gone.

Thank you, @ychin. You're great!

@bighi Just out of curiosity, what's the rationale behind "many others" who alerted you not to try MacVim? It's fine if they don't like it but I would like to know the reason so they can be fixed.

We have weekly tech talks where I work and after the presentation we hang out to chat. And lots of times it's about Vim. Some guys here tried MacVim before, and their complaints were usually about some rendering issues and the problems with dead keys.

As I'm new to Mac, I decided to try MacVim. We couldn't reproduce the renderings issues, so I'd guess they were solved and the guys didn't even know because they never gave it another chance. But the dead keys problem were still enough to prevent us from using it, because we all have computers with US keyboards, and we need to type in multiple languages, so using the US International keyboard is very important for us.

I'm gonna try and convince them to move back to MacVim.

[ychin]

I see. Yes the rendering issues should mostly be resolved with some remaining issues with scrolling performance not being buttery smooth (see #796). The issue took a little while to resolve especially when transitioning to Mojave (since they deprecated non-layered back drawing). I will look into the dead key issues but for now you can turn off marked text drawing although you probably want to change that back when this is fixed.

[g012]

Hello,

Any update on this ? I'm hoping for a fix for years ;) And I don't have the option to switch to another layout, I'm using an exotic French-Dvorak layout since 20+ years, not gonna switch ever again. I must point out that many years ago, there was an option related to that if I remember correctly - it was still not good, like it broke other stuff, but it made dead keys work. And overall, it worked way better for me that way than how it now is since long. Can't we have it back ? And for me it goes beyond dead keys some more : I can't CTRL-R neither. No idea why, it works everywhere else (all JetBrains IDE have no problem with IdeaVim input for instance).

Here's the layout : https://www.algo.be/ergo/v1/pilotes.htm

[ychin]

Sorry, no concrete updates yet. I have done the initial investigation and there is a release that will be focused on input handling so I could fix everything in one go (there are a lot of inter-related features including support modifyOtherKeys for MacVim, making Option key support not suck, dead keys, international keyboard / input fixes). Unfortunately, as it happens other features and higher-priority fixes keep creeping in and I still haven't pushed out release 175 yet. I was originally planning to do the input-focused release in r176, but for now I just renamed the release to be "Release 17X (Input)" 😅 to avoid setting wrong expectations (this is the release tagged with this issue).

I'm finishing up on r175 right now. If things go well and I don't see any other higher priority stuff to work on hopefully I can reassign the release back to r176. Originally I wanted to get overlay scrollbars in first but that was kind of annoying as well and was more a nice-to-have so will probably try to get input handling fixes first.

I must point out that many years ago, there was an option related to that if I remember correctly - it was still not good, like it broke other stuff, but it made dead keys work. And overall, it worked way better for me that way than how it now is since long. Can't we have it back ?

You mean set macmeta? It will make Option/Alt keys behave like a Vim meta key, i.e. it won't trigger the dead key. It's the current recommended fix for something like this for now…

[g012]

You mean set macmeta? It will make Option/Alt keys behave like a Vim meta key, i.e. it won't trigger the dead key. It's the current recommended fix for something like this for now…

No I don't think so, as I use '8' as dead key, this would have no impact for me. It was an option in the GUI menu, in preferences. But that's not important, if it's scheduled to be properly fixed in upcoming releases then that's great news ! Thanks for the hard work.

[ychin]

Oh, I think you probably mean the Draw Marked Text (default: on) and/or the Core Text Renderer (default: on) advanced options? They are still available, but both are a little buggy if you steer from the default. I think they may fix your issue, and I never removed them.