Left/Right keys always behave like Ctrl is pressed in rxvt-unicode / urxvt, even if it isn't

[grandchild]

If I press left- or right arrow, it always jumps back/forward a word, even if the Ctrl key is not pressed.

This commit (daac3c8) added the \eOD and \eOC keycodes to key_info['ControlLeft'] and key_info['ControlRight'] respectively.

Removing the keycodes from the list fixes this issue.

I use rxvt-unicode on Archlinux if that's of any relevance. I have my Capslock key disabled and remapped to Meta/Super (well... the Windows key, you know :smirk:), and I use an en_US standard layout.

Maybe you want to revert that part of the commit?

[ericbn]

I'm good with removing those codes, as they have no use in iTerm2. Also, I noticed other frameworks don't have those either.

@Eriner, you added those. They still make sense to you?

[Eriner]

TBH it's been so long that I have no recollection of adding that (or experiencing the issue that caused the addition). Something something... key-codes vary from one terminal to another. My reservation in removing it is only that I probably only would have added it if it fixed a bug of the key-codes not being properly detected, as stated in the commit message.

@grandchild is rxvt-unicode the only terminal on which you experience this issue? And does it occur in a tty?

[grandchild]

Ah, that is a test I could have done before, yes...

Anyway here come the test results:

Terminal	with \eOD/\eOC	without \eOD/\eOC
urxvt	❌ →/← skips whole words	✔
xterm	✔	✔
tty (agetty)	✔	❌ Ctrl+→/+← moves one char only

So yeah, the question is how to detect this and do something special for either TTY or urxvt (possibly others?)

[grandchild]

❯ echo $TERM
rxvt-unicode-256color

And my tty is agetty, in case that matters.

[ericbn]

Found this table that shows \eOD as kcub1, \eOC as kcuf1, \e[1;5D and \eOd as kLFT5, and \E[1;5C and \eOc as kRIT5.

Maybe the fix in that commit was because the actual bindkey calls for backward-word and forward-word were also added.

EDIT: Now I see the recent comments above. Another possible fix could be to only bind \eOD and \eOC when on tty. Not sure how to test that.

[grandchild]

Ah. And that same table shows rxvt mapping \EOD to kLFT5 (keypad?). I guess that's that.

Wait, is there a difference between \eOD and \eOd?

[grandchild]

On TTY:

❯ echo $TERM
linux

o.O

[grandchild]

I think I'd go with adding \eOD/\eOC only when not $TERM.startswith("rxvt") (sorry, don't know enough bash, hence a bit pythonic)

Since most other terminal emulators from the table linked above don't do what rxvt does, it seems special-casing rxvt* would be better than special-casing tty and relying on xterm to keep working in both scenarios.

[ericbn]

Replacing those two lines in the init.zsh file by

    'ControlLeft'  "${terminfo[kLFT5]}"
    'ControlRight' "${terminfo[kRIT5]}"

worked for me on the macOS iTerm2 and Terminal, and on the Windows WSL terminal.

In fact, all of these terminals use the same codes:

% print -n ${terminfo[kLFT5]} | hexdump -C
00000000  1b 5b 31 3b 35 44                                 |.[1;5D|
00000006
% print -n ${terminfo[kRIT5]} | hexdump -C
00000000  1b 5b 31 3b 35 43                                 |.[1;5C|
00000006

@grandchild, @Eriner, can you please try that?

EDIT: You can also run cat and then actually press CTRL-LEFT and CTRL-RIGHT to double check:

% cat >/dev/null
^[[1;5D^[[1;5C

[grandchild]

I simply ran ${termininfo[kRIT5]} (which is not a command, but the error message (unlike echo) contains the sequence.

Then I ran cat and pressed Ctrl-Left and Ctrl-Right.

These are the results:

urxvt:

❯ ${terminfo[kLFT5]}
zsh: command not found: ^[[1;5D
❯ ${terminfo[kRIT5]}
zsh: command not found: ^[[1;5C
❯ cat
^[Od^[Oc

xterm:

❯ ${terminfo[kLFT5]}
zsh: command not found: ^[[1;5D
❯ ${terminfo[kRIT5]}
zsh: command not found: ^[[1;5C
❯ cat
^[[1;5D^[[1;5C

tty/agetty:

❯ ${terminfo[kLFT5]}
zsh: command not found: ^[[1;5D
❯ ${terminfo[kRIT5]}
zsh: command not found: ^[[1;5C
❯ cat
^[[D^[[C

[grandchild]

Replacing those two lines in the init.zsh file by

    'ControlLeft'  "${terminfo[kLFT5]}"
    'ControlRight' "${terminfo[kRIT5]}"

This had the following results: terminal	Ctrl-Left/-Right & Left/Right behaviour
urxvt	:disappointed: Ctrl-Left/-Right doesn't do anything anymore
xterm	:+1: Both Ctrl-Left/-Right and Left/Right work as expected
tty	:disappointed: Ctrl-Left/-Right go char-by-char

[ericbn]

This is looking like a puzzle now. :- )

Okay, we can definitely forget about trying to use ${terminfo[kLFT5]} and ${terminfo[kRIT5]}.

This is what I get for LEFT and RIGHT in the 3 places I'm testing (macOS iTerm2 and Terminal, and Windows WSL terminal), which are all xterm:

% ${terminfo[kcub1]}
zsh: command not found: ^[OD
% ${terminfo[kcuf1]}
zsh: command not found: ^[OC
% cat >/dev/null
^[[D^[[C^C

It does not even match, but the keys work because Zsh binds ^B and \e[D to backward-char, and ^F and \e[C to forward-char by default in emacs mode.

@grandchild, for completeness can you please post what you get for the commands above (doing LEFT and RIGHT after cat)...?

[ericbn]

Created a script to help checking if the keys in the terminal differ from the ones defined in terminfo: test_keys.zsh

I think a good plan is to hardcode the exceptions by adding something like

  case ${TERM} in
    rxvt*)
      key_info[ControlLeft]='^[Od'
      key_info[ControlRight]='^[Oc'
      # ...
      ;;
    xterm*)
      key_info[Left]='^[[D'
      key_info[Down]='^[[B'
      key_info[Right]='^[[C'
      key_info[Up]='^[[A'
      key_info[End]='^[[F'
      key_info[Home]='^[[H'
      ;;
  esac

[grandchild]

It seemed to me it'd be enough to simply don't set \eOD/\eOC on case ${TERM} in rxvt*).

[ericbn]

Looking better at this, I found that actually other keys are not properly mapped when based on terminfo, as you can see above. At least for xterm, I consistently found that the Left, Down, Right, Up, End and Home keys need to be overridden with other values, when checking in different systems with xterm...

Can you please run test_keys.zsh in your different terminal emulators, so we can check the outputs?

[grandchild]

Sure: https://gist.github.com/grandchild/6a89e0f126f8d14d13c31740156f1195

"Meta" and "BackTab" were not clear for me what they mean, I pressed possible candidates, but in case of "Meta" these where different for different terminals because TTY and my WM both have hotkeys that interfere with some of the interpretations I chose.

[ericbn]

@grandchild, MetaLeft and MetaRight are Alt-Left and Alt-Right respectively. And BackTab is Shift-Tab. We should most probably update these key names to be clearer.

EDIT: You got it right! :- )

EDIT2: I see there's TERM=xterm in all the outputs. It this really what you got? As a note, I've seen keys break if you have bash as the default shell, and then run zsh on top of it, because some OSs have some default settings in zprofile or bashrc that conflict with zsh...

[grandchild]

ah, damn, I have TERM=xterm in my .zshrc, because things complain because rxvt-unicode-256colors is not something many things are aware can be a terminal... I can try again tomorrow, with that removed, for maybe different results.

alt+left/+right will not work in tty mode because they switch to the previous/next tty respectively. I'm not sure that can be even configured away...

[ericbn]

Okay, after digging into this I learned some things:

1. terminfo key definitions should only work when you're in "application mode", which we're setting up here.
2. Testing a key code with cat is done outside of "application mode", so it does not show what we expect. The right way to test it is by pressing CTRL-V followed by the key, because "application mode" is only enabled during zle (Zsh command line editing).
3. \eOD (which is defined in ControlLeft) is the common value of {terminfo[kcub1]} (which is Left), but the latter overrides the former because we do the bindkey call for Left after the ones for ControlLeft. The result of this can easily be seen by doing:

   % bindkey "^[OD"
   "^[OD" backward-char

   So the problem that @grandchild reported is most probably caused by the value of {terminfo[kcub1]} not being \eOD in his terminal. Maybe this is because he has replaced TERM=rxvt-unicode-256colors by TERM=xterm. Same applies for the respective \eOC/ControlRight/Right situation.

So I propose that the short fix should be removing the \eOD and \eOC from ControlLeft and ControlRight.

The longer fix should be only using the terminfo key definitions if the "application mode" definitions are available. It does not make sense to try to use all these otherwise.

[grandchild]

Oh wow, have I got egg on my face now! :disappointed:

Removing export TERM=xterm from my .zshrc fixes the issue, without any changes to modules/input/init.zsh.

I am so sorry about this days-long hunt for nothing...

So I propose that the short fix should be removing the \eOD and \eOC from ControlLeft and ControlRight.

This doesn't seem necessary now, and furthermore, removing it outright breaks Ctrl+Left/+Right in the TTY.

[grandchild]

Just for completeness and posterity, I have replaced my infamous .zshrc line by this:

if [[ "$TERM" == "rxvt-unicode-256color" ]]; then
    export TERM=rxvt
fi

(Yes, it's "256color" not colors, like I wrote above.)

[ericbn]

@grandchild, good to know that solved the issue. I still believe we can simplify the code by using ${terminfo[kLFT5]} for ControlLeft, and other similar key definitions we're missing.

I've updated the test_keys.zsh script to try to set the application mode as expected, and to give better output.

Here is the output I got in macOS, android and lubuntu, in different terminals. In all cases, terminfo got the right values for AltLeft/AltRight/ControlLeft/ControlRight.

The only 2 issues I found are that:

1. Backspace inconsistently varies between different xterm instances I used.
2. The lubuntu tty has no definitions for AltLeft/AltRight/ControlLeft/ControlRight and pressing ControlLeft/ControlRight actually returns the same escape code as for Left/Right.

But still these are not even covered in the current code.

[grandchild]

Sorry, for the delay. Here you go: https://gist.github.com/grandchild/6ea1a8cc11c94c65602ee140f510df7a

[ericbn]

Thanks a lot, @grandchild. I see we got similar results. I'll propose a Pull Request based on our findings here.

[grandchild]

~I tried to find out what kind of TTY Lubuntu is using, but... wow, their docs site (https://docs.lubuntu.net/) shows an error in debug mode, the twitter account https://twitter.com/lubuntudesktop/ is suspended, the https://github.com/lubuntu-dev/docs.lubuntu.net repo has a DMCA takedown on it.~

~Just out of curiosity, @ericbn , do you know what's going on there?~

Ah, never mind, so lubuntu.net is an impostor site, and lubuntu.me is the original? Holy hell...

[ericbn]

Yeah, I was about to comment that https://lubuntu.me is their site. No idea what's going on in there with the other site and social media accounts.

Regarding the current issue here, I'm almost giving up trying to change the current way we bind the Ctrl-Left/Ctrl-Right keys. The current code is a kind of brute force way, but anything that relies more on terminfo might not work in some out of so many different scenarios. So maybe better not touch what's already working.

[grandchild]

Yeah, I also agree, that while it was a very thorough bug hunt :wink: and now not to change anything can seem... frustrating, it seems like the way it is now works probably best (if some people \<cough> wouldn't screw around with $TERM :smile:)

[grandchild]

I :ballot_box_with_check: to close

[Eriner]

I probably only would have added it if it fixed a bug of the key-codes not being properly detected

Sometimes shells are messy, and the joy is in the journey ;)