Terminal hardware is always left in application mode (keypad_xmit) after running a console program

[kkm000]

This problem has been annoying me for quite a while, and I worked around it by emitting the rmkx terminfo sequence in my bash prompt. But I still think it makes sense to report it. Basically, after (almost?) any dotnet command that involves CLR console I/O, the terminal emulator is left in the application mode (keypad_xmit), messing my command line handling (e. g. arrows move by word <ESC> O C, not by character <ESC> C, in readline bash prompt, etc. -- probably because of my customized .inputrc, designed to work with both VT100 and rxvt style emulators). Other programs behave nice; e. g., just typing man man, or invoking less or vim and then exiting them reverts keyboard to local keypad mode. Just for reference, this is a hopelessly headless Ubuntu 18.04 machine:

$ uname -a
Linux yupana 4.15.0-29-generic dotnet/corefx#31-Ubuntu SMP Tue Jul 17 15:39:52 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
$ echo $TERM
xterm-256color
$ dotnet --version
2.1.403

I noticed by using a script(1) dump that the smkx aka keypad_xmit code is emitted more than once, but then there is no reverting rmkx aka keypad_local emitted ever. Here is, for example, how captured prologue of dotnet build looks (not marking literal linefeeds), that emits keypad_xmit twice:

Script started on 2018-10-12 23:02:55-0700
. . .
dotnet build -c Release -v:n
<ESC>[?1h<ESC>=Microsoft (R) Build Engine version 15.8.169+g1ccb72aefa for .NET Core

Copyright (C) Microsoft Corporation. All rights reserved.

<ESC>[?1h<ESC>=Build started 10/12/18 11:00:40 PM.

I just attempted to trace the issue down by eyeballing the console handling code. My terminal emulator is pretending to be xterm-256color compatible (MobaXTerm), and infocmp(1) does show entries for both sequences for this term type:

rmkx=\E[?1l\E>
smkx=\E[?1h\E=

The terminfo entries have manifest numeric IDs in `/usr/include/term.h

#define keypad_local                   CUR Strings[88]
#define keypad_xmit                    CUR Strings[89]

but the index 88 is not even in the enum WellKnownStrings in TermInfo.cs file that reads the terminfo file directly (but KeypadXmit = 89 is). Also, it seems that pal_console.c does a good job of restoring the application mode at a few points by sending the keypad_xmit sequence, apparently since the MR dotnet/corefx#6488 fixing the issue dotnet/runtime#16300, but unless I misunderstand it, never attempts to send a matching keypad_local in any of its Uninitialize() functions, where tty driver attributes are restored.

[danmoseley]

Thank you for the detailed investigation, @kkm000. Do you want to propose a fix in a PR?

[kkm000]

@danmosemsft, a fair question! I do not know, really. If I had the whole thing checked out and churning and able to test, I'd implement and send a fix in an hour. The missing piece looks simple to integrate, would touch only internal type signatures and use already present signal-safe routines in the native shim (I hope I use the correct term).

The big deal is I know practically nothing how the whole stack works. I just found this repo by searching for a type named "ConsolePal" that popped up in a stack backtrace that a c# program using Console.ReadKey() so helpfully puked when I invoked it with "</dev/null". It gave me a hint where to look for a low-level library responsible for tty I/O, that's it. The rest was just some common sense and a bit of reading the code.

I mean, I could certainly try to check out and build the repo; I routinely develop in C++ on both Windows and Linux and C# on Windows. But I have no idea what would I do with the libraries that I build. I know little what is under the hood of the "dotnet" command. I quickly read the documentation on building an testing, but I do not understand how I could incorporate what I build into the rest of the stack. Do I also need to build the coreclr repo, or can I run an application with my own build of corefx but the existing RTM coreclr runtime installed from a .deb? If you could point me to any docs or explain that in a few words, I would certainly give it a try.

[stephentoub]

@kkm000, thanks. Is there a way to detect whether the current setting is already keypad_xmit? It should be fairly straightforward to fix up the uninitialize routine to output keypad_local, but that could itself cause a potential issue if the terminal were already set as keypad_xmit when the process was launched, as we'd then be switching away from what the environment described.

[kkm000]

@stephentoub:

Is there a way to detect whether the current setting is already keypad_xmit?

I do not believe there is, and most probably not without running into another timeout problem like dotnet/runtime#27034. But it's not really necessary. For all I know, no sane program would assume that the terminal is in the keypad mode when it starts. And I know for sure that bash (rather readline) assumes it is not :(

My understanding is that the "local" mode is the default, and it's always ok to revert to it. vim does this unconditionally upon exit (here T_KE corresponds to yet another name for keypad_local, the termcap ks/ke pair). less does it pretty much always as well (when did you last type less --no-keypad? :) ).

vim documentation has a little bit on its keypad mode control.

[rmunn]

@stephentoub wrote:

Is there a way to detect whether the current setting is already keypad_xmit? It should be fairly straightforward to fix up the uninitialize routine to output keypad_local, but that could itself cause a potential issue if the terminal were already set as keypad_xmit when the process was launched, as we'd then be switching away from what the environment described.

My research into terminfo shows no way to query the setting; I agree with @kkm000 that the standard thing to do in Unix world appears to be to assume that "normal mode" (keypad_local) is the default, and if you set "application mode" (keypad_xmit) because you're a full-screen program that wants the application-mode escapes for cursor keys et al, then you're expected to set "normal mode" (keypad_local) before you exit. I believe this behavior of vim and other similar software was responsible for https://github.com/dotnet/runtime/issues/16300#issuecomment-187451220, which you fixed in https://github.com/dotnet/corefx/pull/6488.

If you want to be extra cautious, you could possibly save the value of Console.IsOutputRedirected from EnsureInitializedCore in ConsolePal.Unix.cs and only restore keypad_local if you sent a keypad_xmit code in the first place. That would probably be the closest you can come to checking the current terminal mode and restoring the same mode at the end.

[leo60228]

Is there any progress towards fixing this?

[leo60228]

This bash script detects whether DECCKM is enabled in xterm. It doesn't work in Konsole, though.

#!/bin/sh
exec </dev/tty
old="$(stty -g)"
stty raw -echo min 0  time 5
printf '\033[?1$p'
read status
stty "$old"
reply="${status#$'\033'}"
case "$reply" in
    '[?1;1$y')
        echo "DECCKM on"
        ;;
    '[?1;2$y')
        echo "DECCKM off"
        ;;
    *)
        echo "unknown"
        exit 1
        ;;
esac

[fabriciomurta]

In fact, if after a .NET console command I do this: echo -ne "\033[?1l"

The broken behavior is fixed (in my case customizations in ./inputrc are being ignored after a console app is run).

Unfortunately I cannot just avoid this by Console.Write("\u001b[?1l"); right before the app exits, as yet another [?1h will get printed to the output. (right before closing stdout?)

Here's the capture of the default dotnet new console; dotnet run terminal output:

^[[?1h^[=^[[?1h^[=^[[?1h^[=^[[?1h^[=Hello World!^M
^[[?1h^[=

In case I tried to return to local mode from within the program I would just get:

^[[?1h^[=^[[?1h^[=^[[?1h^[=^[[?1h^[=Hello World!^M
^[[?1l^[[?1h^[=

(1st sequence in 2nd line)

Thus, no good.

[kkm000]

@fabriciomurta, no, it's likely not possible to reset the mode properly from within the application.

My solution has been just to add the reset sequence to PS1. My .bashrc has quite a layer of helper functions to build the prompt at a high level and in a terminal-independent way, so that I can colorize it and add markers based on environment (WSL, clouds, inside Docker...), but after unpacking it, I recover this:

# 'dotnet' leaves keyboard in bad mood.
PS1+="\[$(tput 2>/dev/null rmkx)\]"

It is important to add the \[ and \] brackets around any non-rendered sequences (those not shifting the cursor position), so that Bash does not count them as having any representable length and can correctly compute the column position of the cursor at the end of prompt, otherwise readline will be very confused and angry at you. tput, which is part of ncurses/terminfo distribution, simply outputs the rmkx sequence for your terminal set in TERM, so it's stored in the command prompt, resetting the keypad transmit mode after each interactive command.

kkm@buba:~$ echo $TERM
xterm-256color
kkm@buba:~$ tput rmkx | xxd
00000000: 1b5b 3f31 6c1b 3e                        .[?1l.>

I advise against hardcoding handwritten CSI control sequences into PS1: it will come back at you sooner or later, e.g. in Emacs, in tmux or in other situations of double-emulation, where the actual terminal might not support them. tput is a bulletproof way to get a correct sequence (empty if not supported) provided that the terminfo entry is correct.

The only caveat is that support for RGB terminals in terminfo ("direct" color in their parlance) is flaky, and, e.g, both konsole-direct and xterm-direct lack the rmkx definition. If using direct RGB color terminal, your best bet is compiling your own definition for the emulator that you are using. There are worse problems in the distro, even the latest one (e.g., pretty nonsensical setaf computation). See man terminfo, infocmp, tic.

[fabriciomurta]

Wherever .NET (System.Console?) outputs the terminal code to set a mode there should be a way for it to, likewise, output the terminal code to reset this mode (vt100 docs, scroll down to "modes", where there is "Cursor Key Mode" and "Keypad mode")... just like it sets it.

Maybe just signal if it output the "set" sequence, on program shutdown, output the "unset" one. In what I could find, it does not just to keypad mode, but also cursor key mode, so both should be reset: ESC [?1l and ESC >.

There should be a counter for this setKeypadXmit code. Something to play last when a console application is freeing resources to terminate...

[kkm000]

@fabriciomurta, if you read the whole discussion, you'll note that this has been said before.

My impression is that @stephentoub generally agrees (https://github.com/dotnet/runtime/issues/27626#issuecomment-429888748), and it's likely that a good-written PR would be accepted. @stephentoub, what's your word on this? To recap, we've already established by eyeballing the sources that both less(1) and vim(1) blindly send terminfo keypad_local = rmkx, née termcap ks, upon closing the terminal.

[adamsitnik]

Please excuse me if this is a stupid question, but I am new to Terminals: why do we need to enter this mode? What do we get by doing that?

[leo60228]

It allows distinguishing between keys on the main keyboard and keys on the numpad, which System.ConsoleKey does.

[Frassle]

Just linking https://github.com/dotnet/sdk/issues/15243 here as I think these duplicate each other.

[odalet]

Just bumped into this one after installing .NET 8 on Linux.

TLDR; There seems to be a regression in .NET 8 as I never witnessed this behavior in .NET 6...

My particular repro combination is:

• WSL - Debian 12 / Ubuntu 22.04
• Inside a Windows Terminal 1.20.10303 on Windows 10 22H2
• and using bash (no problem when using Linux's pwsh)

Any dotnet ... commands leaves the terminal with a non-usable keypad

NB:

• did not happen with .NET 6
• does not happen when I launch my WSL instance in a "cmd" terminal (I guess this is because it barely understands escape sequences)
• did not test on a real Linux box, but I expect it would fail as well

Shoutout: the PS1 workaround by @kkm000 works in my case. Huge thanks to him as, before applying his trick, all I knew to do was reset the terminal after every dotnet command...

[rmunn]

For what it's worth, I have NOT experienced a regression under .NET 8 on a real Linux box (Linux Mint 21.3, which is basically Ubuntu 22.04 with a different set of window manager packages, none of which should affect terminal behavior). Running dotnet commands leaves my terminal in a working state, whether they exit normally or are interrupted by Ctrl-C. Here's the first few lines of dotnet --info on my system:

rmunn@laptop:~$ dotnet --info
.NET SDK:
 Version:           8.0.101
 Commit:            6eceda187b
 Workload version:  8.0.100-manifests.ba313bcd

Runtime Environment:
 OS Name:     linuxmint
 OS Version:  21.3
 OS Platform: Linux
 RID:         linux-x64
 Base Path:   /usr/share/dotnet/sdk/8.0.101/

I know that .NET SDK 8.0.201 is out, but I don't have it installed yet. If it causes a regression, I'll post another comment.

[odalet]

@rmunn Interesting, this would then be specific to Windows Terminal? I'll try this on my Linux machine when I have time as well!