Terminal feature information schemes

[BrianInglis]

Looking for but not seeing environment variables or escape sequences that would query mintty and return the mintty version or startup command line. These would be useful in diagnosing issues - verify mintty being run - verify mintty version being run - verify mintty startup command line used. Any history for these kinds of metaqueries of mintty settings other than ENQ \005? Could export these in environment variables to the startup shell environment: could either be like MINTTY_VERSION=mintty 3.0.1 (x86_64-pc-cygwin)... MINTTY_LAUNCH=C:\cygwin\bin\mintty.exe - or like TERM_TYPE=mintty TERM_VERSION=3.0.1 TERM_LAUNCH=C:\cygwin\bin\mintty.exe -

[mintty]

Mintty serves the standard secondary device attributes request echo -e '\e[>c'. It identifies itself with terminal type 77 (ASCII 'M') and the version. ENQ is completely configurable and not really useful. Don't know its historic purpose but you could set up different distinguishable sessions. About invocation parameters, you'd use ps. About setting environment variables, please see the recent discussion in #866 and another issue referred there. I'm not in favour of that because it's not reliable to detect a terminal by environment. Didn't you contribute to the identification issue yourself somewhere recently? (#776)

[mintty]

About TERM_TYPE, it makes no sense for mintty to pioneer such an approach. As soon as xterm or any other major terminal provides that, mintty will do the same.

[nicm]

Was just looking for something like this to allow me to work out whether to send mintty extension escape sequences.

iTerm2 has a sequence \033[1337n:

$ printf '\033[1337n';cat
^[[ITERM2 3.3.20200403-NIGHTLYn

Which is bit more illuminating that device attributes and less likely to conflict with what another terminal might send. I have added it to tmux as well. Perhaps you might consider adding it as well? It would be nice if other terminals adopted it.

I know it would be best to get the extensions into terminfo, have better feature detection, etc, but sometimes it is nice just to be able to say "yes, this is mintty which definitely supports X". Or not having to guess or make the user configure it...

[mintty]

Full version information is available via a standard escape sequence, see my first comment above. I don't see the point in making this available via an additional proprietary sequence.

[nicm]

This use of DA2 is no less proprietary than a different escape sequence, and it relies on nobody else using M (not guaranteed), it is much harder to mixup a response like MINTTY. It would be nice if there was a better standard and iTerm2's is already out there. But it's of course up to you.

[mintty]

The numeric letter convention is used also by rxvt-unicode (U) and screen (S) but agreed, there is no guarantee. Have you tried to request this feature for xterm?

[nicm]

Yes... and tmux sets it to T also. But it is not used by iTerm2, or alacritty, or VTE, or many others. So I am already looking at DA to detect VT420 features, and at iTerm2's escape sequence to detect iTerm2, and now I have another method to detect mintty, so I'm sure you can understand it would be nice to reduce that :-).

I haven't suggested it to xterm - the only non-terminfo feature I need to detect from xterm is margins and I can get that from DA if it says the terminal a VT420. But the mintty extensions are currently mintty only - I can add an extension terminfo capability as well, but it is nice to be able to detect the terminal and turn it on so that it Just Works and every user doesn't need to configure it themselves.

I suppose I the best thing to do would just be to look at DA2 instead of DA, since the first parameter will tell me if it is VT420 and has margins, and I can then also detect mintty as well. If I ever actually need the features list I'll have to add it back but I guess that is unlikely...

[mintty]

Actually I don't even find that sequence documented anywhere. If you point me to a reference and request the same feature from xterm, I'll consider it.

[nicm]

Oh he hasn't got it on his extensions page, good point.

xterm is fairly easy to identify as it is and I don't have anything I need this for in xterm so it might be hard to sell, but maybe I'll give it a go.

I've switched to using DA2 for mintty in the meantime, the reason is that I want to add support for the alternative escape sequence; with luck some other terminals will pick it up because it is a feature that solves a genuine problem.

[BrianInglis]

CSI Ps n is another DSR defined as limited to Ps 5 terminal and Ps 6 cursor status reports. Iterm uses Ps 1337 (leet!) for its custom query code value! It would have been better had iterm followed a pattern similar to xterm and mintty CSI > Ps c DSR, used a custom query code value with CSI ? Ps n, or the unused sequence CSI = Ps n, to provide that information. Perhaps the best solution is NOT to use extension features, other than those supported by common standards for terminfo, xterm, and derivatives, as you don't need the extra info for xterm: don't cater to other terminals, report variations from the standard as bugs! Just say NO to most feature requests!

Perfection is achieved, not when there is nothing left to add, but when there is nothing left to remove. -- Antoine de Saint-Exupéry

[nicm]

Yeah, good luck with that.

[BrianInglis]

It's not that difficult if you present it in terms of development and support time and costs: an hour (or more) each for analysis (this discussion and your thinking time), development, testing and deployment, times your rate; and about the same each year for support as the environment changes: each minor feature will either lose all value when it becomes unused, or become an issue, as the systems, emulators, compilers, and other tools change.

[nicm]

I don't think you are going to get very far with that attitude and it will make most software much poorer.

[BrianInglis]

SE principles state more code => more bugs -- minimize the feature set to the essential and you reduce the bug volume and the attack surface -- the Unix design and development philosophy.

[avih]

Standard usually follow useful de-facto patterns. It's a chicken and egg thing, and it has to be adopted first before it becomes a standard.

The goal of the terminal ID in this case is so that tmux can automatically enable known and useful mintty features which are not yet standardized.

Specifically here - the application-ESC sequence which allows an application to request that ESC keypress is a sequence rather than a single \033, and thus it can avoid using timeout to differentiate between \033 which begins a sequence and one which is a key-press on its own.

I'm pretty sure that you (Brian) would appreciate the importance of this feature.

Additionally, while xterm is indeed many times the de-facto standard, and no one can take away from xterm and its maintainer the contribution to modern terminals, we (at the very least I) don't want that only xterm works everywhere out of the box, while other terminal authors have to either impersonate as xterm or explain to users why mouse or cursor-shape related things "don't work" in vim with their terminal but do with xerm, or a million other similar things - only because applications follow your (Brian) suggestion and support non-standard extensions only with xterm.

[mintty]

Specifically here - the application-ESC sequence which allows an application to request that ESC keypress is a sequence rather than a single \033, and thus it can avoid using timeout to differentiate between \033 which begins a sequence and one which is a key-press on its own.

Actually you could send this sequence to any terminal and simply accept the respective ESC input sequence; if the terminal doesn't know the setting, it will be ignored and you'll get plain ESC.

[nicm]

Actually you could send this sequence to any terminal and simply accept the respective ESC input sequence; if the terminal doesn't know the setting, it will be ignored and you'll get plain ESC.

We don't just send stuff and hope it will be ignored, that way madness lies - there needs to be some reason to believe it will do what tmux expects...

Although how flimsy the reason is does depend! A lot is based on the logic that any terminal with the XT capability must be pretty much xterm-like.

[nicm]

I might actually tie this to XT, like you say any terminal that supports xterm title sequences should ignore this and I don't really want to add a tmux-only extension capability anyway.

We only send DA2 if terminfo has XT so that means I don't even need to detect mintty for this (although I am using it to also turn on some other stuff like 256/RGB colour so people don't need to configure it manually).

[mintty]

terminfo has XT

?

[nicm]

XT is an existing extension capability in terminfo which means that the terminal supports xterm-style title setting (OSC 0, 1, 2). It is present in the xterm terminfo entry which xterm and mintty use and several others:

$ infocmp -x xterm|grep XT,
        OTbs, am, bce, km, mc5i, mir, msgr, npc, xenl, AX, XT,

tmux uses it for a few things as a general purpose indicator that the terminal is "xterm-like", following the logic that it can't have xterm title setting without either accepting or at worst ignoring a few other sequences that xterm or terminals similar to xterm support (like DA2, and focus events).

I think I will send the mintty 7727 extension also if the terminfo entry has XT, since (as you pointed out) I can be pretty safe in assuming that any "xterm-like" terminal will ignore it and mintty uses a TERM setting (xterm) that does have XT.

[mintty]

Thanks, I had forgotten about the -x option. But there's also a documentation gap here; I don't see XT in man user_caps.

[nicm]

It is documented in misc/terminfo.src in ncurses. It is actually a screen extension, but it has been around for years and years and is present in many entries (despite the later comment implying otherwise):

#       XT   (bool)  Terminal understands special xterm sequences  (OSC,  mouse
#                    tracking).

[BrianInglis]

Thomas Dickey keeps xterm definitions out of the standard ncurses lists; see last para also FAQ What $TERM should I use? and only provides them with xterm sources in terminfo (search for XT,) and termcap (search for :XT:) formats. Only the control escape sequences are documented in the [gnt]roff xterm sources and online. I think I first came across it while looking for man/less/lesspipe/lessfilter bold/italic/code terminal font support, that I remember we discussed, and used it to decide whether to request title escape sequence strings from tput for later use in profiles and functions.

[ThomasDickey]

I wouldn't phrase it that way. ncurses has a good-enough subset of xterm's terminal information. The parts not in ncurses are building-blocks which aren't used in any of the ncurses terminal descriptions (and occasionally someone notices something like xterm-mono, which I address).

Along those lines, ncurses and xterm are more in sync, say, than cygwin, mingw and msys2.

[BrianInglis]

I was trying to explain your stated position as closely as possible, without your explicit statement as to why you say you do not include xterm definitions in standard ncurses lists.

[mintty]

So maybe there could be an additional manual page like term_caps which lists terminal group specific capabilities, and it could be referred from the others (section SEE ALSO). That would pull the XT capability out a bit from magic knowledge.

[BrianInglis]

++ xterm_caps listing those excluded from ncurses with their types, properties, and values, as used in the utilities, and which tput obviously recognizes, along with other currently "undocumented" capabilities.

[ThomasDickey]

https://invisible-island.net/ncurses/man/user_caps.5.html#h2-SEE-ALSO

[mintty]

https://invisible-island.net/ncurses/man/user_caps.5.html#h2-SEE-ALSO

Those pages (tic, infocmp) describe the mechanism, but not the capabilities (XT in this discussion).

[ThomasDickey]

It points here - https://invisible-island.net/ncurses/terminfo.ti.html#toc-_N_C_U_R_S_E_S__U_S_E_R-_D_E_F_I_N_A_B_L_E__C_A_P_A_B_I_L_I_T_I_E_S

and gives the available information. Since it's screen's feature, you might want to pursue improved documentation with its developer -- all I can do is comment on how it appears from my viewpoint.

[avih]

@ThomasDickey off topic, just wanted to say thank you for the work and progress you've been doing all those years with xterm, terminals standardization, and maintaining related documentation, which serve as reference to so many.

[BrianInglis]

It would be good if those extra SEE ALSO paras and links made it into user_caps(5), and perhaps also the rest of the content in your ncurses user defined capabilities doc from the terminfo source, or perhaps another other_caps(5)?

[mintty]

Back to the iTerm2 terminal version issue, it just occurs to me that it uses a CSI response with non-numeric parameters. This violates all established conventions about usage of escape sequences. If at all, such a response would have to start with DCS or OSC.

[nicm]

It would be better as DCS or OSC but you don't get it unless you ask for it so I don't think it matters that much, the consistency boat has long sailed with escape sequences.

[ThomasDickey]

s/DSC/DCS/ (see XTerm Control Sequences).

By the way, the webpage is the preferred media because it provides good linkages (after that, PDF because the typography is clearest, with text-file last because it lacks both linkages and typography, and copy/paste from that often helps propagate misinformation).

[mintty]

I was checking whether to eventually close this issue as there doesn't seem to be much progress; but @BrianInglis, as we're having Thomas Dickey in the discussion, you might ask him about the TERM_TYPE environment variable originally requested here :)

[nicm]

I don't really mind what the escape sequence is, I'd just like something that tells me something useful and that multiple terminals offer. At the moment it is ridiculous, there are terminals claiming all sorts of things. I'd like to be able to detect more, even by just matching the terminal, so I can minimise users having to set options or mess with stuff that is complicated, they shouldn't have to understand and that they frequently get wrong. I end up spending a lot of time helping people who have problems with stuff that should really just work out of the box and it is frustrating for all concerned. There are a wealth of new and previously rarely available terminal features and development at the moment and people want to use them - I want to use them.

terminfo works and it is great that @ThomasDickey maintains it, but it (unavoidably) takes a long time to propagate. (Not to mention the fact that too many people misuse xterm* when they are not an xterm, so if for example one terminal is an actual xterm and the other is mintty or gnome-terminal or whatever - the application can't tell they are different.) I'd like it to be easier to bridge the gap between a feature being available in a terminal and it being worth having a terminfo capability and that making it out into the wild.

Anyway this is why I would like this, and it doesn't to me seem like a big request, but I understand why you have concerns :-). I can detect iTerm2 and mintty acceptably for the moment in any case.

[nicm]

A standard environment variable like TERM_TYPE, TERM_VERSION would work for me also if a few terminals support it, although I don't think it is as good as an escape sequence. We would need it to be added to ssh as well, perhaps they could be persuaded to treat it like TERM, I'm not sure how that would go down although I could ask the OpenSSH folks.

[mintty]

@nicm, I understand your concerns very well; I'm doing a lot of terminal-specific behaviour tuning in my text editor mined. I am also willing to adapt something for mintty, with two conditions: it should be well-designed, not hacky, and I do not wish to pioneer on such a feature.

[ThomasDickey]

TERM_TYPE would have the same effect as TERM (developers would abuse it).

[mintty]

I think the idea of TERM_TYPE is that it should not refer to terminfo, thus no need to abuse it. Developers who abuse it anyway should be bashed... The other disadvantage of an environment variable, as I objected before, is that it propagates to instances of other terminals that do not overwrite it.

[nicm]

it should be well-designed, not hacky, and I do not wish to pioneer on such a feature.

OK, fair enough. I can't really usefully pioneer it however, we need a few people to agree I guess.

The other disadvantage of an environment variable, as I objected before, is that it propagates to instances of other terminals that do not overwrite it.

Yes, this is a problem, but because it is standard it is more likely to be overwritten and easier to manage than the likes of VTE_VERSION, XTERM_VERSION and so on. OpenSSH not passing it through is a bigger issue I think.

[nicm]

TERM_TYPE would have the same effect as TERM (developers would abuse it).

Perhaps you are right, if someone writes a new terminal or adds a feature, the easiest way to get existing applications to use it is to pretend to be a terminal that they already know about. Maybe this is not a problem that can be solved, in the end it just means trusting people to accept that TERM_TYPE should be their terminal name, even if they like TERM to pretend it is a different terminal...

It's a pity that more features aren't advertised as part of the feature flags in the device attributes response.

[nicm]

There are long discussions of this here (which frankly I had forgotten about):

https://gitlab.freedesktop.org/terminal-wg/specifications/-/issues/7

With some talk of using DA3, although I don't think anyone ever did. Also some here:

https://gitlab.freedesktop.org/terminal-wg/specifications/-/issues/8

I was hoping a few more terminals would pick up iTerm2's reporting (or anything really) which would be an immediate help, but I definitely understand why you don't want to. Perhaps something will come along eventually.

[mintty]

Just as an idea, the protocol could be a variation of the existing secondary device attributes, with a parameter value 1 rather than 0. So a request CSI > 1 c could get a response DCSname version. Note, however, when any such new control is introduced, you'll get a timeout issue with terminals that do not respond.

[ThomasDickey]

Most of the interesting features are associated with a given level in the secondary DA, but again there are developers who routinely abuse that.

I don't see a pre-existing use of CSI > 1 c, except that it could be confused with the DA2 from VT220 if one only looks at the level.

[nicm]

Most of the interesting features are associated with a given level in the secondary DA

The problem is this only accounts for VTxxx and xterm features, it doesn't let me detect new features in some terminals, things like cursor styles, RGB colour, overline and underline styles, and so on. Probably more in future. And you're right that some terminals do get it wrong.

I think CSI >1c with a DCS response would be great.Timeouts are no more or less an issue that with existing proprietary escapes like iTerm2's.

@egmontkob @gnachman @chrisduerr would you add something like that if we can agree the response form? Or other thoughts?

[BrianInglis]

ISTM support for those features is already there in ncurses for programs and tput for scripts: the real problem is the documentation volume; but all that info is already available to the scripter or programmer, collated into the following docs:

• terminfo source
• XTerm Control Sequences
• ncurses FAQ.

[BTW the Cygwin console running cmd returns C/67;MAJOR MINOR;0 to DA2 when another emulator is not responding to DA2 or running. Debian Linux 5.4.0-2 responds 65;5402;1. I don't know about WSL or Windows Terminal emulator, as I use mintty shortcut variants due to beating head against wall shudder issues, and even as my local Windows X terminal client.]

[nicm]

terminfo now has extension sequences for some but that doesn't help users with older systems which haven't yet upgraded their terminfo database. I agree that having terminfo entries eventually is nice but I'd like this for the gap before a new entry is widely available.

As you have found, DA and DA2 are not a reliable way to work out the terminal. This is why we are suggesting a new sequence.

[gnachman]

iTerm2 has such a mechanism: https://github.com/gnachman/iterm2-website/blob/master/source/utilities/it2check

More simply:

% echo -n '^[[1337n'
^[[ITERM2 3.3.9BETA1n

To use it safely, first send this code and then send DSR 5. If you only get a response to the latter, you know the terminal doesn't support it. Letters in the response are all upper case to ensure it won’t contain an n.