Feature Request: Clients of terminal need a way to inquire about the capabilities of the terminal

[rkeithhill]

Summary of the new feature/enhancement

Maybe this is as simple as an environment variable like VSCode uses for its terminal e.g. TERM_PROGRAM=WindowsTerminal / TERM_PROGRAM_VERSION=0.0.1.0.

Proposed technical implementation details (optional)

Start by creating the environment variables above so those of us using PowerShell can adapt our user experience (via PowerShell profiles) to Windows Terminal.

[WSLUser]

@zadjii-msft Might Unibilium possibly be of help for this?

[DHowett]

That license might make it a hard sell.

[WSLUser]

Perhaps a refactor that's Windows specific could be used with a new license (including switching to C++ from C). I filed an issue as well but I think it's not likely for them to change it. But who knows, it's a new year, maybe the neovim maintainers will be generous enough to remove that barrier.

[zadjii-msft]

That doesn't seem like it would be useful by itself. Couple thoughts:

• I don't totally remember how GPL works, but I'm under the impression that if someone wanted a refactor to have a different license, it would have to not use any of the original code. In that case, is it even the same lib anymore?
• I'd expect that this would still require OS-side work to support terminfos in Windows - which there is no support at all currently.

So I'm gonna say probably no.

[tracker1]

@zadjii-msft The open modifications would only need to be part of the library itself as Lesser GPL (LGPL), and doesn't infect upstream projects. Not sure if/how it would work in terms of how Terminal is made. And I know MS is adverse, as we all are to GPL work, though LGPL is more reasonable.

[jamestalmage]

@heaths Feature detection absolutely is necessary. supports-hyperlinks is a widely used module in the Node.js community for just that purpose. For formatted output (like tables) it's an absolute necessity. For output like that of this module (implementation here), the author is explicitly opting to just not show urls when hyperlinks aren't supported, because their added length would clobber the output. This comment explains the need further.

I'm not sure what portion of the 9 million weekly downloads of supports-hyperlinks are Windows Terminal users, but they're all currently missing out on hyperlinks even though they're supported by their terminal.

[tracker1]

I understand the aversion to avoid stuffing environment variables, and that proper capabilities detection is preferred, the old saying of, "Don't Let The Perfect Be The Enemy Of The Good" applies here.

TERM_PROGRAM and TERM_PROGRAM_VERSION are already widely used fields, and given how muddy TERM has been, where most terminals are simply xterm-256color or similar, trying to "fix" that will likely be more disruptive to existing applications. Terminals evolve, and as bad as it is, knowing the program itself and the version, and being able to do this in a consistent way are more important than trying to create a better solution.

Capabilities lookups may indeed lag here... and even then, there are often cases where a very specific point release of an application may have issues... Broken list api in IE 5.0.0, or broken JSON parser in IE 8.x for example. As developers we need to be able to work around these issues, and not knowing definitively the application and the version in a consistent way is just plain bad.

I know there is WT_SESSION okay, now we know the terminal in a completely different way than any other terminal in existence. And we still do emphatically NOT know the version. I cannot for the life of me figure out why this feature request is so controversial or why it has sat here for the better part of two years. This could easily have been a solved problem with those affected able to make their adjustments and move on by now.

Tagging: @DHowett-MSFT @zadjii-msft

[orta]

In case you want more concrete examples of needing this, I'm midway through adding some hyperlinks in the output from the TypeScript compiler but can't reliably support Windows Terminal without being able to do a version check from inside the process (for other terminals I use TERM_PROGRAM_VERSION.) Given that the support is relatively new and there's probably a non-trivial number of older installs around, we'd end up giving them a bad experience.

[heaths]

@orta depending on how you're linking, you may not need to check. See my comment above. That may not work in all cases, though; and, don't get me wrong, I 💯 agree we need some way of checking TERM support, though I understand from this thread there's no good standard to implement. Cue xkcd's comic on inventing a new standard!

[axelfontaine]

WT_SESSION also doesn't work reliably in Win11 x64 with Windows Terminal set as default.

To reproduce, press the Windows key to open start, then type cmd + enter

This now opens a Windows Terminal cmd.exe tab via %USER_PROFILE%\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\System Tools\Command Prompt.lnk where WT_SESSION is not set.

[zadjii-msft]

WT_SESSION also doesn't work reliably in Win11 x64 with Windows Terminal set as default.

This is a great example of why I wasn't in love with the env var solution in the first place. It's never going to be totally reliable.

The defterm scenario is never going to work with WT_SESSION. When the Terminal is invoked for a defterm connection, cmd.exe is already running. At this point, it's environment variables are already set. It's started, so there's nothing WT can do to change them now. Then, the OS tries to create a console to host the cmd process. conhost does some work, and hooks up the cmd.exe to the Terminal (or some other terminal!). The Terminal can only add the WT_SESSION variable when WT is the parent, launching cmd.exe directly.

[garyo]

This is a great example of why I wasn't in love with the env var solution in the first place. It's never going to be totally reliable.

OK, so env vars won't work in all cases. Is there another option? Perhaps Windows Terminal could ship with a small exe that, when run, checks how it's being run (parent processes, process groups, or some other OS method) and returns on stdout a JSON list of capabilities, version number, etc.? Just trying to be creative here.

[j4james]

Exactly which capabilities do you need to detect? There should already be standard escape sequences for detecting most features that apps might be interested in. We don't support all of those sequences yet, but that's something I'm hoping to get to eventually.

[axelfontaine]

@j4james ANSI escape code and emoji rendering support.

[Jaykul]

This is a great example of why I wasn't in love with the env var solution in the first place. It's never going to be totally reliable.

OK, so env vars won't work in all cases. Is there another option? Perhaps Windows Terminal could ship with a small exe that, when run, checks how it's being run (parent processes, process groups, or some other OS method) and returns on stdout a JSON list of capabilities, version number, etc.? Just trying to be creative here.

The thing is, it does: wt --version is a thing, but it pops up a GUI 🤣

[j4james]

@axelfontaine ANSI escape codes you can detect by sending something like a CPR query and see if you get a response. Trying to detect "emoji rendering" is probably a lost cause.

@Jaykul I'm not sure how you'd expect that to help you anyway. Determining the version number of wt on the host machine tells you nothing about the capabilities of the client terminal.

[axelfontaine]

@j4james "Running under WT?" is a good enough emoji rendering support detection algorithm as far as I'm concerned.

[zadjii-msft]

maybe your algorithm should be return true 😝

(admittedly, I saved the file as UTF-8 then tried printing it to CP437 the first time, so that's on me)

[DHowett]

@j4james "Running under WT?" is a good enough emoji rendering support detection algorithm as far as I'm concerned.

maybe your algorithm should be return true 😝

Yup, this is an excellent example of a principle I've been talking about across this repository! In the fullness of time, "Are we running under WT?" will give you an incomplete and incorrect answer as to whether various things are supported on Windows. There are already folks in the wild who detect WT_SESSION and use that to determine whether to emit direct RGB color, for example... when the console has supported those colors since 2018. Discoverability is hard, but the simple fact remains that the things we do for the Terminal ultimately help the console that ships inside Windows as well. :smile:

(Specifically, emoji rendering works thanks to alabuzhev working on the GDI engine; we would not have gotten to it without his help.)

[axelfontaine]

@zadjii-msft I am well aware a .cmd file is still required to set chcp 65001 before launching my app. After that I currently detect whether WT_SESSION is 36 chars long. If that fails, then it's fallback to no-emoji no-ansi rendering. My comment was primarily about decreasing the number of false negatives to offer the improved experience to as many users as possible.

[zadjii-msft]

Sorry, I wasn't clear. The Window in my gif is conhost, not the Terminal. There's no WT_SESSION there, because it is not the Terminal. That's just the console. Emoji can work in the console as well as the terminal. That's part of the whole design of the Terminal - a most of the improvements we make to the Terminal's buffer and internals are things that the console can benefit from as well.

[heaths]

OK, so env vars won't work in all cases. Is there another option?

Is a perfect solution necessary? It's clear from the thread that there's not really an established pattern, but TERM and COLORTERM come close. Close enough? For most indented purposes, seems "yes".

Running some EXE as suggested above or relying on Windows- or Windows Terminal-specific solutions don't make scripts portable, or at least make them harder to write (would always need a precondition of WT_SESSION which, as @zadjii-msft points out above, also isn't accurate).

Seems the sooner a solution is added that is close to what most other terminals do, the less workarounds get invented that might be even worse.

[j4james]

@heaths If you're just trying to detect true color support, you should be able to use the standard method documented here:

https://gist.github.com/XVilka/8346728#querying-the-terminal

I believe that should be supported in Windows Terminal from version 1.12.

[heaths]

My original reason for jumping onto this thread was to find an easy way a script or program can query. Even the suggested gist - which I had found previously - doesn't always work. Even if you replace : with ; in Windows Terminal, the response you get back isn't the same:

echo -e '\e[48;2;1;2;3m\eP$qm\e\\'
^[P1$r0;48;2;1;2;3m^[\

The gist shows "r;48" instead of what Windows Terminal reports back, "r0;48". So depending on what people are checking for when reading back, it seems to vary from terminal to terminal. Whether you use : or ; certainly does. For example, using : in WT gives back ^[P1$r0m^[\, which indicates WT doesn't support truecolor.

This makes it more difficult to write portable shell scripts as well, where an env var is pretty straight forward to use.

[eryksun]

The defterm scenario is never going to work with WT_SESSION. When the Terminal is invoked for a defterm connection, cmd.exe is already running. At this point, it's environment variables are already set. It's started, so there's nothing WT can do to change them now. Then, the OS tries to create a console to host the cmd process. conhost does some work, and hooks up the cmd.exe to the Terminal (or some other terminal!). The Terminal can only add the WT_SESSION variable when WT is the parent, launching cmd.exe directly.

When a console is allocated on the client side, the internal function ConsoleCreateConnectionObject() calls NtCreateFile() to open the connection to conhost. The connection request triggers the handoff in conhost. After connecting, the client side could call NtDeviceIoControlFile() with an IOCTL that requests session environment variables that need to be set (e.g. "TERM", "WT_SESSION", "WT_PROFILE_ID", "WSLENV"). The base API could also make this IOCTL when the console connection is inherited, or when attaching via AttachConsole(). This would ensure that clients always start with relevant session environment variables. The ConPTY API would include functions to get and set these session environment variables in the console server.

For handoff, the delegated console (openconsole) would need to obtain the session environment variables from the delegated terminal when it calls handoff->EstablishPtyHandoff(...), which could be either from an out parameter or a subsequent COM call. This guarantees their availability when the base API requests them.

[DHowett]

I actually really like this. We've been thinking about what it would mean to extend the Console API surface, too, with both inbox consumers (conclnt) and out-of-box ones. The Windows release cycle gives us a bit of trouble, but since DefTerm is locked behind Windows updates as well ... it isn't as much of a problem. Interesting.

[Jaykul]

@heaths wrote:

echo -e '\e[48;2;1;2;3m\eP$qm\e\\'
^[P1$r0;48;2;1;2;3m^[\

The gist shows "r;48" instead of what Windows Terminal reports back, "r0;48".

The leading 0; is the SGR reset which "returns all attributes to the default state prior to modification".

The response for DCS $ q Pt ST according to ctlseqs is supposed to be: DCS 1 $ r Pt ST

When the query Pt is m you're going to get the SGR back as the Pt. Windows Terminal is returning: 0;48;2;1;2;3m which is the reset plus the background. Some other terminals may leave off the reset. For our purposes, you can (should) ignore it. You set the default background (48), so what you're trying to read is the value of 48.

I wrote Test-RgbMode for example, but I don't have a wide test base to verify it works.

[heaths]

Thanks for the explaining the 0 there. In hindsight I probably should've realized that (I tend to reset colors to be sure myself), but wasn't entirely sure what the printed sequence was representing.

But looking at your gist shows the complexity that any script would have to do in Windows Terminal as opposed to most other terminals with a combination of TERM and COLORTERM. Given the acquisition model for Windows Terminal (most people probably get it through the Windows Store) and update push model, is there any big downside of assuming the vast majority of customers have the latest and TERM and COLORTERM are enough? My concern is more about portability of scripts with common terminals and less about 100% accuracy. Wouldn't you agree that the vast majority of devs wanting to detect capabilities are probably most interested in terminal colors? Sure there's some one-offs for detecting if hyperlinks are supported, etc., but given that most customers would have the latest WT simply checking TERM for, say, "xterm-256color" or whatever it would be set to should be approximate.

Even the detection mechanism you demo above has variants for terminals like using : instead of ; so the logic becomes even more complicated. It's all possible, of course, but you start turning otherwise small shell scripts into much larger ones.

[j4james]

@heaths Note that there are three different truecolor formats, and terminals don't necessarily support all of them. If COLORTERM is set to truecolor or 24bit, that indicates that the terminal might support some form of truecolor, but it doesn't tell you which formats will work. It's probably reasonable to assume the semicolon format, but that's not guaranteed.

For example, using : in WT gives back ^[P1$r0m^[\, which indicates WT doesn't support truecolor.

No - that's indicating that WT doesn't support the colon format, which is exactly the point. A COLORTERM variable wouldn't tell you that.

That said, if you're happy with the all the limitations of environment variables, that's fine. I just want to make sure you are actually aware of those limitations.

[heaths]

I appreciate the limitations, yes. Not only have I been following this thread, but working with @DHowett offline with some GitHub CLI (and related modules) issues, along with a couple GitHub developers. The main reason I brought up my concern/question above was questioning whether or not the limitations of env vars like TERM and COLORTERM are worth it for the vast majority of cases. I posit: probably not. In scenarios where those limitations may be problematic, certainly app/script devs can query caps from the terminal as you've suggested - dealing with all the portability issues across shells.

Scenarios where env vars are probably good enough would be colors, I would think. Let's say that you only set TERM=xterm-256color and some shell decides to use 256 indexed colors instead of truecolor even though they could. Is that detrimental? Or if they merely check TERM for "xterm" to assume it supports OSC 8 (hyperlinks), WT would as long as they have a fairly recent (1.10?) version, which given Windows Store's (eventual) push model for updates is likely, especially as more time passes.

IMO, I just don't see a great reason to avoid using common TERM and COLORTERM even if they aren't 100% correct. For simple scenarios they should be good enough, as they have been even before Windows Terminal despite some differences across various terminals. When it matters, devs should be encouraged to query caps on the terminal. Both can be offered.

[tracker1]

@j4james "Running under WT?" is a good enough emoji rendering support detection algorithm as far as I'm concerned.

Not for people actually making terminal programs that won't always be run in windows... not to mention there are different terminals for windows itself. WT_* may indicate that it's "Windows Terminal" that said it's a horrible experience when trying to support different terminals, or possibly even trying to create something that can detect features.

It's very similar to at least being able to get the application name (TERM_PROGRAM) and the version (TERM_PROGRAM_VERSION) so that you can handle, work around or determine specific implementation bugs, in a consistent way with other environments.

Specific, similar example.... Internet Explorer 5.0.0 had a very specific bug in which it implemented a new interface for managing the options in a <select> element that was fixed in 5.0.1 ... but at the time, it was written to every Windows 2000 and Office 2000 cd... that's an example of a very specific work around... that said, it happens more often than most would like to think.

In this case, the terminal is effectively a browser for a command line application, including the shell environment.

Some features that concern me that would be nice to be able to determine in a consistent way....

• Color support, 16, 256, rgb including either/both delimiter options
• UTF-8 or not, can we find out what the character set is otherwise?
• Emojii (none, single color glyph, or color emojii)
• Image rendering support/options
• hyperlinks
• Regions?
• screen size
• screen resize

Doing so in the most consistent way, termcap library support for WT, etc. But short of having at LEAST the TERM_PROGRAM, it's not at all consistent with other terminals.

For that matter, maybe a settings option for adding/setting/overriding custom environment variables, so the user can CHOOSE to add appropriate options for their environment (wsl, ssh, etc).

[j4james]

Color support, 16, 256, rgb including either/both delimiter options

You should be able to determine this with a DECRQSS query.

UTF-8 or not, can we find out what the character set is otherwise?

This could probably be detected by ouputting a UTF-8 character and then measuring the consumed space with a DSR-CPR query.

Image rendering support/options

You can determine Sixel image support from the DA1 report.

hyperlinks

Not possible yet, but I'm hoping to persuade other terminals to use DA1 for this to.

Regions?

Not sure what you mean.

screen size screen resize

If you mean you want to determine the size of the screen, and whether you can resize the window, a DSR-CPR query would probably suffice.

[heaths]

The issue I was trying to raise in the last couple comments above, though, is why writing to a TTY and checking the results would be the most accurate way to determine capabilities, are you expecting that every program and, more important, every shell script (especially those that want to remain simple but maybe write out some colors) have to write to the TTY, read back, then clear the line just to emit some colored output? The environment variables like TERM, COLORTERM, and TERM_PROGRAM may not be perfect but I imagine for most cases are good enough. So why not support both?

[j4james]

@heaths I'm not expecting anyone to do anything. I'm just saying there are queries you can use if you need an accurate way to determine those features. If asynchronous queries are not appropriate for your application, or you don't particularly care whether your feature detection is accurate or not, then you can of course use any other method you prefer. As I said before, if you're happy with the limitations of environment variable, that's fine - whatever works best for you.

[tig]

Hi all, maintainer of https://gitub.com/gui-cs/Terminal.Gui here.

We would really like a way to ask terminals (not just Windows Term) if a specific unicode glyph is supported at runtime.

The idea being, we have to choose a least-common-denominator glyph for things like the one used for checkboxes. In this example, we choose the square root symbol as it empirically works on every terminal we've tried, with every font we've tried, where other "check mark" symbols do not.

What we'd like to do is emit a DECRQM asking, "Can you render this pretty glyph"? If the answer is no, we'll fall back to our default. If the answer is yes, we'll use the prettier glyph. We could also do this via querying an environment var or something, but it seems to me the most cross-platform way to do this is to extend DECRQM to support such a query?

I know it is possible to ask (at least on Windows) if a particular font supports a particular glyph.

I found this Issue in my searching for existing solutions to this and it seems like a good place to start. Let me know if you have suggestions on how to proceed further. My dream (big dreamer here) is other terminals will also implement such a thing in a standard way...

[j4james]

@tig If I remember correctly, there was a some discussion a while back about coming up with a terminal standard for querying Unicode glyph support. I may be misremembering the details, and I haven't been able to track down the thread where it was discussed, but I think it would probably have covered your use case if it were ever implemented.

That said, this isn't something that Windows Terminal could easily implement, even if such a standard did exist, because these query sequences are handled in conhost, which doesn't know anything about the fonts that the actual conpty client is using. So this might be one of those things that would only work with a pass-through mode (#1173).

In short, it's possible this might be feasible one day, but unlikely in the near future.

[tusharsnx]

It's clear that setting $TERM is not sufficient when an application wants to know if terminal supports FeatureX.

I guess we can have a terminfo for WT, for all kinds of *NIX applications. We can always be TERM="xterm-16color" or even TERM="" (whatever the safest option is) and totally support extra capabilities by announcing it via terminfo. User has to do one time installation of WT's terminfo on their favourite distro (by curl and install) to support all new capabilities. If that's not possible for whatever reason, you are pretty much using the same terminal what you get today without any modifications.

This is supposed to done by terminals and not ConHost or ConPTY. For any terminal out there, if they want to support all capabilities of ConHost then they should need to announce it. ConHost won't do it for them. Applications should just behave the way they do right now when terminfo is not found or they don't recognise the terminal.

Remaining problems:

• What should ConHost/ConPTY do when default terminal is invoked?
• What should be the mechanism for announcement with Windows console applications? (Assuming terminfo isn't widely used on Windows.)