support 256 color

[melspectrum]

please have 256 color support. thanks!

[zadjii-msft]

Thanks for the suggestion! This is actually pretty high on my personal list of requests for the future, however, we weren't able to fit it into the Anniversary Update, though I'd keep my eye out in the future for it.

A coming insider update should have support to at least take the 256-color ANSI sequences and the full RGB ANSI sequences and find the nearest color that's in our 16-color table, and use that to render, which should at least enable applications which use those sequences to not be totally broken.

I'm gonna close this as a dupe of #76, which has more discussion on the subject.

[musm]

Could we get this reopened as #76 was closed since it was decided that the scope there was not 256 color support but just properly mapping 8-bit colors to the 256-bit range.

[benhillis]

Good suggestion!

[be5invis]

It is hard, almost impossible. Conhost uses a fixed 32-bit-per-cell layout (16 bits for properties and 16 bits for character code), which is a derivative of VGA text mode (yes, the DOS-era video memory layout, 8 bits for properties and 8 bits for color). Modifying it will definitely break compatibility, unless MS want to implement a completely different console API set (and provide a compatibility layer for existing console apps).

[musm]

Thanks @be5invis. It's 2016, breaking compatibility with DOS I don't think will in anyway affect anyone using windows 10, maybe a dozen people. It seems more logical to introduce a new fresh and modern console API that can be used for the next decade and retain the old conhost for legacy purposes.

Certainly, this wouldn't be the first time compatibility has been broken and trying to maintain backwards compatibility for something that old is impressive. I like what they did with IE; tons of stuff depended on IE, but they figured a way around it and introduced Edge, by keeping IE installed but for legacy support.

Considering the large number of devs that use the terminal on a daily basis (esp. linux devs to are married to the terminal), i think this would benefit more people than it would affect. Bash on windows is wonderful, but it's missing some polish and I think this would go a long way.

[be5invis]

@musm However, I am talking about NT console API, which is the basis of almost all Windows console applications and many of them are important basic industrial infrastructure, controlling important things like nuclear power plants... The only possible solution is to create a new console API interface and a bug-to-bug compatibility layer.

[musm]

I'm going to ref #417 as @Maximus5 suggested the fix "@zadjii-msft This issue is not actually forces msft to implement truecolor in conhost. In fact, and instead, topic starter asked you to bypass ANSI to ConEmu, which already supports true-color in console. Current implementation of console output in WSL prohibits ConEmu from processing any ANSI written by linux applications. That tight integration with kernel(?) is rather bad. I believe, the issue have to be reopened, ConEmu doesn't limit user to 16-colors palette. And proposed approximation would look awful."

[be5invis]

@musm @benhillis @zadjii-msft What if users do not have installed ConEmu? The true problem is that CONHOST already used all bits of the console buffer. And programs can directly modify it via C/C++ code. A solution may be to provide another completely different console API (stream-based) and provide a compatibility layer to the existing one (blitting the buffer at 60fps?). Maybe you MS guys should have a serious talk with OSG.

[musm]

The suggestions is a hacky one I agree, but something hopefully sufficient for the group using conemu in the interim while MS cooks up (hopefully) a longer term fix for the future. +1 for your idea of a new console API with a compatibility layer is far preferable of course

[be5invis]

@musm Hacky is necessary, because breaking existing console behavior may break important things like nuclear power plants. You definitely do not want that thing to happen. If you want to connect your terminal to WSL, why do not just use SSH?

[musm]

Could you clarify on how I would "If you want to connect your terminal to WSL, why do not just use SSH?"

[be5invis]

SSH connect to localhost

[musm]

Which problem would that solve? I mean SSHing from cmd/powershell from conemu to WSL, how does that fix things? Sorry I missed your point.

[be5invis]

SSH with other tools like putty. That one should support full 256 colors

[DanielGGordon]

I'm a bit confused here - I'm using Conemu - so is there a way for me to get 256 colors? Is there something I need to change? It looks like I still only have 16

[musm]

No, not possible

[DJviolin]

@be5invis Then why not fork cmd and start a separate console project for Windows 10, for example cmd256? Wouldn't this be the best solution? Having full ANSI colors in Windows, but not touching the original cmd and finally vim will look as good as in xterm.

Or call it cmd-ansi or anything.

[be5invis]

@DJviolin As told by a MS employee from OSG, there was such a plan (simulates a graphics card -- yeah, really), but they choose to use the native console, perhaps for the future lxss-win32 interop.

[DJviolin]

MS people! Why not keep the lxss integration with cmd and having an alternate option to start bash.exe from cmd-ansi.exe (or any other windows commands)? Than you will have full color palette and lxss not tied to this console exclusively.

[be5invis]

@DJviolin Manually start LXSS session and SSH to localhost?

[cheshrkat]

Reading through this thread - philosophically, if lxss gets caught up in worries about older windows shells, it's hard to understand the point of the project. Using the analogy from earlier... LXSS needs to be Edge vs. the IE8 of cmd/powershell.

Anyway to say what I actually came to say... +1 to fixing colours, it's not a trivial thing when it prevents things like vim colour schemes (my use case was setting up https://github.com/crusoexia/vim-monokai); and my usual PS1 didn't work because colours just disappeared. Way too easy to think of something like this as a nice-to-have, but in reality if LXSS doesn't replicate the basics of using bash on other platforms it's going to struggle to gain traction (and I really want it to gain traction :)).

[DJviolin]

We talking about a thing that already works in ConEmu with injection. None of us wants to change the cmd's default behaviour, we would be happy with an alternative console as well. Honestly, defining custom fonts, set font antialiasing, having 256 colors can be a deal breaker for many developers. We want VIM in it's full glory, inside the console, not in a heavy GUI application. Plus scrolling through a source file without any popping artifacts fast (like what VIM suffered in ConEmu).

[cowwoc]

Consider removing the duplicate tag, since this issue is no longer duplicate to #76.

[jquick]

+1, ive had to resort to sshing into localhost with putty/mobaxterm to get support.

[zadjii-msft]

Woah woah woah, boy did this discussion get out of hand. First off, a bit of a clarification, because this happens in every console issue thread and unfortunately it's not clearly explained anywhere as far as I know: cmd.exe is different from conhost.exe. cmd is a shell, much like bash, zsh, fish, or powershell. conhost is the console window itself, like xterm or Terminal. All the shells above run within the conhost window, which displays the stdout and handles the stdin for your shell.

Now, conhost, is the place where the 16 color cap exists - the way the text is stored, it's only possible to keep 16 colors on any character. Fixing that is a fairly sizable amount of work, and one that we're definitely trying to work on. If it comes, it's going to be via the escape sequences mechanism already in place in conhost, not via extending/changing the console API.

I am going to de-dupe this, because #76 was solved by supporting the true RGB color sequences by matching to the nearest color in Win10 AU, but true RGB color support isn't quite there yet.

[be5invis]

However, you still need to provide a compatibility layer for existing applications, and a switching mechanism for new applications to switch to new host.

???? iPhone

? 2016?8?25??13:34?SRGOM notifications@github.com<mailto:notifications@github.com> ???

@zadjiihttps://github.com/zadjii is there a chance you guys will ever build a different conhost? There's no way (that I know of) to programmatically change font size and face in this. The storage (registry or shortcut file) isn't clear. Even lines aren't wrapped properly. It has come far from what it was before 10 but it still feels very primitive compared to Linux terminal emulators which are first class. I'd use Conemu were it not so slow to start. I did an experiment before anniversary updates and I think comemu has more throughput too.

You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/Microsoft/BashOnWindows/issues/345#issuecomment-242285136, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAOp2wfprR1pscz8xAMntfL6BCFXpuQ_ks5qjSlbgaJpZM4IcbDU.

[be5invis]

@zadjii-msft Any plan with non-BMP, and advanced typography? Will you release the APIs into win32, or make it WSL-only?

[zadjii-msft]

@be5invis I'm not really sure how much of the plan I'm allowed to talk about exactly - we wouldn't want a NMS style overpromise/under-deliver situation on our hands. I can say that as far as any VT sequences we end up supporting are concerned, our goal has always been to avoid the expansion of the API surface. A few flags were added to the I/O modes (GetConsoleMode) in order to enable/disable the VT support, but otherwise any new features are hidden strictly behind VT sequences. For example, using an alternate buffer is supported by conhost, but only via VT sequences. @SRGOM again, I can't talk too much about the plan, but I agree that it's frustrating that the properties and settings are VERY confusingly stashed away in shortcuts and registry entries. It's a pretty awful situation without a clear remedy. Though, I'm not sure what you mean by "the lines aren't wrapped correctly" - it's a little awkward occasionally in bash with bash redrawing it's prompt and input while we simultaneously try to reflow the buffer, but otherwise it's usually fine....

[be5invis]

@zadjii-msft I can understand your consideration. Line wrapping may relate to DBCS characters. Currently wide characters (like 安) cannot be typeset correctly under cp65001. IME is unavailable either. And, yes, we all want an official PTY.

[zadjii-msft]

So, here's an update we know you've all been dying for:

https://blogs.msdn.microsoft.com/commandline/2016/09/22/24-bit-color-in-the-windows-console/

Full RGB color is supported as of yesterday's insider build.

[musm]

yay! thank you !!!

[Maximus5]

Is there a way to read 24bit color atributes? If not - implemented feature would be useless in ConEmu/cmder/ConsoleZ and other terminals.

[be5invis]

@maximus5 That is why we need an official API of pty, in Windows, win32 I mean.

[be5invis]

@zadjii-msft How to use the 24-bit colors in Win32?

[zadjii-msft]

We're not planning on expanding the Win32 API surface for this feature. Every function we add to the Console API just creates another point of incompatibility with linux terminal applications.

If you want to use the new color support in your application, you'll have to use VT sequences. Support can be enabled with SetConsoleMode (see https://msdn.microsoft.com/en-us/library/windows/desktop/mt638032(v=vs.85).aspx#Output_Sequences).

The sequences to emit actual colors are as follows

Pm = 3 8 ; 2 ; Pr; Pg; Pb -> Set foreground color to the closest match in xterm's palette for the given RGB Pr/Pg/Pb. Pm = 3 8 ; 5 ; Ps -> Set foreground color to Ps. Pm = 4 8 ; 2 ; Pr; Pg; Pb -> Set background color to the closest match in xterm's palette for the given RGB Pr/Pg/Pb. Pm = 4 8 ; 5 ; Ps -> Set background color to Ps.

(from http://invisible-island.net/xterm/ctlseqs/ctlseqs.html)

We're not going to be adding any API for reading the colors back out with 24-bit color, and as far as I know there are no VT sequences for reading the buffer or it's attributes.

[oblitum]

This is nice. What's the expected time frame to get this in release form? I'm not relying on insider builds.

[innateessence]

Thank you!!! :D

[StoneCypher]

@zadjii-msft - awesome, been waiting almost ten years for this. Like oblitum, I'm curious when people not on insider builds might get this.

[zadjii-msft]

@StoneCypher @oblitum I honestly can't say. I don't really know what the Windows timetable looks like for the next release. There have been two major updates in the year since Windows 10 shipped, so I'd say the next will also be sometime in the next ~6 months.

[StoneCypher]

@zadjii-msft - oh. This isn't coming to non-insider builds until next windows major?

[oblitum]

@zadjii-msft ok, thanks for the information.

[oblitum]

I think if I come back to windows I'm instead just gonna try some of these new distros for WSL that started to pop out + X, there's an Arch variant already, rolling releases FTW :)

[zadjii-msft]

@StoneCypher That's correct. There's no out-of-band release process for conhost unfortunately. The only way to get an update is with an OS update.

[DJviolin]

@oblitum Do you have a source for this? If it's possible Debian would be really great to use in WSL.

[oblitum]

@DJviolin updated my comment.

[oblitum]

@DJviolin I've seen a lot of stuff these days about slackware/arch etc, with X apps etcs, some weird stuff, maybe there's something about Debian on the web, I don't recall whether I've seen something about that or not.

[bitcrazed]

@oblitum Re. alt distro's: Note that they're unsupported at this time, but do feel free to give them a try: We don't do anything to prevent them from working; we're just focusing all our effort on delivering as solid an experience as possible on Ubuntu at this time.

[antitoxic]

@zadjii-msft

We're not going to be adding any API for reading the colors back out with 24-bit color, and as far as I know there are no VT sequences for reading the buffer or it's attributes.

That means only powershell and cmd will support this, right? Which means I can't use any external manager for things like tabs, color schemes, tiling. Can you suggest a way to overcome this problem?

[be5invis]

@zadjii-msft Yeah, could you please provide an example in C, calls WINAPI directly, and able to produce 24-bit color?

[zadjii-msft]

@antitoxic It means only conhost.exe console windows (the default for cmd.exe, powershell.exe, bash.exe, etc) will be able to get this behavior, correct. Changing the API surface is pretty much out of the question for this release.

And to be completely honest, 3rd party consoles aren't really a supported feature. They work because you are able to query the buffer and get it's contents (in lieu of proper pty-like behavior), but it's not something we go out of our way to support.

If a third party console REALLY wanted 24bit color, I'd imagine they might be able to disable VT processing mode of the console window, then parse the VT sequences themself. It's certainly not a great solution, but it might work (I honestly have no idea if it would). It would mean that the source console window and the 3rd party one would have vastly different buffer contents - which I'd imagine would cause a lot of difficulty.

---

@be5invis In terms of producing the colors:

bool EnableVTMode()
{
    // Set output mode to handle virtual terminal sequences
    HANDLE hOut = GetStdHandle(STD_OUTPUT_HANDLE);
    if (hOut == INVALID_HANDLE_VALUE)
    {
        return false;
    }

    DWORD dwMode = 0;
    if (!GetConsoleMode(hOut, &dwMode))
    {
        return false;
    }

    dwMode |= ENABLE_VIRTUAL_TERMINAL_PROCESSING;
    if (!SetConsoleMode(hOut, dwMode))
    {
        return false;
    }
    return true;
}
int __cdecl wmain(int argc, WCHAR* argv[])
{   
    argc; // unused
    argv; // unused
    //First, enable VT mode
    bool fSuccess = EnableVTMode();
    if (!fSuccess)
    {
        printf("Unable to enter VT processing mode. Quitting.\n");
        return -1;
    }
    int red = 0; // set these to whatever
    int green = 0;
    int blue = 0;
    printf("\x1b[38;2;%d;%d;%dm", red, green, blue); // produces RGB foreground
    printf("\x1b[48;2;%d;%d;%dm", red, green, blue); // produces RGB background

    int index = 0;
    printf("\x1b[38;5;%dm", index); // produces xterm color table index foreground
    printf("\x1b[48;5;%dm", index); // produces xterm color table index background
}

Most of that is copy-pasta from MSDN