Make certain that the cursor can invert/not-occlude/display the character under it

[DHowett]

I've been using WSL for a while and it's been great, but using nvim with this issue still pending is at times unusable. Other terminals are not having this issue (see ConEmu or Hyper for example). I hope this issue gets re-opened.

Originally posted by @n1ghtmare in https://github.com/microsoft/terminal/issues/1203#issuecomment-801461436

I'm with @n1ghtmare on this one. Most syntax highlighting colors in Vim have a wide array of colors that very much clash with the cursor color when the cursor color isn't able to be swapped to a contrasted color dynamically. This has been demonstrated in the few screenshots I uploaded before. It affects everyone using terminal Vim or another terminal based text editor.

I know @pianocomposer321 said to pick a better color for the cursor but this doesn't make a difference in the end. If he adjusted his screenshot to be over a comment instead of the keyword then the comment's character under the cursor would be invisible. Also in his current screenshot the contrast is very low, it would certainly not be good enough for a video recording or giving a talk.

As good as the MS terminal is, I still tend to use wsltty (which dynamically sets its cursor color based on contrast calculations) in my day to day because having a block cursor in terminal Vim is much better than the vintage cursor in the MS terminal. Not just from a legibility POV, but it allows you to configure Vim so that your cursor shape changes from the block cursor to a line cursor depending on if you're in normal or insert mode.

Originally posted by @nickjj in https://github.com/microsoft/terminal/issues/1203#issuecomment-801475389

Subscribers, thumbs-uppers from that comment: @krage, @saurik, @krage

[skyline75489]

Is this blocked by #6193 or other issues? Or we can just implement the algorithm for calculating the contrasted color and call it a day

[zadjii-msft]

I KNEW THERE WAS AN EXISTING ISSUE!

Part of the trick here is that when the Renderer is drawing runs of text, it doesn't know where the cursor is. So it needs to be able to split the run with the cursor in it, so it can draw {text left of the cursor, text where the cursor is, text to the right of the cursor }.

Presumably, if there's a way of doing it without doing all the work in #6193, that'd be cool. I had all these ideas before in #6151 for different ways of specifying how the text at the cursor should look. Something like "cursorTextColor": null|"textForeground"|"#rrggbb".

[DHowett]

The thing I was toying with--which initially made me think that this would be easy--was drawing a D2D "fill effect" (that is: a command buffer that produces a flood fill) through the image/effect drawing pipeline with the XOR composition mode. It was promising, but it only worked where we weren't already drawing a background. Everything goes pear-shaped when you are drawing a background color. :smile:

[DHowett]

However, it looks like we can use the 2-pass cursor renderer @zadjii-msft wrote to backplate the color for a destination-inversion cursor.

[DHowett]

[nickjj]

Looks like a really good start, nice work. Could this be something that gets backported into the current stable release as a hotfix once you're happy with its performance and stability? It feels like such a huge feature that it transforms the entire experience for anyone who might be using Vim.

[Pirastrino]

Imo, there are 3 possible solution to this:

• calculate contrast color from the current foreground
• use terminal background color
• define background and foreground color for the cursor

Calculated contrast color may not have enough contrast to the cursor's background color (even visible in the post above (purple cursor + inverted color - blue) and it may be distracting for some users. I agree it should be the default behavior but also, there should be a possibility to use a custom cursor foreground color defined by user or system background as default, something like: Cursor color = (contrast color | system background | custom cursor foreground) in profile settings.

But I see that the main issue here is the cursor position between renders so thank you for your work @DHowett!

[zadjii-msft]

Well, what needs to happen now is for us to come up with a design that reconciles the current cursorColor with the proposed cursorTextColor, with the now possible invertCursor functionality. Once we've got a sensible way of expressing these options, it needs to make it into a PR, merge it, then selfhost it internally to make sure it feels right. That'll probably take right up to the next release anyways, so I'm not sure releasing as a hotfix would end up saving anyone any time 😄

[DHowett]

For those of you who appreciate long commentary,

// CURSOR INVERSION
// We're trying to invert the cursor and the character underneath it without redrawing the text (as
// doing so would break up the run if it were part of a ligature). To do that, we're going to try
// to XOR the content of the screen where the cursor would have been.
//
// This renderer, however, supports transparency. In fact, in its default configuration it will not
// have a background at all (it delegates background handling to somebody else.) You can't XOR what
// isn't there.
//
// To properly invert the cursor in such a configuration, then, we have to play some tricks. Examples
// are given below for two cursor types, but this applies to all of them.
//
// First, we'll draw a "backplate" in the user's requested background color (with the alpha channel
// set to 0xFF). (firstPass == true)
//
// EMPTY BOX  FILLED BOX
// =====      =====
// =   =      =====
// =   =      =====
// =   =      =====
// =====      =====
//
// Then, outside of _drawCursor, the glyph is drawn:
//
// EMPTY BOX  FILLED BOX
// ==A==      ==A==
// =A A=      =A=A=
// AAAAA      AAAAA
// A   A      A===A
// A===A      A===A
//
// Last, we'll draw the cursor again in all white and use that as the *mask* for XORing the already-
// drawn pixels. (firstPass == false) (# = mask, a = inverted A)
//
// EMPTY BOX  FILLED BOX
// ##a##      ##a##
// #A A#      #a#a#
// aAAAa      aaaaa
// a   a      a###a
// a###a      a###a

[pianocomposer321]

Just wanted to pipe up here for a second, specifically replying to @DHowett's last message. I always considered ligatures being drawn as their individual characters when the cursor is over them as a feature, not a bug. It's nice to be able to see what characters comprise the ligature if you need to, and have it display as a ligature otherwise.

That said, I can see it the other way too, but I'm just saying that if it was up to me I wouldn't jump through too many hoops to make ligatures display "correctly".

[DHowett]

Just wanted to pipe up here for a second, specifically replying to @DHowett's last message. I always considered ligatures being drawn as their individual characters when the cursor is over them as a feature, not a bug. It's nice to be able to see what characters comprise the ligature if you need to, and have it display as a ligature otherwise.

That said, I can see it the other way too, but I'm just saying that if it was up to me I wouldn't jump through too many hoops to make ligatures display "correctly".

Thanks for the input! Yeah... I waffled back and forth on this. Right now, given the architecture of our renderer, it's actually harder for us to split the ligature and re-strike the character :smile:

We condense full runs of text into as few rendering calls as possible, and we don't have support for splitting a run where we think the cursor is ... which is 100% something we need to fix later.

[DHowett]

It took me an entire week to realize that what most people want is a reverse video cursor (red on black text becomes black on red under the cursor), not a literal XOR cursor (red on black text becomes cyan on white under the cursor).

You can guess which one I've been working on.

Anyway -- is that an acceptable stop-gap? Just to make sure it's visible.

We've got a plan now, at least, for how we'll implement reverse video.

[nickjj]

Anyway -- is that an acceptable stop-gap? Just to make sure it's visible.

Yes, as long as the cursor is easily visible in all cases then you can always iron out the details later.

When in doubt there's always wsltty's code too. I don't know what they do but as an end user the cursor was always very visible. That project's source code is out of my element but the cursor code is somewhere in https://github.com/mintty/mintty.

[j4james]

Anyway -- is that an acceptable stop-gap? Just to make sure it's visible.

Yeah, I didn't want to complain because I thought maybe this is what everyone else wanted, and it's definitely better than nothing, but I was rather hoping for a simple reverse video cursor (specifically for block cursors - underline cursors just match the text color). But that can wait.

[DHowett]

Given the short runway we have for 1.8 and servicing changes into 1.7, I'm going to let this one sit for early entry into the 1.9 cycle. We had some disagreements as to how the setting should be represented, and coming to a conclusion on that takes precious cycles from release engineering.

Options-

1. {scheme}.cursorColor and {profile}.cursorColor gain support for inverse and (later) reverse
   • Eventually, this means cursorColor will support foreground+background, inverse, reverse.
   • This requires some UI work around pickable things that are not colors. It is somewhat unclear: these are not colors.
   • Color schemes are no longer color schemes; they are forced to decide the visual properties of the cursor
2. {profile}.cursorColor becomes deprecated¹ (as well as .foreground, .background, and .selectionColor; it only seems right) and we add the {profile}.cursorStyle enum, which supports color, inverse, and reverse.
   • I am hesitant to introduce a new setting for 1.7, but it would be in our best interest to do so.
3. As yet unexplored option 3. Proposals welcome.

¹ We should do these deprecations anyway. These decisions were made before we fully split schemes and profiles; profiles used to be able to set the whole color table but we yanked that feature after we learned nobody was using it (because it had been broken for four releases and nobody noticed...)

[saurik]

FWIW, I've never experienced a reverse video cursor (or, if I have, it was in the 80s/90s); I'm intrigued by it, and maybe would fall in love with it, but I'm concerned that I would personally (and this stuff is all so personal) find it to be extremely distracting while moving it across lines that have lots of colors and it starts flashing. The behavior I'd probably want (and I don't mind if you don't do it: I've been following all of these issues, and was mentioned in the top of this thread because of said interest on previous ones, but really only because I know some of the people involved and the cursor has in fact been a blocking issue for me even trying this app... but the reality is that I have a terminal that works just fine for me and I don't need a new one) is that the cursor has a fixed color (and very very importantly is just a solid block that doesn't blink) and then the text it is over also ends up with a fixed color, probably (as @Pirastrino described it, "the terminal background color"). My recent experiences are with mintty changing the text to black (which happens to be the terminal background color), with iTerm changing the text to white (which is ridiculous and sometimes almost unreadable, but wasn't unusable!... for all I know this is configurable and I never cared enough to check), and Apple Terminal just leaving the text color alone (the cursor was always a separate color from all the other colors).

[leland-kwong]

I like what @saurik mentioned about a filled box with the text color being the terminal background color. Also checked VS code, and it looks like that is how it is being done there as well.

[nickjj]

Hi,

I noticed a few versions shipped since this issue.

Not trying to rush anything but is this still being actively worked on? It's becoming more of an issue now if you happen to use zsh with fzf in Vim because fzf will change your cursor to a block cursor (unrelated bug it seems), but the end result is you're stuck with a block cursor where you can't see anything under it.

[zadjii-msft]

At the moment this is kinda just in a holding pattern - @DHowett's a little swamped with other priorities at the moment. I know that it's on his personal vendetta sticky note on his desk, but alas, there's other business that needs dealing with as well. That's what you get when you try to have a team lead write code 😜

[SilverEzhik]

Hi there, this has been bugging me a lot (I kept losing my cursor on the screen!) and I made a workaround that at least lets me have an inverted block in Neovim while having the shell and etc use an empty box.

Vim

This is done in Neovim using some autocmds - this probably will work in standard Vim as well and can also be simplified to have something like a black cursor with white text instead of inverted colors, but I haven't tried it.

" inverted cursor workaround for windows terminal
if !empty($WT_SESSION)
    " guicursor will leave reverse to the terminal, which won't work in WT.
    " therefore we will set bg and fg colors explicitly in an autocmd.
    " however guicursor also ignores fg colors, so fg color will be set
    " with a second group that has gui=reverse.
    hi! WindowsTerminalCursorFg gui=none
    hi! WindowsTerminalCursorBg gui=none
    set guicursor+=n-v-c-sm:block-WindowsTerminalCursorBg

    function! WindowsTerminalFixHighlight()
        " reset match to the character under cursor
        silent! call matchdelete(99991)
        call matchadd('WindowsTerminalCursorFg', '\%#.', 100, 99991)

        " find fg color under cursor or fall back to Normal fg then black
        let bg = synIDattr(synIDtrans(synID(line("."), col("."), 1)), 'fg#') 
        if bg == "" | let bg = synIDattr(synIDtrans(hlID('Normal')), 'fg#') | endif
        if bg == "" | let bg = "black" | endif
        exec 'hi WindowsTerminalCursorBg guibg=' . bg
        " reset this group so it survives theme changes
        hi! WindowsTerminalCursorFg gui=reverse
    endfunction

    function! WindowsTerminalFixClear()
        " hide cursor highlight
        silent! call matchdelete(99991)

        " make cursor the default color or black in insert mode
        let bg = synIDattr(synIDtrans(hlID('Normal')), 'fg#')
        if bg == "" | let bg = "black" | endif
        exec 'hi WindowsTerminalCursorBg guibg=' . bg
    endfunction

    augroup windows_terminal_fix
        autocmd!
        autocmd FocusLost * call WindowsTerminalFixClear()
        autocmd FocusGained * if mode(1) != "i" | call WindowsTerminalFixHighlight() | endif

        autocmd InsertEnter * call WindowsTerminalFixClear()
        autocmd InsertLeave * call WindowsTerminalFixHighlight()
        autocmd CursorMoved * call WindowsTerminalFixHighlight()
    augroup END
endif

Shell

I also added this to my shell profile to have the cursor get reset after Vim closes:

if [ ! -z "$WT_SESSION" ]; then
    PS1="\[\e[0 q\e[?12l\]$PS1"
fi

There are a few things going on here:

• \e[0 q will reset to the default cursor (this is the only way I found to get the empty box cursor, make sure that it's selected in your WT profile. This also won't work without the space between 0 and q)
• \e[?12l will disable blinking (\e[?12h enables it)
• Wrapping it all in \[ and \] helps with some visual artifacts in shell prompts (I do this with PS1 color sequences as well)

During all this I also found out that it's possible to set the cursor background color using both hex and X11 colors with this OSC escape sequence: \e]12;#FF00FF\a. Not sure if this is documented anywhere?

tmux

I also added this line to my tmux configuration so that switching between panes correctly switches the cursor shape to the default:

if '[ ! -z "$WT_SESSION" ]' 'set -ga terminal-overrides ",*:Ss=\\E[%p1%d q:Se=\\E[0 q\\E[?12l"'

[phortonssf]

@SilverEzhik Works pretty great, thank you. however it only worked once I ran cursor+= in my init.vim. Once I had that, it was quite good. Great job with the color reverse. The cursor not resetting back in windows terminal seems to be a neovim issue they are working on. Also this line didnt seem to be working for me to set the cursor to blink or other modes etc.

set guicursor+=n-v-c-sm:block-WindowsTerminalCursorBg

[Hasanabbas]

Any update on when this could be implemented?

[zadjii-msft]

Nope. We'll make sure to update this thread when there is. In the meantime, might I recommend the Subscribe button? That way you'll be notified of any updates to this thread, without needlessly pinging everyone on this thread ☺️

DHowett did have a prototype he messed around with on the long weekend

but it's a little far away from shipping ATM.

[DHowett]

You're right to ask about the current state of this issue. Since my original work to enable cursor inversion (which we never put behind a setting -- ugh, I apologize¹) we learned in a series of issues that there's a much better way for us to draw text on the screen.

Right now, all text in Terminal is broken up by line and then by "attribute run" (foreground, background, etc.) into what we've called "clusters". Each cluster is then drawn individually, background and then foreground, at terrible expense. That we draw the background and foreground all at once is the crux of the difficulty here. To properly render ligatures and handle grayscale AA and stuff, we thought it would be important to not break clusters down further. However, to put a cursor/selection background and foreground down, we'd need to do exactly that.

We accomplished cursor inversion in that renderer by adding another (expensive) phase at the end of rendering where we draw down the cursor shape with an XOR mask.

¹ We got lost in discussing how best to expose this setting rather than making a decision and living with it for a little bit. That's on me.

---

In the "much better" rendering engine, color is deferred to a pixel shader that runs as the final step. Adding support for a split cursor color is a delta of around 5 lines of code and adds no additional complexity to the text gather/scatter pass.

My current proposal is: the new rendering engine will automatically render the cursor background in the requested color, and the character on top of it in either white or black depending on the perceived lightness of the cursor. No new settings, no dichotomy between inverse/reverse/obverse/transverse, and no separate customizable foreground. For now.

(Rainbow selection as depicted in the screenshot is also like ten lines of code instead of about a million, so that's cool.)

---

This is what we're trying to light up:

[AbstractMayhem]

Hi @DHowett ,

Thanks for the update. I have been following all the threads related with this issue.

Will the feature be available in the next windows terminal preview release?

Thanks

[DHowett]

Some version of this should be available under an experimental flag in the upcoming 1.13 preview release.

[nickjj]

@DHowett Hi, did this end up making it into the 1.13 preview release, if so which flag should we enable? I didn't see a reference back to this issue in the release notes.

[litesam]

I see the issue is not in the Experimental renderer of terminal preview release: (with experimental renderer on)

But I see a issue with rendering certain glyphs, I use oh-my-zsh and the arrow renderer prompt has a issue: (with experimental renderer on)

[Hasanabbas]

The issue is that the invert color only happens if the cursor color on your theme is exactly white (#FFFFFF):

As soon as I change the cursor color to #FFFFFE the inversion doesn't work:

And yes, I have the same issue as @litesam

[chemPolonium]

Turn the experimental render on fixes the issue under Campbell theme (cursor color #FFFFFF) and Campbell Powershell theme (cursor color #FFFFFF) and One Half Dark theme (cursor color #FFFFFF) but it seems not work well under Solarized Dark theme (cursor color #FFFFFF) and Solarized Light theme (cursor color #002B36)

[nickjj]

Which option is the experimental renderer?

The only rendering options I see in the settings are to redraw the entire screen or use software rendering, neither are marked as experimental. I also don't see the word "experimental" in the JSON file except for the retroTerminalEffect setting.

[chemPolonium]

@nickjj In the seprated profile (windows powershell, powershell, cmd etc.) 高级 is advanced

[zadjii-msft]

from the docs: Experimental text rendering engine (Preview)

[ftabaro]

Thanks @DHowett for your work. What about having the char under the cursor take the background color? This ensures the contrast is high and maximum readability imo.

[kippesp]

Black on white is definitely better that it was. This announcement blog is pretty good elaborating on the setting: https://devblogs.microsoft.com/commandline/windows-terminal-preview-1-13-release/

[ToxicSmurf]

I was looking around, hoping to find a solution for this. Little did I know, I just had to update! Thank you so much.

[nickjj]

I don't see that option as being available in 1.13, you can see behind the about box there's no experimental option on the bottom:

I also added the option manually to the JSON file and it had no effect (I restarted the terminal).

Is there something not mentioned in the docs that you need to do to show / allow experimental features?

[zadjii-msft]

The experimental renderer is for now, only available in Preview builds of the Terminal. It's going to remain preview-only while we sort out some remaining bugs in it. The Preview builds are generally stable enough for daily usage, at least in my experience.

[paulharris]

I just tried the preview. Open settings, set any profile to enable experimental-renderer (I tried PoSH and git-bash), Save, Open the profile. Win-Term just disappears (crash, I presume).

[broander]

Trying the preview with experimental renderer enabled. Seeing the same issue as above that the color invert only works with a #FFFFFF cursor color.

Also seeing a new issue with the rendering of italicized text which I have my comments render as:

You can see it's cutting of the edges of characters (see the phrase "TMUX and IOS using OSC52", which no doubt has something to do with how the new experimental renderer works based on the explanations above. It is not an issue with the old engine, here's a screenshot of the old engine:

[habamax]

Indeed, latest preview with AtlasEngine on:

If only there was a cursor foreground setting :)

[avih]

If only there was a cursor foreground setting :)

+1

This issue has been unsolved for years (together with #1203).

You're trying a million ways to automate a solution (render the block behind the font, contrast the fg to this or that, etc), but before automation, please allow the user to specify an explicit (block) cursor fg color.

Once the user can specify one, please go wild with automated solutions, but do let the user still specify one if they want.

[zhihaoy]

6337 made an assumption that foreground color is sufficiently different from the block cursor's background, which doesn't hold true. I still need a feature to change the TEXT color when the block cursor is over it. Inverting is too smart for me; I will set it to white in a light theme to distinguish the dark text from my dark block cursor if possible.

[james1236]

In the preview version with the new AtlasEngine renderer enabled:

Before

After

Almost fixed, but the cursor is much darker for some reason (even though cursorColor is #FFFFFF), why is the cursor color changing instead of the text? This is more annoying than the invisible text issue, so I'll be going back to the old version.

If only there was an option to set the cursor text color to black, as was suggested in the original issue 3 years ago...

[paulharris]

This did not look the same for me, the cursor was white and the text was slightly not white - not inverted - so was very hard to see the text behind the cursor.

[MustafaFayez]

what do i do to fix this issue? I use 1.17 preview version now

[sitompul]

@MustafaFayez https://github.com/microsoft/terminal/issues/9610#issuecomment-1441144003

[lhecker]

It's a little difficult for me to say whether I should close this issue with the PR that I've just opened... If I'm not misunderstanding the original issue description at the top, then it'll fix the issue. The PR uses an algorithm to determine the contrast between the cursor and the underlying background and the text on top of it, and ensure it always remains "distinguishable".

In other words: After that PR, if you use AtlasEngine, no matter whether you use inversed cursors or if you use colored cursors, the text will always remain visible.

The discussion however has drifted away quite a bit since then, which I can fully understand, given that our cursors are kinda... meh. But I feel like that these issues are not part of the original issue anymore. They're also much more easily to address. Among those are suggestion to allow users to configure the cursor background and foreground color individually, as well as to implement reversed cursors. I hope to get to that in the near term, although community contributions for that would be greatly appreciated, because I'm not sure if I'll have time myself.

On that note, it might be interesting to mention that the new colored cursors will be sort of "semi-reverse" cursors and look like this (this is when I set my cursor to #ff0000):

[3N4N]

Thank you! @lhecker. I hope to soon use block cursor like god intended (I'm currently using vintage shape because of this indistinguishable-character issue).

[The other issues/feature-requests] are also much more easily to address. Among those are suggestion to allow users to configure the cursor background and foreground color individually.

This really seems like a no-brainer. Since it's easier to address, as you say, why not implement that before the automated inversion solutions? I am repeating @avih here, I know, but this easy-to-address feature could help me use Vim with proper guicursor, instead of having to use the boring vintage cursor.

[lhecker]

Since it's easier to address, as you say, why not implement that before the automated inversion solutions?

The "automated" mode is ~4 lines of code and took me <5min to write and test, whereas the other issues/feature-requests require a lot more work. Given that I'm technically already 110% booked out with work for months on end, I think you can see why I've picked the easy option for now. 🥲

I hope to soon use block cursor like god intended (I'm currently using vintage shape because of this indistinguishable-character issue).

You could try god's intended cursors right now, if you're curious. 😏 Here is a preview build based on that PR. I put it on my own host for convenience (and because the file will automatically expire in a week that way), but it's properly built and digitally signed by Microsoft. Please let me know if you find any issues with the way the cursors work now!