No Visual highlight with new themes on tty (t_Co=8)

[sandwm]

When using vim on a Linux tty and pressing V (or its variants) I noticed there is no visual indication of which parts of the text are selected. My colorscheme is elflord, but other themes are affected too (see first comment below). The issue started about a week ago, when Arch Linux shipped the new vim version (8.2.4981) which replaced the legacy themes with the new ones.

Here is the output of source /usr/share/vim/vim82/syntax/hitest.vim with legacy elflord.vim:

And here it is with the current elflord.vim:

You see that Visual as well as some other highlighting groups which previously had some highlighting applied are now missing it.

For better reproducibility and to make sure it's not related to my personal setup, I reproduced the issue with a fresh Arch Linux virtual machine: Get the iso image from https://archlinux.org/download/, start the VM from the iso and run:

pacman -Sy vim vim-runtime  # update vim packages
vim /etc/vimrc # add 'colorscheme elflord'
mkdir -p ~/.vim/colors/
cd ~/.vim/colors/
curl -o elflord.vim https://raw.githubusercontent.com/vim/colorschemes/master/legacy_colors/elflord.vim
# first image above produced by these two commands
vim -c 'source /usr/share/vim/vim82/syntax/hitest.vim'
vim -u NONE -c 'colorscheme elflord' -c 'source /usr/share/vim/vim82/syntax/hitest.vim'
rm elflord.vim
# second image above produced by these two commands
vim -c 'source /usr/share/vim/vim82/syntax/hitest.vim'
vim -u NONE -c 'colorscheme elflord' -c 'source /usr/share/vim/vim82/syntax/hitest.vim'

There was no relevant difference in the output between including -u NONE -c 'colorscheme elflord' or just using the /etc/vimrc (apart from having a minimal status line or not).

The first command (pacman) updates the vim packages so they contain the new themes. This won't be necessary once the 2022.06.01 image is released, which will already contain a newer vim version. If you want to see the very minimal /etc/vimrc and archlinux.vim, which is sourced by the former, you can find them here: https://github.com/archlinux/svntogit-packages/tree/packages/vim/trunk

Of course apart from the pacman command all other steps should also work on any other distro and a recent vim.

[sandwm]

Additional info:

• :set t_Co? gives t_Co=8 on the tty
• /usr/share/vim/vim82/colors/elflord.vim is identical to https://github.com/vim/colorschemes/blob/master/colors/elflord.vim
• The issue affects almost all themes shipped by vim 8.2.4981, except default.
  • These have no highlighting for Visual, as well as several other groups:
    • blue
    • delek
    • desert
    • elflord
    • evening
    • industry
    • morning
    • murphy
    • pablo
    • peachpuff
    • shine
    • slate
    • torte
    • zellner
  • These indeed have highlighting for Visual, but not for several other groups (not sure if that's a bug, too):
    • darkblue
    • koehler
    • ron

Note that I didn't do a full comparison for the non-elflord themes of legacy vs. new version, but just ran this in /usr/share/vim/vim82/colors/:

for file in *.vim; do vim -u NONE -c "colorscheme ${file%.vim}" -c "source /usr/share/vim/vim82/syntax/hitest.vim"; done

[habamax]

Thanks for the report, @sandwm

We aimed for GUI > 256 > 16 > 0 and for some reason missed 8.

I believe, this has to be fixed

[brammool]

Can you set it to use 16 colors? What do you mean with "Linux tty", what is 'term' set to?

[romainl]

Can you set it to use 16 colors?

If memory serves, doing set t_Co=16 in Vim does the job.

What do you mean with "Linux tty", what is 'term' set to?

Another name for what OP is talking about is "virtual console" or "virtual terminal" or "VT". You can get there from your desktop environment with Ctrl+Alt+F7, Ctrl+Alt+F8, etc. $TERM is usually linux, there.

[sandwm]

Yes, like @romainl said, I meant a virtual console like tty1, tty2, ... accessible via Ctrl+Alt+F1, Ctrl+Alt+F2, ... $TERM is linux. :set t_Co=16 works.

[brammool]

Well, you can set it to 16 colors, but it looks like it really only supports 8. With the default colors Visual uses background 8, which is just black and not grey. With 8 colors it uses reverse, which does work. Thus 16 colors is wrong.

Looks like the color schemes should support 8 colors, there is no workaround.

[romainl]

In the virtual console, we can…

• use colors 0-7 for fg and bg (red, darkred, lightred are all synonyms),
• use colors 0-7 for fg with bold to obtain brighter colors,
• get white text with ctermfg=white cterm=bold.

and we can't…

• use colors 8-15 directly for anything,
• have a white bg.

which means that…

• we have 16 colors at our disposal for fg
• white is only available for text,
• there is only one shade of grey for bg and it is quite light

Anything else I missed?

[habamax]

have a white bg.

What would we do with delek, peachpuff, morning, shine, zellner?

[romainl]

What would we do with delek, peachpuff, morning, shine, zellner?

Either accept to have it light grey instead of white or (╯°□°)╯︵ ┻━┻.

[habamax]

Either accept to have it light grey

Ah, light grey is available, this might work

[habamax]

To track status:

• [x] pablo
• [x] industry
• [x] elflord
• [x] blue
• [x] darkblue
• [x] delek
• [x] desert
• [x] evening
• [x] morning
• [x] murphy
• [x] peachpuff
• [x] shine
• [x] slate
• [x] torte
• [x] zellner
• [x] koehler
• [x] ron

[habamax]

@romainl pls check light bg colorschemes

PS, I have to do other things so the last one is on you :)

[romainl]

@habamax thank you for all that great work.

[sandwm]

Thanks to both of you! That looks much better in several places. :)

Unfortunately there's also a regression: SpellBad, SpellCap, SpellRare, SpellLocal lost all highlighting, compared to what's in vim 8.2.5046. (Actually, for SpellLocal that's not really a regression, since previously it had fg color = bg color.) I don't really mind, since I don't do spell checking in a tty, but wanted to note it nonetheless. Checked with elflord and pablo.

[habamax]

@sandwm it actually shouldn't. Any exact colorscheme or all of them?

[habamax]

This is shine:

desert:

evening

etc

[habamax]

Ah, elflord and pablo:

elflord:

pablo:

[habamax]

Aha, if it is in tty (or whatever it is called in Ctrl+Alt+F3) it is indeed have no highlighting.

Not sure how to tackle it...

[habamax]

I can't make screenshots there :)

But having those 3 lines I have nothing only for "Normal" (first line), everything else is highlighted.

[habamax]

underline makes it all the same color and for elflord it is the same as normal text.

We could remove it for 8c completely... @romainl what do you think?

[sandwm]

Yes, my comment was about vim in a tty with t_Co=8. I made screenshots from a virtual machine (vim with source /usr/..../syntax/hitest.vim).

Before:

After:

I think underline isn't supported there anyhow, so that could be removed. I'm gonna check.

[habamax]

@sandwm could you pls check if

• :hi SpellRare cterm=NONE
• :hi SpellLocal cterm=NONE
• :hi SpellBad cterm=NONE
• :hi SpellCap cterm=NONE

Make it properly highlighted?

[habamax]

I think underline isn't supported there anyhow, so that could be removed. I'm gonna check.

We can't probably detect if it is tty or xterm with 8c support, so if we are to remove it -> it would be for all terminals.

[habamax]

And we can make it reverse though...

[sandwm]

@sandwm could you pls check if

* `:hi SpellRare cterm=NONE`

* `:hi SpellLocal cterm=NONE`

* `:hi SpellBad cterm=NONE`

* `:hi SpellCap cterm=NONE`

Make it properly highlighted?

Yes, that works.

We can't probably detect if it is tty or xterm with 8c support, so if we are to remove it -> it would be for all terminals.

Shouldn't xterm have more colors? For me :set t_Co? in an actual xterm gives 256 (and underlines work).

[sandwm]

I think underline isn't supported there anyhow, so that could be removed. I'm gonna check.

Yes, no underline in tty (on a color display – so any display relevant here).

cterm=reverse works, although I think personally I would prefer just changing the color. That's more readable and a word which is supposedly spelled wrong (even if it's just a term the dictionary doesn't know) doesn't need the highlighting of e.g. an ErrorMsg.

[sandwm]

We can't probably detect if it is tty or xterm with 8c support, so if we are to remove it -> it would be for all terminals.

Shouldn't xterm have more colors? For me :set t_Co? in an actual xterm gives 256 (and underlines work).

Ah, looks like some older xterms indeed only supported 8 colors: https://stackoverflow.com/questions/10003136/what-is-the-difference-between-xterm-color-xterm-256color/10039347#10039347 I don't know whether they supported underline.

Side note: Even with export TERM=xterm-color (for 8 color xterm) which also leads to tput colors reporting 8, in vim I get t_Co=256 (which is indeed what this xterm supports).

[habamax]

I will have a look into this once everything is merged in

[habamax]

I couldn't find the way in templates how to clear the default reverse, e.g. if I remove underline then spell highllghts are reverse by default (at 8c).

So I made them reverse explicitly.

[habamax]

All of the colorschemes have 8c support now.

@romainl please check the latest darkblue and it might be a good idea to create PR against main vim repo to include them before vim9 is out

[habamax]

https://github.com/vim/vim/pull/10563