Adding a new colour scheme to parse $LS_COLORS

[Nelyah]

Following the discussion from https://github.com/ranger/ranger/issues/169#issuecomment-406814926 Plugin and colour scheme to parse the $LS_COLORS The colour scheme still needs some work. It works for the extensions (such things as *.txt), but doesn't work for special files (denoted di, ln, ...)

[rpdelaney]

os.stat.st_nlink can be used to determine if a file is a hard link. I don't know if we get access to the file path in the color scheme though. Probably other special file types can be polled with the same method.

[vifon]

The colors seem off for me. I'm attaching the screenshot.

[toonn]

@Nelyah, pinging you about this. It's something many people've wanted so it'd be great if you could fix this up/or rationalize why it can't be fixed up so we can merge this : )

[Nelyah]

Thanks for the reminder, I had indeed left it all a bit behind. I'm working on it right now, and I will probably give a more detailed update later (hopefully tonight). :)

[Nelyah]

I pushed another commit, which should properly parse the LS_COLORS. Let me know if it works for you.

I added as well device and socket colouring, however it seems to be more than just doing a os.stat on the file. From what I have seen, I should look into the container/* directory for directions about how to handle other specific kind of UNIX files.

As of now, the colour scheme recognises:

• executable
• fifo
• link
• Block Device and Character Device
• Socket
• Directory
• all specified extensions

I specifically added a rule to prevent the directory from being coloured since they (mostly) are exectuable.

Also, one issue I've been running into (but which seems irrelevant to this issue), is ranger not having the LS_COLORS variable properly set. If you intend to use ranger as a default file manager, the LS_COLORS value should be available session wide, and not only from the shell.

[toonn]

Briefly testing this seemed to work. About the plugin, what's your reasoning for having that code in a plugin? (I'm just thinking about simplicity of distribution it'd be easier if there was only one file. Otherwise we have to completely rethink the colorschemes repo ; )

[Nelyah]

I originally followed the documentation which was suggesting to define new context tags from the plugin folder.

However, I put the code from the plugin file to the colorscheme one and it seems to work. I pushed the last version with the colorscheme file at the top level of the git repo so you wouldn't have to reorganise it :)

edit: Also added a rule to no colour links as executables (same as with ls I believe).

[Nelyah]

I removed the try/except block inside the class. That also made me think about the necessity of putting some code outside the class itself. Do you think it would be a good idea to put the "global" code in an __init__ function inside the class (which I should probably rename ls_colors)?

This would probably allow for better interaction with potential plugins.

[toonn]

Maybe keeping it in the class is better yeah. That way themes based on this one can override the behavior. And I agree about the name. It's not necessary but it's nicer when people start extending your colorscheme.

[Nelyah]

I haven't had much time these past few days, just to let you know I have not forgotten it. I'll try to get on to it this week-end!

[Nelyah]

I (finally) added the code correctly, and called the super for the Colorscheme class (compatible python2).

As mentionned in the commit, I also fixed an issue that would make files highlighted if they ended with the extension (but without the .). For instance, considering the extension .bar, the file foobar shouldn't get coloured while foo.bar should.

[rpdelaney]

Awesome. I'll be taking a look at getting this installed locally soon. Also eagerly awaiting this being added to master.

[Nelyah]

I just want to let you know that I'll go for an extended (~1 year) trip around the world and since I like to travel light I won't have any computer (except a smartphone), which means I won't be able to patch should any issue arise.

I'm leaving on the 28th of October, and on the case where we haven't resolved this PR by then someone will have to take it up from there (I could add that someone as a contributor to my repo).

[toonn]

@Nelyah, no problem. We already have access to your repo because of the PR. I'm only waiting for @Vifon to verify the earlier issues were resolved. This should be merged before you leave.

[vifon]

Currently it works great. Just one nitpick: the tags are no longer in a distinct color. Is that intentional?

[vifon]

I have found 3 more problems:

• the selection doesn't seem to get highlighted
• if I name a directory for example something.png, it gets colored as a directory but it's bold since the PNG files are bold in my config
• if a file has a colored background, it behaves strangely in ranger

[rpdelaney]

I guess @Nelyah has left for their trip. I'll try to see if I can find time to pick up the slack. I very much want to see this landed.

[rightaway]

It looks good thank you!

[rpdelaney]

@Vifon @toonn I'm working on some fixes for your comments. Since this isn't my PR I can't push to it. Should I send a separate PR?

[toonn]

@rpdelaney, I don't think there's a better way, just refer to #2 in your PR so they're both linked.

[Nelyah]

@rpdelaney I'm not able to code anymore at the moment but I could add you to the contributers?

[rpdelaney]

@Nelyah has added me to their fork, so I'll be able to update this PR as I get the time. Thanks Nelyah.

[vifon]

Awesome, thanks both of you!

[vifon]

@toonn I've tested it again as you asked and the selection problems seems to be gone. The one with the bold files is still valid and I don't have anything to test the background coloring on right now.

[rightaway]

Anyone who could review this? I'm using it with this patch but would love to have it in the official build.

[toonn]

@rightaway, there's some remaining issues:

• dir.png will be colored as a png rather than as a directory
• @Vifon talked about funky behavior for items with a colored background
• We need to decide on how to handle tag contrasting, should tags contrast to yanked/cut files, marked files?

[desbma]

I've been using this for month and it is working great.

It should be merged in my opinion, even if there are still issues, it is just an optional theme, and it is better than the status quo.

[toonn]

Good to hear. The issues I listed still have to be resolved before I'll consider merging this. I'll try to pick it up again soon™.

[rpdelaney]

We need to decide on how to handle tag contrasting, should tags contrast to yanked/cut files, marked files?

Can you say more about this?

[toonn]

Tags in the default theme contrast to the files they tag, they're red while the files are colored variously iirc. But this isn't true when the file is marked or highlighted or in the copybuffer.

[atticus-sullivan]

Just tried this the first time and it looks good so fare. The only thing I noticed is that the colour for folders is different depending on where they are displayed, in the middle column, the right or the left one. See the picture below.

[atticus-sullivan]

Oh and I think here is something wrong too: ls -al output:

and how ranger displays this:

[toonn]

Please check the problems @Vifon mentioned as well.

[Nelyah]

@atticus-sullivan Could you join the output of dircolors as well? I'm not getting this behaviour on my end

Other than that, since I came back and have some time nowadays, I'll try to have a look and see this MR through!

[atticus-sullivan]

$ dircolors LS_COLORS='rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:'; export LS_COLORS (have modified colors in my terminal emulator, but the same color escape sequence should be evaluated to the same color)

[Nelyah]

@atticus-sullivan Your bug should be gone now, there were indeed some confusion and errors when interpreting colours.

Btw, I added some check to exit ranger in case ls_colors attributes wouldn't be castable to int. Is there a preferred way to output log in such cases in ranger?

---

@toonn Do you have a file exemple with coloured background?

Also, do you have some specifications for tag handling? Here is the the relevant piece of code

if context.reset:
    return style.default_colors
elif context.in_browser:
    if context.selected:
        attr = style.reverse
    if context.tag_marker and not context.selected:
        attr |= style.bold
        if fg in (style.red, style.magenta):
            fg = style.white
        else:
            fg = style.red
        fg += style.BRIGHT
    if not context.selected and (context.cut or context.copied):
        attr |= style.bold
        fg = style.black
        fg += style.BRIGHT
        # If the terminal doesn't support bright colors, use
        # dim white instead of black.
        if style.BRIGHT == 0:
            attr |= style.dim
            fg = style.white
    if context.main_column:
        # Doubling up with BRIGHT here causes issues because
        # it's additive not idempotent.
        if context.selected:
            attr |= style.bold
        if context.marked:
            attr |= style.bold
            fg = style.yellow

[Nelyah]

@atticus-sullivan, can you confirm that the first bug has gone as well? (where the colour wouldn't be the same depending on the column)

@toon Just pushed a new commit. This should have fixed the bug for having the directory being given .png files attribute if it ended in .png itself.

That should be 2 issues resolved from your earlier comment. I only miss the specs for what should be tagged in what colour :)

edit -- Oups, scratch that last part, I believe you already added some changes for that, looking at the history. @Vifon Maybe has a file example with a coloured background?

[atticus-sullivan]

Actually I don't remember exactly in which directory the second issue was, but I tried several folders in /dev (it was somewhere there) and it looked great. I couldn't reproduce the weird colouring in the tree columns as well. For me this looks fine.

[rpdelaney]

@toonn how are we doing? :)

[toonn]

.> <.< Who? me??

This got a bit neglected, I'll review and merge soon™.

[rpdelaney]

@toonn Anything we can do to help? :)

[toonn]

Someone in the irc channel actually said they wanted to do an exa colorscheme, which would be a superset of this. However, the only thing blocking this is it not being clear which bits are still lacking. So everyone should start bother @vifon to find out what issues they found. : )

[SmartFinn]

38;5;xx;x notation for 256 colors doesn't parse well, attr 5 should be ignored but it isn't. I faced with non-stop blinking in gnome-terminal.

[dnastase]

Hi, just bumping this as well. This would be a very nice plugin to have :)

[jmcantrell]

I tried this color scheme on my system and got strange blinking colors.

I'm using https://github.com/trapd00r/LS_COLORS.

My terminal is xterm.

Output of uname -a:

Linux garnet 5.12.6-arch1-1 #1 SMP PREEMPT Sun, 23 May 2021 00:45:50 +0000 x86_64 GNU/Linux

Output of ranger --version:

ranger version: ranger 1.9.3
Python version: 3.9.5 (default, May 12 2021, 17:14:51) [GCC 10.2.0]
Locale: en_US.UTF-8

Output of dircolors:

LS_COLORS='rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:';
export LS_COLORS

[toonn]

Oof that's terrible. Could you minimize your LS_COLORS? Pretty hard to figure out what might be causing this since it's rather long.

[rpdelaney]

Could it be related to this comment?

[jmcantrell]

Here are some specific ones.

Python files:

LS_COLORS='*.py=38;5;41' ranger --cmd="set colorscheme ls_colors"

Executable files:

LS_COLORS='ex=38;5;208;1' ranger --cmd="set colorscheme ls_colors"

[jmcantrell]

I just realized that attribute 5 is blink, which makes me wonder why my terminal isn't blinking outside of ranger. Maybe I should be using different color definitions?

[jmcantrell]

After some digging, I found that these color definitions are xterm control sequences, which are supported by dircolors. If your terminal is capable of 256 colors, the following can apply.

From https://www.xfree86.org/current/ctlseqs.html:

If 88- or 256-color support is compiled, the following apply.
P s = 3 8 ; 5 ; P s → Set foreground color to the second P s
P s = 4 8 ; 5 ; P s → Set background color to the second P s

According to that, the blink attribute is being used as a delimiter between the foreground/background code and the color value. This is obvious from: https://github.com/ranger/colorschemes/blob/de432556aa173d1a4566e24028441e637812fac7/ls_colors.py#L144-L149

Although, it appears that even when 256 colors are not in use, the blink attribute is interpreted as another bold:

P s = 5 → Blink (appears as Bold)

Now, in this color scheme, when parsing the attributes, blink is interpreted as-is, but should probably be ignored. https://github.com/ranger/colorschemes/blob/de432556aa173d1a4566e24028441e637812fac7/ls_colors.py#L103-L104

@Nelyah It's clear from looking at the code that you're already aware of everything I've said above about xterm control sequences. Are there any other implications to simply ignoring the blink attribute on lines 103-104? Removing them gets rid of the blinking, but it's also causing crashes that I'm unable to see.

Does anyone know how to use ranger's logging within a color scheme? Using a standard logger overwrites the interface with messages, and it's difficult to use.