🐛 delta causes fancy unicode symbols such as Nerd Font Icons to be displayed as escaped in angle brackets

[vorburger]

$ git clone https://github.com/vorburger/vorburger-dotfiles-bin-etc.git
$ git checkout 975d4c977d41d8dd096067fa6d0588edee2c9fc7
$ ./fonts-install.sh
$ git --no-pager log | head -12

Note the funky "font symbol" I've used in those 2 commit messages (the fancy 'A' - see it?), this is from https://github.com/ryanoasis/nerd-fonts/, which my ./fonts-install.sh that's invoked above installed (if you then also appropriately configure to use that [patched!] Fira Code Nerd Font in your Terminal).

Now check out what delta seems to do:

$ git --no-pager log | delta

commit 975d4c977d41d8dd096067fa6d0588edee2c9fc7 (HEAD -> main, origin/main, origin/HEAD)
Author: Michael Vorburger <SPAM@vorburger.ch>
Date:   9 minutes ago

    Fonts: Install <U+F031> Fonts in *-gui.sh instead of *install.sh

commit 7f9b65a35974881c0712f1fab386bb8edf9a6f8d
Author: Michael Vorburger <SPAM@vorburger.ch>
Date:   32 minutes ago

    Fonts: Fira Code != Fira Code NERD <U+F031>

So that "fancy" 'A' got replaced by <U+F031>- even though my (appropriately configured) Terminal could display it.

Is there a way to configure delta to pass it through? For reference, check out this out for good ol' less:

$ git --no-pager log | less
commit 975d4c977d41d8dd096067fa6d0588edee2c9fc7 (HEAD -> main, origin/main, origin/HEAD)
Author: Michael Vorburger <SPAM@vorburger.ch>
Date:   11 minutes ago

    Fonts: Install <U+F031> Fonts in *-gui.sh instead of *install.sh

commit 7f9b65a35974881c0712f1fab386bb8edf9a6f8d
Author: Michael Vorburger <SPAM@vorburger.ch>
Date:   34 minutes ago

    Fonts: Fira Code != Fira Code NERD <U+F031>

so same, it also does this (unwanted) "escaping", but less --raw-control-chars does the trick:

$ git --no-pager log | less --raw-control-chars

Of course, one would want to have the cake and eat it too, and still have fancy "ANSI coloring" as well... 😄

[dandavison]

Hi @vorburger, what encoding does the text use that you're sending to delta? delta currently only guarantees to correctly handle utf-8. (ref https://github.com/dandavison/delta/issues/188)

[vorburger]

@dandavison My git's output should be in UTF-8, based on this:

$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

[vorburger]

@dandavison My git's output should be in UTF-8, based on this:

@dandavison actually, forget about guessing what encoding git is sending to delta - a friend of mine pointed out this is simpler to reproduce, debug and illustrate; here is how it (for me, on a Fedora 39 Workstation e.g. in GNOME Terminal = VTE, with $TERM set to screen-256color and other settings from my dotfiles) behaves:

$ python3 -c 'print("\uF031")'
# Prints the "fancy" Font Symbol 'A'

$ python3 -c 'print("\uF031")' | less
<U+F031>

$ python3 -c 'print("\uF031")' | less --raw-control-chars
# Prints the "fancy" Font Symbol 'A'

$ python3 -c 'print("\uF031")' | delta
<U+F031>

It looks like it's something particular with those special https://github.com/ryanoasis/nerd-fonts/ icon symbol Unicode Codepoints? Because a good ol' standard Smiley Emoji ☺(U+263A) works for me, check this out:

$ python3 -c 'print("\u263A")'
☺

$ python3 -c 'print("\u263A")' | delta
☺

$ python3 -c 'print("\u263A")' | less
☺

[nogweii]

I've run into the same issue with Nerd Fonts, but a bit surprisingly emoji work just fine. (I used them for the file labels.)