search

adam7 / delugia-code

Can we add Nerd Fonts to Cascadia Code Font using a GitHub Action
MIT License
562 stars 34 forks source link

`\ufbab` character does not work anymore #64

Closed kapsiR closed 1 year ago

kapsiR commented 3 years ago

I'm using Windows Terminal with PowerShell (pwsh.exe)

Before with Delugia Nerd Font Version 22d4cae (v2105.24):
grafik

Now with Delugia (Delugia-Complete.zip), v2105.24-1-ga2e91f1:
grafik

Finii commented 3 years ago

Thanks for reporting this.

I'm still camping but will be back in civilized regions on the weekend.

It would help me finding the glyph if you could copy&paste it into a comment here (it may render wrong/differently in the browser but that does not matter).

kapsiR commented 3 years ago

Sure: ﮫ

Have a great time!

Finii commented 3 years ago

You gave the unicode code in the title 🤦

Great!

Those who can read but do not have no advantage on those who can not read...

Finii commented 3 years ago

Hmm, I see the normal hourglass in the file, at least in the light version:

image

Further examination ...

Holy cow, it is present in italic but not in regular??!

image

Furthermore a lot of glyphs in that vicinity seem to be corrupted.

Finii commented 3 years ago

Well, FBAB is normally an arab glyph, and now we have the correct arab glyphs there

image

The top shows the regular font, where Cascadia added arab glyphs recently. The bottom shows the light font, where they obviously forgot to add them.

We never overwrite existing glyphs, so we do not add the hourglass. Note the differing descriptive name (that we do not alter on existing glyphs obviously).

The very small pictures above always shows what the font's glyph should roughly look like. With the bottom font most glyphs are filled correctly (according to unicode), but unexpected to ppl using that codepoints as empty and add symbols there.

See https://github.com/microsoft/cascadia-code/releases/tag/v2106.17 that arab has been added.

Now we face the decision to remove arab or move the symbols to unexpected locations, or .... *ponder*

Finii commented 3 years ago

:thought_balloon: We even diagnose this ....

image

Finii commented 3 years ago

Here is an overview of all glyphs that prevent patching some symbols in. Almost all come with the MaterialDesignIcons. Next comment is a summary.

code Cascadia Code glyph Symbol glyph
0u2665 Black Heart Suit Unicode Character heart (octicons)
0u26A1 High Voltage Sign zap (SymbolsNerdFontComplete)
0uFB01 Latin Small Ligature Fi phone-classic
0uFB02 Latin Small Ligature Fl priority-high
0uFB2A Hebrew Letter Shin with Shin Dot sigma-lower
0uFB2B Hebrew Letter Shin with Sin Dot source-branch
0uFB2C Hebrew Letter Shin with Dagesh and Shin Dot source-merge
0uFB2D Hebrew Letter Shin with Dagesh and Sin Dot tune
0uFB2E Hebrew Letter Alef with Patah webhook
0uFB2F Hebrew Letter Alef with Qamats account-settings
0uFB30 Hebrew Letter Alef with Mapiq account-settings-variant
0uFB31 Hebrew Letter Bet with Dagesh apple-keyboard-caps
0uFB32 Hebrew Letter Gimel with Dagesh apple-keyboard-command
0uFB33 Hebrew Letter Dalet with Dagesh apple-keyboard-control
0uFB34 Hebrew Letter He with Mapiq apple-keyboard-option
0uFB35 Hebrew Letter Vav with Dagesh apple-keyboard-shift
0uFB36 Hebrew Letter Zayin with Dagesh box-shadow
0uFB38 Hebrew Letter Tet with Dagesh cards-outline
0uFB39 Hebrew Letter Yod with Dagesh cards-playing-outline
0uFB3A Hebrew Letter Final Kaf with Dagesh checkbox-multiple-blank-circle
0uFB3B Hebrew Letter Kaf with Dagesh checkbox-multiple-blank-circle-outline
0uFB3C Hebrew Letter Lamed with Dagesh checkbox-multiple-marked-circle
0uFB3E Hebrew Letter Mem with Dagesh cloud-sync
0uFB40 Hebrew Letter Nun with Dagesh directions-fork
0uFB41 Hebrew Letter Samekh with Dagesh eraser-variant
0uFB43 Hebrew Letter Final Pe with Dagesh face-profile
0uFB44 Hebrew Letter Pe with Dagesh file-tree
0uFB46 Hebrew Letter Tsadi with Dagesh gas-cylinder
0uFB47 Hebrew Letter Qof with Dagesh grease-pencil
0uFB48 Hebrew Letter Resh with Dagesh human-female
0uFB49 Hebrew Letter Shin with Dagesh human-greeting
0uFB4A Hebrew Letter Tav with Dagesh human-handsdown
0uFB4B Hebrew Letter Vav with Holam human-handsup
0uFB51 Arabic Letter Alef Wasla Final Form marker
0uFB53 Arabic Letter Beeh Final Form microscope
0uFB54 Arabic Letter Beeh Initial Form move-resize
0uFB55 Arabic Letter Beeh Medial Form move-resize-variant
0uFB57 Arabic Letter Peh Final Form phone-minus
0uFB58 Arabic Letter Peh Initial Form phone-plus
0uFB59 Arabic Letter Peh Medial Form pot
0uFB5B Arabic Letter Beheh Final Form serial-port
0uFB5C Arabic Letter Beheh Initial Form shape-circle-plus
0uFB5D Arabic Letter Beheh Medial Form shape-polygon-plus
0uFB5F Arabic Letter Tteheh Final Form shape-square-plus
0uFB60 Arabic Letter Tteheh Initial Form skip-next-circle
0uFB61 Arabic Letter Tteheh Medial Form skip-next-circle-outline
0uFB63 Arabic Letter Teheh Final Form skip-previous-circle-outline
0uFB64 Arabic Letter Teheh Initial Form spray
0uFB65 Arabic Letter Teheh Medial Form stop-circle
0uFB67 Arabic Letter Tteh Final Form test-tube
0uFB68 Arabic Letter Tteh Initial Form text-shadow
0uFB69 Arabic Letter Tteh Medial Form tune-vertical
0uFB6B Arabic Letter Veh Final Form chart-gantt
0uFB6C Arabic Letter Veh Initial Form chart-scatterplot-hexbin
0uFB6D Arabic Letter Veh Medial Form chart-timeline
0uFB6F Arabic Letter Peheh Final Form file-restore
0uFB70 Arabic Letter Peheh Initial Form language-c
0uFB71 Arabic Letter Peheh Medial Form language-cpp
0uFB73 Arabic Letter Dyeh Final Form bandcamp
0uFB74 Arabic Letter Dyeh Initial Form credit-card-plus
0uFB75 Arabic Letter Dyeh Medial Form Unicode Character itunes
0uFB77 Arabic Letter Nyeh Final Form Unicode Character calendar-range
0uFB78 Arabic Letter Nyeh Initial Form Unicode Character currency-usd-off
0uFB79 Arabic Letter Nyeh Medial Form Unicode Character flash-red-eye
0uFB7B Arabic Letter Tcheh Final Form Unicode Character piano
0uFB7C Arabic Letter Tcheh Initial Form Unicode Character weather-lightning-rainy
0uFB7D Arabic Letter Tcheh Medial Form Unicode Character weather-snowy-rainy
0uFB7F Arabic Letter Tcheheh Final Form Unicode Character tower-beach
0uFB80 Arabic Letter Tcheheh Initial Form Unicode Character tower-fire
0uFB81 Arabic Letter Tcheheh Medial Form Unicode Character delete-circle
0uFB83 Arabic Letter Ddahal Final Form Unicode Character hamburger
0uFB85 Arabic Letter Dahal Final Form Unicode Character inbox
0uFB87 Arabic Letter Dul Final Form Unicode Character reorder-vertical
0uFB89 Arabic Letter Ddal Final Form Unicode Character tag-heart
0uFB8B Arabic Letter Jeh Final Form Unicode Character solid
0uFB8D Arabic Letter Rreh Final Form Unicode Character baby-buggy
0uFB8F Arabic Letter Keheh Final Form Unicode Character bomb
0uFB90 Arabic Letter Keheh Initial Form Unicode Character calendar-question
0uFB91 Arabic Letter Keheh Medial Form Unicode Character camera-burst
0uFB93 Arabic Letter Gaf Final Form Unicode Character coins
0uFB94 Arabic Letter Gaf Initial Form Unicode Character crop-rotate
0uFB95 Arabic Letter Gaf Medial Form Unicode Character developer-board
0uFB97 Arabic Letter Gueh Final Form Unicode Character do-not-disturb-off
0uFB98 Arabic Letter Gueh Initial Form Unicode Character douban
0uFB99 Arabic Letter Gueh Medial Form Unicode Character emoticon-dead
0uFB9B Arabic Letter Ngoeh Final Form Unicode Character folder-star
0uFB9C Arabic Letter Ngoeh Initial Form Unicode Character format-color-text
0uFB9D Arabic Letter Ngoeh Medial Form Unicode Character format-section
0uFB9F Arabic Letter Noon Ghunna Final Form Unicode Character home-outline
0uFBA1 Arabic Letter Rnoon Final Form Unicode Character message-bulleted-off
0uFBA2 Arabic Letter Rnoon Initial Form Unicode Character nuke
0uFBA3 Arabic Letter Rnoon Medial Form Unicode Character power-plug
0uFBA5 Arabic Letter Heh with Yeh Above Final Form Unicode Character publish
0uFBA7 Arabic Letter Heh Goal Final Form Unicode Character robot
0uFBA8 Arabic Letter Heh Goal Initial Form Unicode Character format-rotate-90
0uFBA9 Arabic Letter Heh Goal Medial Form Unicode Character scanner
0uFBAB Arabic Letter Heh Doachashmee Final Form Unicode Character timer-sand-empty
0uFBAC Arabic Letter Heh Doachashmee Initial Form Unicode Character transit-transfer
0uFBAD Arabic Letter Heh Doachashmee Medial Form Unicode Character unity
0uFBAF Arabic Letter Yeh Barree Final Form Unicode Character watch-vibrate
0uFBB1 Arabic Letter Yeh Barree with Hamza Above Final Form Unicode Character dolby
0uFBBC Arabic Symbol Double Vertical Bar Below Unicode Character all-inclusive
0uFBC0 Arabic Symbol Small Tah Above Unicode Character arrow-right-box
0uFBD4 Arabic Letter Ng Final Form Unicode Character format-font
0uFBD5 Arabic Letter Ng Initial Form Unicode Character format-page-break
0uFBD6 Arabic Letter Ng Medial Form Unicode Character format-pilcrow
0uFBD8 Arabic Letter U Final Form Unicode Character garage-open
0uFBDA Arabic Letter Oe Final Form Unicode Character google-keep
0uFBDC Arabic Letter Yu Final Form Unicode Character heart-half-full
0uFBDF Arabic Letter Ve Final Form Unicode Character hexagon-multiple
0uFBE1 Arabic Letter Kirghiz Oe Final Form Unicode Character hook-off
0uFBE3 Arabic Letter Kirghiz Yu Final Form Unicode Character language-swift
0uFBE5 Arabic Letter E Final Form Unicode Character laptop-off
0uFBE6 Arabic Letter E Initial Form Unicode Character lightbulb-on
0uFBE7 Arabic Letter E Medial Form Unicode Character lightbulb-on-outline
0uFBFD Arabic Letter Farsi Yeh Final Form Unicode Character pencil-circle
0uFBFE Arabic Letter Farsi Yeh Initial Form Unicode Character pentagon
0uFBFF Arabic Letter Farsi Yeh Medial Form Unicode Character pentagon-outline
0uFC5B Arabic Ligature Thal with Superscript Alef Isolated Form Unicode Character volume-plus
0uFC5C Arabic Ligature Reh with Superscript Alef Isolated Form Unicode Character volume-minus
0uFC5D Arabic Ligature Alef Maksura with Superscript Alef Isolated Form Unicode Character volume-mute
0uFC90 Arabic Ligature Alef Maksura with Superscript Alef Final Form Unicode Character arrow-collapse-down
0uFCD9 Arabic Ligature Heh with Superscript Alef Initial Form Unicode Character metronome-tick
0uFD3C Arabic Ligature Alef with Fathatan Final Form Unicode Character video-4k-box
0uFD3D Arabic Ligature Alef with Fathatan Isolated Form Unicode Character video-input-antenna
0uFD3E Ornate Left Parenthesis Unicode Character video-input-component
0uFD3F Ornate Right Parenthesis Unicode Character video-input-hdmi
Finii commented 3 years ago

If we would drop the Arabic and Hebrew Letters, we end up with this set:

code Cascadia Code glyph Symbol glyph
0u2665 Black Heart Suit Unicode Character heart (octicons)
0u26A1 High Voltage Sign zap (SymbolsNerdFontComplete)
0uFB01 Latin Small Ligature Fi phone-classic
0uFB02 Latin Small Ligature Fl priority-high
0uFD3E Ornate Left Parenthesis Unicode Character video-input-component
0uFD3F Ornate Right Parenthesis Unicode Character video-input-hdmi
Finii commented 3 years ago

Here the conflicting glyphs:

image

I think we want to keep the ligatures, and maybe the parens are more useful then the video symbols.

This would mean:

Edit: Correct image, did use wrong font

Gorthog commented 3 years ago

So if I want to use Hebrew I can't use Delugia font? Is this something specific to Delugia or it's the same problem for all powerline fonts?

Finii commented 3 years ago

Neither.

Pure Powerline symbols are already in Cascadia Code, so there are no 'other glyphs in Cascadia Code that are overwritten' for Powerline capabilities.

In fact if you just need Powerline symbols you can/should use Cascadia Code directly. There is also no problem if you use Delugia PL.

The problem comes in with the Material Design Icons which occupy the same code points as Hebrew/Arabic letters. This means Delugia Complete. In fact the Material Design Icons are shifted by decision of Nerd Fonts into the same range as the Hebrew/Arabic letters. By themselves they are in another range :-o

So it is the same problem not for all powerline fonts, but for all Nerd Fonts.

But you are somehow right and it was good that you commented, somehow I thought 'noone would ever want to use that glyphs, right?' which is of course ... my fault :disappointed:

So our decision space is in fact a bit more broad. Let me state the facts and possible solutions that I see in the next comment.

kapsiR commented 3 years ago

For my use case (Windows Terminal), I have no problem with changing my characters to another code point.
That might be different for others of course, but I think including "hebrew/arabic people" is the right way of doing it.

Finii commented 3 years ago

Problem Statement

Arabic and Hebrew letters, recently added to Cascadia Code, occupy the same code points as some symbols we want to patch in. The letters occupy the correct code points according to Unicode, the symbols are strictly on wrong code points (but they do not have 'official' code points).

Problem Details

Material Design Icons occupy F001 - F847 Nerd Font patches them in shifted to F500 - FD46

Why do they shift it? Font Awesome occupy F000 - F2E0

The Unicode Blocks are defined as this:

Private Use Area E000 - F8FF CJK Compatibility Ideographs F900 - FAFF Alphabetic Presentation Forms FB00 - FB4F Arabic Presentation Forms-A FB50 - FDFF

Originally the Material Design Icons occupy the private area which is fine. Because Nerd Fonts adds so many icons the private area is faaaaaar to small and they enter neighboring blocks :roll_eyes:

Solution Ideas

  1. Drop Arabic / Hebrew
  2. Drop Material Design Icons
  3. Shift Material Design Icons to another range

Solution 1 is anglocentric and is not desirable in a global world imho. Solution 2 will result in a lot complaints because some users want some of the symbols. Solution 3 will also result in complaints because users will not find the symbols at the 'normal' positions.

'normal' means as defined by Nerd Fonts, the Material Design Icons originally sit at other code points anyhow.

I will search for another slot.

Finii commented 3 years ago

For my use case (Windows Terminal), I have no problem with changing my characters to another code point. That might be different for others of course, but I think including "hebrew/arabic people" is the right way of doing it.

ACK. I got side tracked (real work), will continue here later...

Gorthog commented 3 years ago

@Finii thanks for the detailed and comprehensive answer.

I downloaded and tried the latest version of Delugia font. It actually works pretty well for Hebrew as is right now image

So I think it might be a viable option to shift Hebrew / Arabic to another range because the glyphs I see here for Hebrew are quite rare, as they include punctuation. This day and age it's almost never used. I estimate the number of users that use punctuation in their terminal to be close to zero...

However, need someone who can verify if it's the same for Arabic. Maybe the shifted characters are commonly used in Arabic.

Anyway, the solution most likely will shift an existing range position - how should a user adjust character to the new position of the shifted glyphs?

Finii commented 3 years ago

Here an overview where Nerd Fonts places the symbols sorted by destination codepoint:

Icon Set Original Destination Private Use Area
Power Symbols 23FB - 23FE 23FB - 23FE
Octicons 2665 2665
Octicons 26A1 26A1
Power Symbols 2B58 - 2B58 2B58 - 2B58
Pomicons E000 - E00A E000 - E00A ok
Powerline Symbols E0A0 - E0A2 E0A0 - E0A2 ok
Powerline Extra Symbols E0A3 E0A3 ok
Powerline Symbols E0B0 - E0B3 E0B0 - E0B3 ok
Powerline Extra Symbols E0B4 - E0C8 E0B4 - E0C8 ok
Powerline Extra Symbols E0CA E0CA ok
Powerline Extra Symbols E0CC - E0D4 E0CC - E0D4 ok
Font Awesome Extension E000 - E0A9 E200 - E2A9 ok
Weather Icons F000 - F0EB E300 - E3EB ok
Seti-UI + Custom E4FA - E52E E5FA - E62E ok
Devicons E600 - E6C5 E700 - E7C5 ok
Font Awesome F000 - F2E0 F000 - F2E0 ok
Font Logos (Font Linux) F100 - F11C F300 - F31C ok
Octicons F000 - F105 F400 - F505 ok
Octicons F27C F4A9 ok
Material F001 - F847 F500 - FD46 NO :cry:

What I see here is a nice gap from E7C6 to EFFF. This are 0x83A slots. Material Design occupies 0x847 slots. Almost :-(

to be continued

Finii commented 3 years ago

Yesterday I came up with the following possible solutions.

They all shift part of the Material Design Icons to another range, because there is no unfragmented large enough chunk of code points in a private area available. Well, in fact there is the block Supplementary Private Use Area-A (F0000 - FFFFF), but I am unsure if we want to utilize three byte code points.

In reply to @sinapis:

option to shift Hebrew / Arabic to another range [...] how should a user adjust character to the new position

I believe it is impossible to shift the letter characters. Usually there is some means to enter them (apart from inputting raw codes), and that needs to be tweaked. Which is hard.

Shifting the symbols on the other hand is not so hard. Some prepared nice looking setups might break, but usually the symbols are entered by raw code numbers, and so that has to be adjusted once in the individual setup. Not nice but viable. For that reason I do not consider shifting the letters. Either drop them or keep them in their rightful places.

Split Range Variant A

When we try to

we could relocate:

Icon Set Original Old Destination Destination Private Use Area
Material F001 - F400 F500 - F8FF F500 - F8FF ok
Material F401 - F847 F900 - FD46 E900 - ED46 ok

Split Range Variant B

We can sacrifice the CJK Compatibility Ideographs block (F900 - FAFF), which is empty in Cascadia Code and will probably not result in unexpected :roll_eyes: side effects for people using Delugia, we can do even:

Icon Set Original Old Destination Destination Private Use Area
Material F001 - F400 F500 - FAFF F500 - FAFF violate CJK comp block
Material F401 - F847 FB00 - FD46 EB00 - ED46 ok

Split Range Variant C

When we do not try to keep the hex codes similar we can create less fragmentation in the user map, by doing:

Icon Set Original Old Destination Destination Private Use Area
Material F001 - F400 F500 - FAFF F500 - FAFF violate CJK comp block
Material F401 - F847 FB00 - FD46 E800 - EA46 ok

An then I noticed this:

image

WAT? 6 Octicons are immediately overwritten by Material Design Icons? Who designed that relocation table? I need to investigate that.

Finii commented 3 years ago

Regarding the Octicons...

The table is misleading. The source range is examined, and only filled glyphs are copied to the destination range, but packed (with no non-filled slots in the destination).

There are only 169 (0xA9) glyphs in the F000 - F105 range of Opticons instead of the expected 262 (0x106). Destination is thus F400 - F4A8 and no overlap.

image

Edit: Correct Destination address

aaronbell commented 2 years ago

Per https://github.com/ryanoasis/nerd-fonts/issues/365 this is a known issue with the Nerd Fonts placement of Material Design. I had hoped it would be sorted by now but alas.

I would strongly recommend against “sacrificing” the CJK slots (especially as I would like to fill them) and instead suggest use of Supplementary PUA A or B (https://www.unicode.org/charts/PDF/UF0000.pdf) which have more than enough spots to accommodate the full Material Design glyphs. This is where I planned to put them :).

Otherwise we will be having this conversation again in the future…

Finii commented 1 year ago

Some more information about this at Nerd Fonts: https://github.com/ryanoasis/nerd-fonts/issues/365#issuecomment-1254155491

Finii commented 1 year ago

Actually we are ahead of Nerd Fonts' Caskaydia Cove now.

With the latest update to Nerd Fonts v2.3.x we get the MaterialDesignIcons in PUA A, as details by their Plan 5+. But while they choose to have old and new codepoints together until the next major release, we always kept the correct CJK glyphs and just did not have the corresponding MDI icons.

Whatever. Use this table to get the new codepoints for your missed / existing codepoints of MaterialDesignIcons:

From the table you see that you can access FBAB (timer_sand_empty) with codepoint F06AD now

image

On some systems you need to use \U instead of \u for 5 digit unicodes. On other you can/have to put two UTF16 codepoints after another (like \udb81\udead), but some other systems will complain.

https://www.nerdfonts.com/cheat-sheet image

If you are using oh-my-posh, this might be interesting:

Edit: Add info on oh-my-posh