Adjusting half-blocks, quadrants,... coordinates to unify mosaics

[PhMajerus]

This update adjusts the points coordinates of some previously existing blocks/mosaics characters to fit them in the same grid as used by octants and eights. This is required because the new octants and eights from Symbols for Legacy Computing do not duplicate existing patterns and expect those existing one to join perfectly with them to provide the whole set of all possible pseudopixels mosaics.

Description of the Pull Request

This update verifies and adjusts the existing characters that are now required to join seamlessly with the extended pseudo-pixels mosaics introduced with the symbols for legacy computing. Some of the existing characters were not proper mosaics, most notably ▞, where the two black rectangles overlap because they don't use the same y coordinates. This shouldn't happen as all the mosaics are supposed to fit precisely on a unified grid. Only a few characters required adjustments, but this PR also documents all the glyphs that have been checked to ensure alignments of all the mosaic characters.

BLOCKS: U+00A0 nbspace : already correct U+2588 fullBlock & fullBlock.stypo : already correct (used as reference bounding rectangle for all pseudopixels mosaics)

HALF-BLOCKS: U+2580 upperHalfBlock & upperHalfBlock.stypo : y=50% fixed from 707 to 873 (gdi) and 710 (stypo) U+2584 lowerHalfBlock & lowerHalfBlock.stypo : already correct (confirming upperHalfBlock y=50% was wrong) U+258C leftBlock & leftBlock.stypo : already correct U+2590 rightBlock & rightBlock.stypo : already correct

QUADRANTS: U+2596 lowerLeftBlock & lowerLeftBlock.stypo : already correct (confirming all other corrections above and below) U+2597 lowerRightBlock & lowerRightBlock.stypo : y=50% fixed from 707 to 873 (gdi) and 710 (stypo) U+2598 upperLeftBlock & upperLeftBlock.stypo : y=50% fixed from 707 to 873 (gdi) and 710 (stypo) U+2599 upperLeftAndLowerLeftAndLowerRightBlock & upperLeftAndLowerLeftAndLowerRightBlock.stypo : already correct U+259A upperLeftAndLowerRightBlock & upperLeftAndLowerRightBlock.stypo : y=50% fixed from 707 to 873 (gdi) and 710 (stypo) U+259B upperLeftAndUpperRightAndLowerLeftBlock & upperLeftAndUpperRightAndLowerLeftBlock.stypo : y=50% fixed from 707 to 873 (gdi) and 710 (stypo) U+259C upperLeftAndUpperRightAndLowerRightBlock & upperLeftAndUpperRightAndLowerRightBlock.stypo : y=50% fixed from 707 to 873 (gdi) and 710 (stypo) U+259D upperRightBlock & upperRightBlock.stypo : y=50% fixed from 707 to 873 (gdi) and 710 (stypo) U+259E upperRightAndLowerLeftBlock & upperRightAndLowerLeftBlock.stypo : Some y=50% fixed from 707 to 873 (gdi) and 710 (stypo), some were already correct U+259F upperRightAndLowerLeftAndLowerRightBlock & upperRightAndLowerLeftAndLowerRightBlock.stypo : already correct

OCTANTS: U+2582 lowerOneQuarterBlock & lowerOneQuarterBlock.stypo : already correct U+2586 lowerThreeQuartersBlock & lowerThreeQuartersBlock.stypo : already correct

EIGHTS: U+2581 lowerOneEighthBlock & lowerOneEighthBlock.stypo : gdi was correct, y fixed from -183 to -182 (rounding unification for all Eights) for stypo U+2583 lowerThreeEighthsBlock & lowerThreeEighthsBlock.stypo : y fixed from 534 to 535 (rounding unification for all Eights) for GDI, stypo was correct U+2585 lowerFiveEighthsBlock & lowerFiveEighthsBlock.stypo : already correct U+2587 lowerSevenEighthsBlock & lowerSevenEighthsBlock.stypo : y fixed from 1887 to 1888 (rounding unification for all Eights) for GDI, stypo was correct U+2594 upperOneEighthBlock & upperOneEighthBlock.stypo : y fixed from 1887 to 1888 (rounding unification for all Eights) for GDI, stypo was correct U+1FB83 upperThreeEighthsBlock & upperThreeEighthsBlock.stypo : from my PR #723 U+1FB86 upperSevenEighthsBlock & upperSevenEighthsBlock.stypo : from my PR #723

U+2589 leftSevenEighthsBlock & leftSevenEighthsBlock.stypo : already correct U+258A leftThreeQuartersBlock & leftThreeQuartersBlock.stypo : already correct U+258B leftFiveEighthsBlock & leftFiveEighthsBlock.stypo : already correct U+258D leftThreeEighthsBlock & leftThreeEighthsBlock.stypo : already correct U+258E leftOneQuarterBlock & leftOneQuarterBlock.stypo : already correct U+258F leftOneEighthBlock & leftOneEighthBlock.stypo : already correct

U+2595 rightOneEighthBlock & rightOneEighthBlock.stypo : already correct U+1FB87 rightOneQuarterBlock & rightOneQuarterBlock.stypo : from my PR #723 U+1FB88 rightThreeEighthsBlock & rightThreeEighthsBlock.stypo : from my PR #723 U+1FB89 rightFiveEighthsBlock & rightFiveEighthsBlock.stypo : from my PR #723 U+1FB8A rightThreeQuartersBlock & rightThreeQuartersBlock.stypo : from my PR #723 U+1FB8B rightSevenEighthsBlock & rightSevenEighthsBlock.stypo : from my PR #723

## References This fixes the inconsistent alignments problem explained in issue #644, and ensures unified grid coordinates with PR #708 and #723.

PR Checklist

• [x] Closes #644
• [x] CLA signed. If not, go over here and sign the CLA
• [ ] Requires FONTLOG.txt to be updated (only small fixes, nothing to add to the log)
• [ ] Requires images to be updated (no new character and changes small enough that they probably wouldn't even show at that size)
• [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #644

Validation Steps Performed

Based on purely mathematical grid coordinates already used for octants, checked visually in VTT, Terminal Preview 1.20.10822.0 and Canary 1.21.240424002-llm, and Visual Studio editor.

Note there is another related issue that impacts some of those characters, but this PR at least provides the correct glyphs and improves the situation. I believe the remaining alignment issue to be a problem in Terminal itself as it works perfectly in Visual Studio editor, and the original Cascadia Mono 2111.001 exhibits the same issue. More details about this in #644. Having the proper coordinates at least ensures they won't induce in error someone trying to fix the Terminal rendering and expecting the alignments to work with a font using inconsistent glyphs coordinates. A believe this PR to be a step in the right direction.

[PhMajerus]

Visual comparison:

Before: Note how the characters do not have a consistent horizontal middle separation line.

After:

The reason the line isn't perfectly centered is due to another issue, apparently the "normal" glyphs coordinates are used instead of the "stypo" one for those characters, even though they are substituted in rclt. Previous builds of Cascadia exhibit the same issue, this is not due to this PR. This can be clearly observed in Terminal Canary, which apparently doesn't crop the top extending outside of the cell anymore:

Complete test sheet:

(Command: curl https://raw.githubusercontent.com/PhMajerus/Documents/main/CheatSheets/Blocks%20tables.txt ; curl https://raw.githubusercontent.com/PhMajerus/Documents/main/CheatSheets/More%20blocks%20tables.txt)

[DHowett]

Would you be able to give this a shot in Windows GVIM?

[PhMajerus]

For reference, here are the coordinates I'm using for all the mosaics:

[DHowett]

This is what I'm seeing in vim - but I need to go make sure that I have the right version installed and not a cached copy.

[PhMajerus]

That looks suspiciously like the older coordinates, feels like a caching issue to me.

[DHowett]

Actually, switching back to the old version the block glyphs get shorter. I wonder if the rclt/stypo switch is being compiled wrong (ala #721)

[PhMajerus]

I adjusted the coordinates of both normal and stypo versions, so whichever is used, it may be taller or shorter than expected if it messes up the substitution, but it cannot have the lower and higher pseudo-pixels of ▞ touching/overlapping at different heights.

The rclt substitution work for all my sextants, octants, etc..., and I checked some of the problematic characters and they were in the rclt of CascadiaCode-Regular.ufo, in the features/rclt.fea, and in features/rclt_PL.fea.

I'm testing now after just changing the code point for one of the problematic characters without changing its name, so hopefully I'll be able to see if they get substituted properly in Terminal when it's not part of the known classic block elements.

[DHowett]

To your other questions in the main thread - Terminal Preview DXEngine used to scale block elements, and Atlas used to cut them off. Now DXEngine is dead, and as of 1.21 Atlas doesn't do any clipping. If you leave builtinGlyphs off, you'll get a 1:1 "correct" representation of the font.

[PhMajerus]

The one thing that could think of is if there is some legacy hinting based on the previous outline that forces it to use the previous coordinates when snapping to pixels.

[PhMajerus]

I'm pretty confident Terminal is doing some special handling of classic block elements.

Here is an ansi-art using octants, and some of the characters are upper half blocks (U+2500).

Note how the two areas pointed in red don't align properly: (Test file: curl 'https://raw.githubusercontent.com/PhMajerus/ANSI-art/main/Unicode/R-Type%20(Octants).txt')

Now look at this version:

The only difference is this:

And the corresponding replacement of U+2580 to U+E000 in the text file! I didn't change any rclt, I just recompiled the font with that one codepoint change and everything else left as it.

It really looks like Terminal has leftover special handling of the classic block elements from back when tricks were used to make them fill their cell properly.

[DHowett]

@lhecker You may be the expert @PhMajerus needs here

[PhMajerus]

Yeah, I didn't dare to summon him, but I figured he'd be the one I need 😆 BTW, I'm available for a Teams call if you want to brainstorm over this faster than through a github chat.

[DHowett]

For gvim, I just installed the wrong binary out of my build output folder. GAH.

[PhMajerus]

For gvim, I just installed the wrong binary out of my build output folder. GAH.

Yeah, I didn't want to sound too confident, but that sounded like the most plausible explanation 😉

[DHowett]

You deserve the confidence ;)

[PhMajerus]

Oh, I have another demo file you might enjoy: curl https://raw.githubusercontent.com/PhMajerus/Documents/main/HowTos/HowTo%20Terminal%20Colors.txt

[lhecker]

As it turns out this API is "at fault": https://learn.microsoft.com/en-us/windows/win32/api/dwrite_1/nf-dwrite_1-idwritetextanalyzer1-gettextcomplexity Unfortunately, not using that API straight up doubles our CPU usage in the renderer, so I'd prefer not getting rid of it. Ideally we would find some way to mark the 0x2500 block glyphs as complex. As far as I can read DWrite's implementation, having it in gsub/gpos should mark it as complex. Otherwise, DWrite bases the complexity of glyphs on whether they have a cmap entry and if so if the codepoint is marked as simple/complex in the internal Unicode database (where the block glyphs are marked as simple).

[DHowett]

Nah, it looks like it is in gsub. We'll figure it out! Anyway, I'm merging this since it is correct.

Thanks all for playing!

[PhMajerus]

@lhecker Thanks for looking into it. I'm not familiar with DWrite to be of any help, but this seems to confirm that the problem isn't in the font, but instead in how DWrite handles the blocks characters that are part of the BMP.

FYI, more experiments explained in #644 seem to indicate that DWrite selects the non-stypo version of these glyphs which explains why they are taller than they should. This also means that if you stretch/squash them according to U+2588 "full block" as we discussed in https://github.com/microsoft/terminal/pull/16729, they should all render correctly as well. I'm just not sure how you would then handle the supplementary (Non-BMP) block-based characters as they cannot use the same scaling coefficients based on U+2588 because they, on the other hand, properly use their stypo variants.

Anyway, that seems to confirm that the best is for the font to use the proper coordinates, and then find a way for AtlasEngine to find a way to handle them correctly. At least then the pseudo-pixels will align perfectly, while if we keep the previous coordinates for the base block element characters, they don't align perfectly today and won't either when AtlasEngine achieve the goal. My coordinates used in supplementary (Non-BMP) block elements seem to perfectly align with the Builtin Glyphs ones, as expected of mathematically computed coordinates, so in the meantime that's one way to get all of them aligned perfectly.

[PhMajerus]

Thanks! Glad this could then be used to help test the future improvements in AtlasEngine. Now I'm impatient to get the next Terminal Preview with the new Cascadia bundled 😁