Remember to have block and line drawing characters fill their cells properly

[PhMajerus]

Just a reminder for the new Windows Terminal, remember to make sure block and line drawing characters fill their cells properly at all zoom levels. This is really important for ANSI-art and TUI apps.

See color bleeding and horizontal lines in the larger Windows Terminal tab. Also, some characters are randomly replaced by � (<?>) characters when changing zoom level or when printing out lot of VT markup.

[zadjii-msft]

You know, I coulda swore @miniksa had a PR for this just last week, but I dunno what happened to it...

[PhMajerus]

Some more examples, comparing mlterm (running in Ubuntu on WSL with X410) with Windows Terminal

See it exhibits not only the block and line characters alignments and joints issues, but also identical colors rendering as different colors (the whole ANSI-art is using only the 16 palettized VT colors). Note also in the center, black (\e[40m) rendering as acrylic transparency.

You can get the test file from the following repository : https://github.com/PhMajerus/ANSI-art/blob/master/TestPattern%20ANSI.ans

[PhMajerus]

The identical colors rendering as different colors mentioned above has been diagnosed in #677 and hopefully fixed in #688.

[ghosty141]

I just tested it and it looks like the colors are now matching correctly. So, this is gonna be fixed in #688.

[ghosty141]

I did find another bug though, look at how some characters get rendered incorrectly if I didn't execute another command before showing the test pattern. For example: show pic -> incorrect chars clear -> show pic -> incorrect chars clear -> any command (even clear) -> show pic -> correct output.

One more thing, try zooming in (Ctrl mwheel) and at the right level of magnification the output looks perfectly fine.

[PhMajerus]

@Ghosty141 thanks for fixing and testing it, that's one thing out of the way.

What you're observing are the two other issues I described in the first message.

The terminal cells and the size of the block drawing characters do not match at all zoom levels, making them overlap or not fill their cell, creating visual horizontal lines. This is easily observed if you display █▀▄░▒▓, their heights won't be consistent. Strangely, trying these at different sizes in Word with the Consolas font shows them always filling the exact same rectangle, only Conhost and Terminal seems to stretch these characters at some sizes, sometimes by as much as 2x.

The last issue with the � characters appearing a bit randomly seems like a bug in Terminal itself, and is probably in the rendering engine as it depends on the zoom level and at the same time seems to always appear at the exact same place for the exact same buffer to render. Note that � could be U+FFFD, the "replacement character". So it could be drawn by some API when requested to draw invalid data.

[miniksa]

@PhMajerus, I'm aware of the randomish U+FFFD issue. I have an outstanding task to try to debug why that's happening. I'll get to it eventually.

[shanselman]

Interesting, is this related also to the way that F7 in cmd.exe renders itself with spaces/lines?

[miniksa]

F7 is implemented in the COOKED_READ logic for CMD.exe and actually changes which symbols it uses for the outline based on codepage... or at least it used to.

[DHowett-MSFT]

from @vblazhkun, #2006

Environment

Windows build number: [Version 10.0.18362.239]
Windows Terminal version (if applicable): 0.2.1831.0

Any other software?

WSL Ubuntu: [1804.2019.521.0]

Steps to reproduce

1. Install Pragmata Pro Mono Liga 0.827 font (any other Unicode font should work as well).
2. Open the Windows Terminal with WSL Ubuntu tab.
3. Download and display the UTF-8 Demo Text. $ curl https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt

Expected behavior

No visible interline gap (the screenshot is from Cmder 180626 with WLS Ubuntu):

Actual behavior

Visible sparse interline gap:

[shanselman]

Just checking in on this. This is Fira Code using GUI.CS

[vblazhkun]

@shanselman I have just checked with the latest dev build, but the inter-line gap is still there.

[DHowett-MSFT]

Some terminals, including VTE and Konsole draw the U+2500..257F (or even up to 259F) glyphs manually, rather than taking them from the font. This prevents such problems (and also avoids ugly looking antialiasing). I haven't seen any monospace font that would just look perfect on its own.

Interestingly, independently from each other, both VTE and Konsole came up with the idea of defining the look of most of these codepoints as a 5x5 matrix (where how the given 5x5 grid is stretched to the character cell is not necessarily the same for thick lines vs. double lines).

VTE link1, link2

Update: Konsole recently changed not to use a 5x5 matrix but other means of drawing.

from @egmontkob

[egmontkob]

If you're about to implement manually drawing these glyphs, please also consider the U+1FB00 block to be added to Unicode 13 (further info via the VTE issue).

[egmontkob]

Let's note that this issue now tracks two technically independent stories:

• A thin dark horizontal line between the background of every second or so text row, related to HiDPI;

• Line/box drawing glyphs (their foregrounds) not perfectly aligning.

[mdtauk]

@egmontkob And the possibility of the Terminal rendering the box and block drawing glyphs itself, so alignment will be perfect, no matter what Line Spacing or character cell widths and heights the current font is set to?

[egmontkob]

Yup, this is what I meant with my second bullet point :)

[iscooool]

The same issue here. But I can fix it by setting "fontSize": 13. my resolution is 2560 X 1600, scale factor 150%. when "fontSize": 12 after setting "fontSize": 13

[sywesk]

Same issue here, HiDPI problem: 1.75 scaling factor / 4k resolution.

[tryptamic]

@egmontkob And the possibility of the Terminal rendering the box and block drawing glyphs itself, so alignment will be perfect, no matter what Line Spacing or character cell widths and heights the current font is set to?

I think the terminal should not render the glyphs itself but use the normal ones from the font that is configured otherwise it breaks with the general appearance of the configured font.

And generally asked, is there any progress on this matter? I try to imagine why it is so complicated after all. All other consoles known to me seem to have no trouble with this.

[zadjii-msft]

At the moment, there hasn't been any specific progress on this. It's triaged into the 1.0 milestone, so we plan on getting to it before we release a 1.0 version of the Terminal, but there's been like 4000 other things that have also been raised since this issue was filed, so we've been a little busy and just haven't gotten to this yet. Sorry about that!

[SiarheiKuchukBelitsoft]

Wov, I see there're already so much work is done...

Anyway guys you are doing something awesome. Scripts UI refreshment is something entire software world would be happy to explore!

[miniksa]

• Drawing the blocks correctly looks like it could be related to #900 ish
  • https://github.com/microsoft/terminal/issues/455#issuecomment-490249051
• The gapping between the lines should be solved already by #5149
  • https://github.com/microsoft/terminal/issues/455#issuecomment-560280863
  • https://github.com/microsoft/terminal/issues/455#issuecomment-558489512
  • https://github.com/microsoft/terminal/issues/455#issuecomment-518455051

[miniksa]

* Drawing the blocks correctly looks like it could be related to #900 ish
  * [#455 (comment)](https://github.com/microsoft/terminal/issues/455#issuecomment-490249051)

Nope, I believe it has to do with the vertical line padding/spacing. I'll dig.

[heaths]

I just installed 0.11 and, while improved (most blocks are rendering without spaces), the solid arrow characters in Cascadia Code PL 1911.210 (latest) still show a space:

[miniksa]

Here's the good news, I've figured out a way to correct the box/line glyphs in the vertical direction:

Bad news, it's not perfect yet. The gaps are gone between the lines, but the diagonal shadings still don't line up. And I haven't actually figured out the right metrics here, this is just a proof of concept to see if I can find the knob WHERE I can adjust the heights.

Also, I haven't yet figured the little bit of horizontal gapping that occurs at certain resolutions and font sizes.

[egmontkob]

Diagonals suck big time, given the nonzero line width. E.g. if you want them to align perfectly, you do need to overflow to adjacent cells. Also figure out how the line should end (e.g. suddenly (rectangle shape) or with a half circle), which shape gives proper joining if they cross, or a / and \ meet each other, for all this to look "perfectly", whatever that means. Ideally without the drawing of a cell depending on other cells, as that would be pointless overengineering. There's no single perfect solution, but there are a couple of good enough ones.

[miniksa]

Thanks, Egmont. I'll probably defer perfect diagonals until post-1.0 in a follow on issue.

Digging in further this morning, @DHowett-MSFT and I figured out that something isn't authored quite right with Cascadia Code that is contributing here. We're looking into that too.

[miniksa]

Test fonts:

Screenshots

| Font Name | 0.11.1153.0 | `eb03caa3` |---|---|---| | Cascadia Code |  |  | | Cascadia Patched |  |  | | 3270-Medium |  |  | | Anonymous Pro |  |  | | DejaVu Sans Mono |  | | | Envy Code R |  |  | | Fira Code |  |  | | Iosevka |  |  | | monofur |  |  | | monofur italic |  |  | | Source Code Pro |  |  |

Font Name	fsSelection Flag on Bit 7	WinMetrics == TypoMetrics	Worked Before	Works with Line Gap Fix
Cascadia Code	❌	❌	❌	❌
Cascadia Patched	✔️	❌	✔️ (but squished)	️✔️
3270-Medium	✔️	❌ (win 1013 tt 1000)	❌	❌
Anonymous Pro	❌	✔️	❌	❌
DejaVu Sans Mono	❌	❌ (win 2384 typo 2048)	❌	❌
Envy Code R	❌	❌ (win 2370 typo 1896)	✔️	✔️
Fira Code	✔️	✔️	✔️	✔️
Iosevka	✔️	✔️	❌	❌
monofur	❌	✔️	❌	❌
monofur italic	❌	❌ (win 2400 typo 1950)	✔️	❌
Source Code Pro	❌	❌(win 1257 typo 1000)	❌	❌

Font Name	designUnitsPerEm	ascent	descent	lineGap	capHeight	xHeight	M advanceHeight	M verticalOriginY	M topSideBearing	M bottomSideBearing	█ advanceHeight	█ verticalOriginY	█ topSideBearing	█ bottomSideBearing
Cascadia Code	2048	1977	480	0	1420	1060	2048	1568	148	480	2048	1568	-240	-270
Cascadia Patched	2048	1568	480	409	1420	1060	2048	1568	148	480	2048	1568	-240	-270
3270-Medium	2048	1884	514	0	1307	1004	2048	1521	214	527	2048	1521	-363	13
Anonymous Pro	2048	1675	373	0	1305	932	2048	1675	370	373	2048	1675	370	373
DejaVu Sans Mono	2048	1901	483	0	1493	1120	2048	1556	63	492	2048	1556	-20	-20
Envy Code R	2048	1896	474	0	1422	1106	1896	1422	0	474	1896	1422	-553	-158
Fira Code	1950	1800	600	0	1377	1053	2400	1800	423	600	2400	1800	0	0
Iosevka	1000	977	272	0	735	530	1249	977	242	272	1249	977	0	-1
monofur	2400	1952	452	0	1500	1051	2404	1952	452	452	2404	1952	2	2
monofur italic	2400	1950	450	150	1500	1051	1950	1500	-1	449	1950	1500	-450	0
Source Code Pro	1000	984	273	0	660	480	1000	750	94	250	1000	750	-50	50

[miniksa]

OK, so, the line gap thing isn't really working. So now I'm going crazy in Excel finding something that I can use to adjust the baseline, the overall height, or whatnot either for the entire font or for the glyphs only in the U+2500-U+259F range (with the ascenderOffset flag inside the CustomTextLayout)

[miniksa]

Another note: Anonymous Pro has line drawing chars but no box drawing chars, so that's a fallback and probably why it's looking extra weird.

[PhMajerus]

@miniksa This is amazing compared to the previous rendering, great work! The shading blocks are a very difficult case, and if you look again on my first screenshot of that test pattern using mlterm, you'll see they don't line up perfectly either.

The problem is the original EGA/VGA font (8x14 font in ROM on old graphic cards) had the pattern perfect for alignment, but even the CGA (8x8) was somewhat off for the dark shade block. If I remember correctly, the default font used in Win95 console needed an alternate of shaded blocks to fit properly side by side (as in, inverting the foreground and background colors and using the opposite shading block to make their joins merge correctly using an invisible checkerboard pattern).

So I wouldn't worry too much about it for v1, the result you got already seems on par with other mature terminal emulators.

I find the colors narrowing a bigger problem for ANSI-art and Text UI, especially the convertion of black background to transparent background when they were explicitly black instead of CSI 49m.

Can you include the following in your tests ? echo -e "\e[30m\u2590\e[47;7m#\e[27;49m\u258C\e[m" Should display silver # symbol over a black square even with acrylic blur background, but it is shown as ▐#▌ instead, the background of the # cell being converted to transparent instead of staying black like the two sides.

[zadjii-msft]

Can you include the following in your tests ? echo -e "\e[30m\u2590\e[47;7m#\e[27;49m\u258C\e[m"

That might actually be #2661

[PhMajerus]

@zadjii-msft Yeah, your ColorType IsDefault flag should take care of it, but since we're getting close to v1 I'm trying to avoid something slipping through that gets noticed too late... Like #399 is still bothering me every single day in 20H1.

I just want to make sure you have a test case where the background color of a cell happens to be the exact same color as the default background color as seen by legacy conhost concepts, but needs to stay solid opaque, while CSI 49m gives acrylic transparency. Really hope these VT stuff will all work for v1.

Thanks for all the work on fixing VT edge cases, It is looking great so far.

[miniksa]

OK I spent all day fiddling around.

I have the math that matches the results from the states above.

Unfortunately I put the "with gap correction" to the left of "w/o gap correction"... but it goes as follows.

If there are 3 green boxes next to each other for that font... it's going to look fine in the Terminal. If there are not, it will gap somewhere. (2nd and 3rd columns describe where it's gapping.)

The left most one is green if the designed height of the █ glyph is tall enough to fit the cell space we've given it after we've ceiling'd the baseline so it lands on a solid pixel and ceiling'd out the cell as a whole to an integer height.

(We integer-ify the baseline to ensure that the bottom of characters sits nice and crisply on a pixel. We integer-ify the cell height out to a perfect integer to enable scrolling offload.)

The middle column is green if the top of the █ glyph will touch or stretch outside the top of the cell after it's been drawn at the modified baseline. The right most column is green if the bottom of the █ glyph will touch or stretch outside the bottom of the cell after it's been drawn at the modified baseline.

There were 4 triple-greens in the old algorithm (which aligned with the specific fonts in the test data). And there are 3 triple-greens in the linegap-adjust algorithm (also aligning with the specific fonts in the test data.)

I believe there's two steps to success here:

1. Given that boxes/lines don't specifically care about the rounded baseline for the solid pixel, translate the glyphs back upward to where they should have been. Match the centerpoint of the glyph to the centerpoint of the cell.
2. For glyphs that still wouldn't be big enough after translating them into the centerpoint of the cell, use a scale transform to fill the cell (or slightly overscan it by a few DU to guarantee it looks good).

Beyond that, I've written some code today that:

1. Adds an analysis step to CustomTextLayout that can identify runs of box/line drawing characters and segment them out into their own runs.
2. Tag all of these identified runs with a payload object that can direct custom rendering.
3. Added the custom rendering object to the runs so it is carried through.
4. Piped custom rendering objects the whole way down through CustomTextRenderer
5. Repurposed the unused "draw glyphs manually" method to become our box drawing method. Corrected it to draw the outlines and the fills. Set sample translate/scale transformations between geometry source and sinks. Tested this out manually and showed that it visually looks correct... as long as we have the right metrics.

My goal for Friday will be:

1. Take the math out of the Excel spreadsheet and code it into the analyzer so it can determine the translation and/or scale factors needed to adjust box glyphs so they look proper.
2. Expand the custom object to hold this data and carry it down to the renderer.
3. Update and verify this works against the custom text renderer by filling the geometry transform matrix. Then refine by: A. Don't do all this if they would already fit and look just fine. What a performance nightmare! B. See if I can generalize the box/line drawing data into character classes that can be loaded/cached on font change instead of loaded in real time. C. Ensure this still works for font fallback scenarios as some of these fonts implement SOME of the line/box glyphs and not others and we'd want the fallback glyphs to look decent while we're doing all this complex work anyway.

And the backup plan is:

• Instead of actually determining scale/translate transform metrics for the geometries, instead implement a U+2500-U+259F-part switch case that just draws them with FillRectangle instead and ignores the glyphs in the fonts. I don't want to do this because I like the style/flavor that fonts can provide here. But it's a backup plan to all this math.

Either the main or backup plan solution should eliminate all gapping in both the X and Y dimensions for all of these glyphs no matter what font is used and no matter if that font implements all of these glyphs or not.

[PhMajerus]

That’s a lot of work to work around font rendering issues.

It seems that having a lot of complicated mitigation code might make it more difficult for font designers to test their fonts, as they might be tinkering with their design to try to make something that looks good in a few test apps, and the different heuristics used by the code might appear random from the font designer point of view. I believe we need a switch to explicitly turn off all mitigation code paths so they can test their fonts against the more predictable optimized renderer and try to build a font that doesn’t rely on fallback paths. That would be both in settings.json and at the renderer XAML control level for other apps hosting the Terminal.

About using the font glyphs versus a fixed set of block elements and line drawing glyphs, I agree using the font to allow fonts to have their own identities.

For that idea, I have a suggestion that I believe would provide added value to the Windows Terminal in general and provide a fallback mechanism for our current issue when needed. Most fonts won’t provide all the glyphs that the Terminal can display, we’ll have Latin font, CJK font, Arabic font, Korean font,… at this time we can select a single font, and the Terminal uses some other fallback font for missing characters. But how about letting the user specify a set of fonts by Unicode blocks/ranges? Just like MS Word allows the user to select a Latin font and another Asian text font, the Terminal could allow the user to select a main font as well as explicit alternate fonts for the CJK block, the Arabic block, the Math symbols block, and of course, the Block elements and Box drawing blocks. This would provide better control over the look of mixed scripts in the Terminal, and allow an optimized font that doesn’t need the performance hit of the fallbacks to be used for blocks and lines if the user’s favorite font doesn’t provide good ones. Instead of a custom-drawing FillRectangle-based solution, Windows Terminal could ship with a font that provides just the block elements and line drawing glyphs to be used only for those Unicode blocks that works with the optimal rendering code path. This also has the added benefit of making it possible to separate things like the Powerline and Nerd glyphs into separate fonts. Instead of merging them into special versions of Cascadia Code, and missing them when using a font that didn’t merge these glyphs, the Terminal could simply use Powerline-only and Nerd-only fonts for those Unicode blocks/ranges according to the same user-defined “fonts set”. It makes even more sense with the extra Symbols for Legacy Computing added to Unicode 13.0 (https://www.unicode.org/charts/PDF/U1FB00.pdf), as we could have for example Commodore 64, Apple II, Atari 400/800, etc... fonts designed just for the block that provides the legacy computing symbols.

[miniksa]

Thanks for your response, @PhMajerus. I still believe it's up to me to fix it in a complex way because I'm the one breaking it in the first place for many of these fonts. I'm the one adding additional vertical line padding in some cases and throwing off the prescribed dimensions of the fonts. I do it in order to use the scrolling mechanisms of Present1, maintain the integer-pixel cell dimensions that most of our code and client applications rely on, and to try to preserve the crispness of the glyphs through these adjustments. I know the "just don't mess with it" solution would probably resolve this on its own, but then I'd have to give back all the performance I got from differential drawing and Present1 and I'd have a new host of problems with hit testing and line wrapping and API-mapping.

I do believe in adding a fallback mechanism including being able to specify which ranges it applies to, but that's definitely going to come after 1.0 with the complexity of the settings configuration required. Such a thing will also still require me to adjust the analysis steps to further parse and identify the ranges to apply each font to, so that would have to be done for either path.

Finally... I think adding a setting to turn it off might be advisable and possible.

[miniksa]

OK as of this (https://github.com/microsoft/terminal/commit/fe4e1c7fa02a38b158ca331f8785b7bc1c1217e1), here's what we have: EDIT: also as of this (https://github.com/microsoft/terminal/commit/f6e4542340f100762573f8ce8f95887d7724dd08), the next column:

Test fonts:

Screenshots

| Font Name | `fe4e1c7f`| `f6e45423` | |---|---|---| | Cascadia Code |  | | | Cascadia Patched |  | | | 3270-Medium |  | | | Anonymous Pro |  | | | DejaVu Sans Mono |  | | | Envy Code R |  | | | Fira Code |  || | Iosevka |  | | | monofur |  | | | Source Code Pro |  | |

Font Name	fe4e1c7f	f6e45423
Cascadia Code	✔️	✔️
Cascadia Patched	✔️	✔️
3270-Medium	✔️	✔️
Anonymous Pro	❌ (fallback)	❌
DejaVu Sans Mono	❌ (vertical adjust?)	✔️
Envy Code R	✔️	✔️
Fira Code	✔️	✔️
Iosevka	✔️	✔️
monofur	✔️	✔️
Source Code Pro	❌ (vertical adjust)	✔️

• [x] - Figure out what DejaVu Sans Mono's deal is - Scale + translation
• [x] - Figure out vertical adjust in Source Code Pro - Scale + translation
• [ ] - Find the weird extra horizontal pixel here and there
• [ ] - Make this work down font fallback route
• [ ] - Use cached adjustment value if font fallback font matches the O.G. font

[miniksa]

Got the font fallback part. Generating draft PR now. Here's an Anonymous Pro sample. Not perfect, but pretty darned good.

EDIT: Do note that most of the box glyphs are fallbacks from Segoe UI Symbol.

[miniksa]

OK, I've solved most of it. There's still a few circumstances with a minor vertical banding between blocks in the horizontal direction, but I've filed #5816 as a follow on for that because the PR at #5743 adds so much goodness for a majority of cases.

[ghost]

:tada:This issue was addressed in #5743, which has now been successfully released as Windows Terminal Release Candidate v0.11.1333.0 (1.0rc2).:tada:

Handy links:

• [Release Notes](https://github.com/microsoft/terminal/releases/tag/v0.11.1333.0 (1.0rc2))
• Store Download