Monospacing broken by certain Unicode characters

[Kroc]

NB: TextMate is the only editor I know that gets this right!

Many kinds of Unicode characters are wider than normal, some have narrower width; in a monospace font, the width should always be fixed.

The fault in VSCode is that such wide/narrow Unicode characters throw off the spacing for the rest of the line -- making it impossible to get back to the same alignment as other lines! (tabs or otherwise)

TextMate displays all Unicode characters at fixed width regardless of their proportional width, including zero-width characters. This is the best behaviour for monospaced source code.

To avoid some level of confusion over zero-width characters (like Null, zero-width-joiner, bi-di codes &c.) being displayed at full width, whitespace codes could be used (such as when whitespace is visible) -- Unicode provides some visible symbols for non-visible characters, e.g. ␀ (0x2400)

So, ideally, what I'm asking for is:

• All characters, always the same width
• Better display of non-visible / zero-width characters if possible

I don't have access to TextMate at the moment, but you can use this file as a test, which was written in it: https://raw.githubusercontent.com/Kroc/DOMTemplate/master/domtemplate.php

Kind regards, Kroc Camen.

[bpasero]

Alex, not sure if this is a bug or feature request.

[alexdima]

For VSCode, it is Chromium that ultimately renders the text. We provide the font-family and other CSS options, but Chromium deals with the actual font and rendering decisions. In other words, we cannot at our level of abstraction and we should not (given the limiting DOM API) deal with character level positioning.

TextMate, by rendering each character at the same width might get itself into other trouble. How does TextMate render full width characters, since by definition these characters should be wider:

Ｈｅｌｌｏ  ｗｏｒｌｄ

The only actionable thing we can do is the second bullet point you mention: to better display non-visible / zero-width characters. This we can do by "massaging" the text we give to Chromium and substitute those characters with other representations.

Opening the link above or its source: view-source:https://raw.githubusercontent.com/Kroc/DOMTemplate/master/domtemplate.php in Chrome:

[Kroc]

Noticed this got updated; could this issue be solved by the font -- that is, could Microsoft design a font that renders all characters with the same width, even the esoteric Unicode stuff? (I assumed most monospace fonts just go "eh, that'll do" once they've got through the common stuff). Full-width characters could be either be exactly 2x width, or crunched down to 1x width; the different typographical style should be indicative enough.

I think there's a big difference between fonts intended for end-user reading, and a font designed specifically for editors that tries to make indistinct characters visually separable (e.g. en-space / em-space), whilst aligning to columns (critical only to programmers & editors) -- i.e. allowing the developer to think about the text at the binary level, which normal fonts obscure.

[alexdima]

This is an upstream issue (possibly within Chromium), and there are no known workarounds that we could implement on our side. This means there is no possible code change for us to do in VS Code sources to improve things.

To keep the number of issues in our inbox at a manageable level, we’re closing issues that have been on the backlog for a long time but haven’t gained traction or that are not actionable. Thank you for your time and understanding.

[roa-nyx]

Sorry to reopen this discussion but I feel it's just too important.

Recently the concept of watermarked content and code snippets (via use of zero-width characters) has become a topic of some scrutiny across various communities online.

Could this be considered a security threat?

Regardless I do agree that at least zero-width characters should be displayed as something visible to allow at-a-glance recognition of potential hidden dangers in copy+pasted content. This isn't limited to just copying from online sources, as I find even copying content from designs (Photoshop) sometimes will result in hidden characters being injected in content.

@alexandrudima you mentioned that the content could be massaged prior to Chromium's rendering of it. Is there any plans to implement this?

If not, another (possibly simpler?) feature might be to provide an option to automatically sanitise pasted content of zero-width characters. Perhaps the option can allow for those hidden characters to be converted to their visible counterparts or removed depending on user/workspace preference?

Right now Textmate, Vim, Emacs and several other code editors apparently support the display of zero-width characters, and I think it is a wise policy to adopt to provide a true full-service code editor.

[Kroc]

My suggestion for this would be to have a representative block for unprintable characters, much how the following symbol . has a background in GitHub, mousing over this would show a pop-up with the Unicode grapheme name, e.g. Zero-width Joiner, perhaps like the key pop-over on phone keyboards. This way the non-printable characters do not take up more width than one-character and they can each be examined without having to have a unique symbol for each.