[css-text] Show invisible characters

[Crissov]

Especially for rendering source code and editing text, non-printing whitespace and control characters should optionally be shown ”with ink“.

Although these sometimes use a deemphasized, grayish color, I think this would best be done within text-transform or perhaps even in a new sub-property. Since @text-transform #3132 has not gotten anywhere yet, I‘m proposing new bikesheddable values:

• visible-spaces: show U+0020 SPACE as middle dot, U+0009 HORIZONTAL TABULATOR as right arrow (at least in LTR contexts), and perhaps more, especially from U+2000…6F
• visible-keys: show U+0020 SPACE as U+2423 ␣
• visible-controls: show C0 control codes, i.e. U+0000…20 and U+007F (but not C1, U+0080…9F) as their respective Control Picture from U+2400…21, e.g. ␠
• visible-whitespace: show U+0009…0D, U+001C…1F and U+0085 as …
• visible-breaks: show CR and LF as PARAGRAPH SIGN, and LINE BREAK as arrow, keep their line breaking behavior
• visible-lines: show CR, LF and CR+LF as U+2424 ␤

PS: It is quite possible this was discussed before, but I could not find a respective issue, so it might have been only on www-style long ago.

[johannesodland]

Great proposal! That would be helpful.

Including U+00A0 (No-Break Space) and U+00AD (Soft Hyphen) would also be beneficial, but I’m not sure how they should be represented.

[tabatkins]

A wrinkle here is that showing the characters shouldn't affect the actual text formatting, especially the linebreak ones. They need to be displayed as purely visual overlays, rather than like normal characters.

[Crissov]

I think it is sometimes acceptable or even expected that visibly showing space characters would affect spacing, probably also depending on the settings for whitespace collapsing, i.e. does substitution happen before or after collapsing.

[frivoal]

I don't think it'd be appropriate to text-transform a line break into a character that visually indicates that there should be a line break, but doesn't actually provide one. All in all, the feature seams useful, but it's not clear to me that text-transform is the right property for this.

[faceless2]

This is the "I want to do complex replacements on text" problem we've seen a few times before. https://github.com/w3c/csswg-drafts/issues/4875

[johannesodland]

To further elaborate on the “text editing” use case:

When editing text for publication it is necessary to be able to insert and inspect invisible characters. Without this capability, such characters may be inadvertently omitted, compromising the quality of the final text.

While tools like Word provide solutions to view invisible characters, the web lacks a standardized method. Users and publications rely on every product implementing a custom solution for text replacements, but support is mixed. This missing support impacts language and typography considerably.

For instance, in Norwegian, a space serves as a thousands separator. To prevent line breaks within numbers, we utilize a no-break space, e.g., “1 000 000”. However, the challenges in inputting and inspecting the nbsp have led many web publications to deviate from standard Norwegian typography, often using a full stop instead.

Furthermore, languages like Norwegian frequently use long compound words. Here, soft hyphens are vital to allow soft breaks and prevent unsightly text wrapping. Yet, the inability to visualise these soft hyphens means many publications lack proper text breaks. Sometimes a single word can be left on the preceding line as a whole compound word is pushed to the next.

For editing purposes it is not a problem that the visualised characters impact the formatting. Furthermore, this use case can be solved by other means than full "complex replacement on text".

Without improved support for viewing invisible characters, the web could inadvertently alter local typographical standards, much like the typewriter did in its era.

[frivoal]

No question it's useful in an editor. The more pressing question, in my mind, is whether that should be provided by the web platform itself, or whether this is to be handled by the web-based editor itself. We have a whole history of tension between trying to provide high level features baked into the platform for editing, and providing the right primitives to enable (js-based) editors to built such features effectively.

[johannesodland]

I agree that many features are better left to higher level components like editors.

However, the reality is that it is not feasible to expect every framework, editor component and cms to implement this functionality.

The content we published are sourced from a multitude of systems. We have requested this feature from the cms’es we’re using. The solutions they provide are either lackluster or non existing. In addition there’s a large number of custom form fields and editors that would all have to be customized to provide this functionality.

It feels rather hopeless.

The end result is that our language and typography is slowly changing due to poor technological support.

Having a standardized platform provided way of displaying otherwise invisible characters would go a long way towards amending this.

[SebastianZ]

We have a whole history of tension between trying to provide high level features baked into the platform for editing, and providing the right primitives to enable (js-based) editors to built such features effectively.

And as @tabatkins pointed out earlier, this is purely a visualizing feature. Though as @johannesodland wrote, it should also be capable of displaying special characters like soft hyphens, zero-width spaces (U+200B, U+FEFF), etc. So it might affect text formatting in some cases. To me, showing invisible characters seems a natural fit for CSS. And it is something where CSS should pave the cowpath.

Because of the requirement of being a purely visual effect, I also think text-transform may not be the right fit. Maybe a new text-visibility or character-visibility property or something along those lines.

A few questions that come to my mind:

• Should the visualizing characters be predefined?
• Should authors be able to change them? If so, to what degree (i.e. only replace the predefined ones or also declare new ones or remove predefined ones)?
• Should they be stylable? (Many editors display them differently from the rest of the text. So if the answer is yes, then we'd need a new pseudo-element for that.)
• Should this be a pure CSS solution or a combination of CSS and DOM APIs like for the Custom Highlight API?

Sebastian

[Crissov]

I think there’s also a math use case with these:

• U+2062 INVISIBLE TIMES ⇒ × or ·
• U+2063 INVISIBLE SEPARATOR ⇒ ,
• U+2064 INVISIBLE PLUS ⇒ +

Within (editor) text set with a monospaced font, it may be desirable to disambiguate the various spaces in U+200x and a handful others by showing visible glyphs for them. Since these are not readily available as Unicode characters, text-transform may indeed be the wrong place to implement this. The same goes for many (keycap) symbols from ISO 9995, ISO 7000 or IEC 60417 which are available for characters like ZWJ, ZWNJ, CGJ, NBSP.

* U+200B ZERO WIDTH SPACE  * U+00AD SOFT HYPHEN  * U+200C ZERO WIDTH NONJOINER  * U+2060 WORD JOINER  * U+034F COMBINING GRAPHEME JOINER  * U+0020 SPACE  * U+00 NON-BREAKING SPACE 

[oliviercailloux]

Maybe a silly idea, coming from a Unicode FAQ. Perhaps the rendering system might allow to opt for a special rendering mode where invisible characters would actually display simply by using a font that contains visible glyphs for normally invisible characters? So that instead of “replacing” U+0020 SPACE by U+2423 ␣, the rendering would simply use a font containing a glyph for U+0020 that represents a space, similar to ␣.