Fixing responsive text tooling

[scottkellum]

The issues with current text styling methods

• Typesetting decisions are based on text column width, not viewport width, but we only have tools to change typesetting based on viewport width with media queries and viewport units. This results in excess styling for text as it needs to be styled for each area of a layout it appears in.
• There is no way to manipulate font variation settings fluidly as layout changes with CSS. There is a draft for optical size to be changed dynamically but there is no tooling for authors to change width, weight, grade, or any other axis in a fluid manner even though this is expected functionality from many font authors. CSS value typing prevents calc() from being useful here.
• Linear scaling requires lots of adjustments and excess breakpoints. With CSS Locks you will find smaller breakpoints are ignored, but if you were to adjust typesetting at watch sizes you would find more locks are needed. Scaling on a curve, like cubic bezier would be ideal.
• Min(), max(), and clamp() still require lots of un-intuitive math to figure out. Figuring out the right mix of fixed length units + viewport units to get the slope of text scaling to be just right can require either lots of guess and check adjustment or writing a formula to have min and max fall at specific points. Also, all this math gets thrown off by users changing their font size.

Possible solutions

• The first point can be addressed in part alongside the element/container query spec work. My opinion on how this should be done is leveraging layout contexts defined by positioning elements. There is precedent for layout properties to change their behavior in these contexts, as well as z-index stacking to reset. It also simplifies how you reference a context as there is only one container to be referenced, the stacking context that is positioned.
• There is a draft for scroll-linked animations in CSS. Expanding this to X-linked animations and allowing element width to be an input to an animation would solve all the problems here as well as providing tools for front end developers beyond typography.
  • The following is a nice to have, and not core to solving this problem: I would also raise issue that keyframe animations should accept units other than %, including length units when they are typed as length, to allow for a better author experience and finer control.

You can find a prototype of the general concepts outlined here: https://codepen.io/scottkellum/pen/OJPPOGK?editors=0100 — note that this proposal goes beyond what is shown here and aims to take advantage of the potential of native CSS as opposed to working with existing paradigms.

IP Disclosure

Typetura (Scott Kellum) has two parents regarding applications on top of this approach US10769348 and US10592087. There is prior art for the approach itself in the form of FlowType.js and Textblock.io along with the Typetura JS engine.

[lianghai]

Scott, as web layout isn’t the field I’m most familiar with, I don’t immediately know how to follow up on this proposal. But you’ll be welcome to brief the attendees on this proposal at the upcoming Meeting 4.

[scottkellum]

Thank you @lianghai. I’ll admit, I am more interested in the text aspects of this group than the font aspects of this group. Not that I don’t care about fonts, I care a lot about different ways to typeset with them! So much room for improvement in these areas. As layout is getting a lot of love (with grid and flexbox) and fonts are getting a lot of attention (variable fonts etc) text needs a lot of attention to catch up to both layout and font tech in my opinion.

[lianghai]

No need to “admit” anything. The whole point of setting up this CG is to encourage industry-wide communication and cooperation.

As layout is getting a lot of love (with grid and flexbox) and fonts are getting a lot of attention (variable fonts etc) text needs a lot of attention to catch up to both layout and font tech in my opinion.

“Text”, as a major subject in this industry, has a very broad spectrum of meanings. Better to clarify what exactly you refer to, when pitching other participants. Prematurely dividing concepts into “font”, “text”, and “layout” isn’t quite helpful. My initial impression of your proposal is that it’s about some interaction between block layout and inline font formatting—or is it?

[scottkellum]

Glossary of what terms mean to me:

• typesetting: font formatting, text styling, line-height, font styles.
• layout: arrangement of block level elements on a page.
• font: A file that contains a typeface.
• text: characters/glyphs presented on a page.

This proposal is about the effects layout has on typesetting and text. To reiterate above with the glossary here, layout and fonts have seen tons of improvement over the past few years but typesetting and text tooling hasn’t evolved to keep pace with layout and font tech. CSS lacks the adequate tools to take advantage of things like variable fonts or respond have text respond to layout changes introduced by things like flexbox and grid.

This proposal is to explore ways text styles can respond to layout changes and take full advantage of new font features.

[simoncozens]

I (personally) see this particular CG as operating at a (much) lower-level than block level layout, for which the CSS WG might be a better home. But this is why we have a triage process.

[tiroj]

Responsive typography inevitably links lower-level word and line layout with higher level text block and device frame layout, where changes in the latter can and should produce adjustments in the former. So I think understanding how that happens, and ensuring predictable and interoperable behaviour—taking into account that this will make use of font data—is within scope.

For example, the recent leading-trim property proposal relies on font data—and assumes this data is correct—to improve text block alignment, and announcement of the proposal included mention of the possibility of defining new font data to enable leading-trim to work for additional, non-European scripts. So that is a case where a desire to do something that has affect at the text block layout level not only depends on data at the lower, font level, but may make unwise assumptions about existing data and overlook prior ideas for new data—e.g. BASE table v2 ideas around script- and language-specific vertical metrics—that have taken place in closed discussions.

[scottkellum]

Thanks @tiroj,

Yes the leading-trim proposal was one that peeked my interest in joining this group.

Something that might concern more people here is that variable fonts still lack some basic and expected interpolation control in CSS. There is currently no way to interpolate an axis as the browser scales (without JavaScript). The behavior people expect from variable fonts on the web doesn’t exist.

[simoncozens]

So it turns out that I have a lot to say about this. The reason I haven't said very much is that I was supposed to be giving a talk about all this at a conference but it's got cancelled due to COVID!

The first thing is that responsive layout obviously interacts very strongly with variable fonts but also goes beyond variable fonts. For example, prior to variable fonts, there was a facility in OpenType (the JSTF table) which expected to be able to stretch and shrink certain glyphs as well as make certain substitutions based on the needs of the layout table. But it was never implemented, and I think there are several reasons for that. One is that (prior to variable fonts) the mechanism for stretching and shrinking glyphs was never defined.

But more fundamentally, there is no mechanism within the layout pipeline for information to flow from layout engine to shaper. (There's a similar story behind the jalt - justification alternatives - feature that was never implemented, and never can be for the same reasons.) The pipeline goes shaper to layout engine. The shaper determines how wide things are and hands a bunch of appropriately-sized boxes to the layout engine, which then tries to put them into lines. But for JSTF or jalt - or any kind of responsive text - to work, the layout engine and shaper need to have a "conversation" about how wide things should be. A good example of this problem is DJR's Fit font. You can adjust your browser size on his website and the text will be stretched or shrunk appropriately but the lines will never reflow or be broken in a different place. And they can't, because the line breaking engine doesn't "know" about the potential of shrinking the glyphs so much that another word can fit on the line.

So this is why John's discussion in #24 of post-linebreaking substitution is, in my opinion, misguided. In a sense it is the Hz-Program strategy: break the lines according to the normal principles, then "deform" the glyphs to reduce the line-breaking badness. But to achieve optimum line breaks in the first place - for DJR's Fit to ever reflow - you will need to take advantage of the variability of glyphs in the same way that best-fit line breaking algorithms already take advantage of the variability of spaces. So you need to have that "conversation" with the shaper before line breaking happens.

Now with advent of variable fonts there are a number of things that a line-breaking algorithm can do to determine the best fit: stretching and shrinking spaces as normal, but also stretching and shrinking glyphs (using variable font axes), and even substituting alternate glyphs as suggested in JSTF. (Go play with that example, I'll wait.) In some cases it will have to combine elements of each - Arabic is a great example where ideally you want a balance between contextual alternates as a priority, then stretching spaces (a little bit), then kashida stretching.

In other words, layout engines - and we're talking web layout engines in particular - need to be majorly rewritten in order to support variable font justification really well. This is because right now layout engines assume that text is static and spaces are variable. To work with variable fonts, they will need to see space as variable and text as both variable and substitutable. The old "boxes and glue" model doesn't work so well in this world any more. (It's all glue.) This diagram shows the sort of calculations that such a line breaking engine will need to do:

(Boxes show the choices available at each stage, and white boxes show the optimum collection of boxes selected by the algorithm to minimise the line breaking badness. There's eight separate permutations it needs to check out, and that's even without thinking about the variability of the glyphs.)

When variable fonts came out, I wanted to add support for them to SILE, but I was stopped by two issues - the first was that getting a variable font into PDF requires embedding static instances and I couldn't do that at the time, but the second was this matter of rewriting the line breaking algorithm. I started https://github.com/simoncozens/newbreak as a project to look at how to implement a new line breaking algorithm with variable-width text. I think there's a lot of potential in that algorithm but I need to integrate it into SILE (and of course, into every browser...) to be sure.

dombreak, which is the Javascript/HTML implementation of newbreak, has that "conversation" with the shaper like this: you tell it how you want to vary the text - for example, which variable font axis to choose, or which glyphs to substitute - it then asks the shaper to each "variable" run of text at various different combinations, before working out the optimum set of substitutions, set of stretch values and space stretchiness to optimise the line break. (You can also tell it what order to try and at what rate things should stretch - "first try all the justification alternates, then stretch spaces and kashidas in parallel but with kashidas stretching at twice the rate that spaces stretch.") That's how it implements the "conversation".

In short, I think what you want is worthy but it's asking an awful lot. We're not just talking about exposing some properties to CSS, but we're also talking about fundamentally rewriting the layout engine code in browsers and other applications to support variable text as well as variable spaces. I think we should do it, but as you say, it's a long-term goal.

[khaledhosny]

(There's a similar story behind the jalt - justification alternatives - feature that was never implemented, and never can be for the same reasons.)

InDesign implements jalt (at least for Arabic) and I know of another implementation from a research paper. I don’t know much about both implementations, but I suspect it is done post shaping and line breaking (like one of the glyph expansion modes in PDFTeX), basically lines are broken as usual that after line breaking jalt alternates are tried to fill as much of the remaining space as possible then falling back to expanding space. Not very clever but probably can get the work done most of time.

[scottkellum]

Notes from the October 19th meeting:

We discussed the scope and goals of this initiative. It spans various initiatives and may be better suited for the CSS WG. There is a lot of crossover for other working groups and the layout spec. I don’t expect his group to find a solution to these issues, but instead explore what we need for fonts and text from layout and lobby the CSS WG on those points.

• Simon and John discuss impact of variable fonts on text justification. On a line-by-line level we need to understand the layout constraints of the width, and how to proportionally adjust the wdth or similar axis to justify appropriately.
• Laurence responded to the limitations of clamp and calc and the lack of math functions like sin() so that scaling can happen on a curve. (Note that CSS unit typing is also a limitation of calc and font variation settings)

Related APIs and tools:

• Font variation settings in Fonts Module Level 4
• Resize observer, for identifying containers
• Houdini for extending CSS functionality
• CSS calc math for scaling fonts
• Typetura JS engine for typesetting interpolation

Other proposals and related issues:

• Proposal to abstract interpolation in CSS beyond animations, as a prerequisite for
• Issue stating lack of expected control over font-feature-settings
• Initial outreach to expand the Animation spec to cover this use case

[scottkellum]

@simoncozens I wasn’t even thinking about justification as an application for this, but this is super interesting and applicable stuff and I appreciate you sharing this in detail. When talking with typographers the top complaint I hear is that the web doesn’t have great hyphenation/justification/line breaking and that can be expanded on through this as well as other issues. I’m curious if this justification proposal can be achieved at a lower level in the text shaping? As a CSS author I’d love to write something where you can proportionally scale justification settings, maybe like justify: 1 1 0 where the first digit is proportion of change in word spacing, the send digit is the proportion of letter spacing, and the third (optional) digit is the proportion of width axis that goes into justification. But I feel like this might merit another issue. EDIT: this is kind of a thing but it’s a pretty blunt tool right now: https://developer.mozilla.org/en-US/docs/Web/CSS/text-justify

The old "boxes and glue" model doesn't work so well in this world any more. (It's all glue.)

I also really love how you phrase this.