[avar2] interaction with variable fea

[dfrg]

We are currently working on variation support in our fea compiler and have run into questions regarding how this should work when a font contains an avar version 2 table.

The current spec proposal for variable fea isn't explicit, but appears to indicate that axis coordinate values are defined in user space. Given that subsets of axes can be provided for value records/condition sets, and avar2 does not generate a unique mapping to normalized coordinates for subsets, compiling layout tables appears to be impossible with this approach.

Is this analysis correct? If so, are there any suggestions for the best path forward? The initial issue proposing variable fea syntax seemed to define coordinates in normalized space which would solve this problem at the cost of some ergonomic pain.

[behdad]

Humm. You are correct I'm afraid :(.

[behdad]

IMO the coordinates should be in design space. Which would be weird given that the "design space" concept mainly exists in .designspace documents and elsewhere, but not in OpenType / .fea. But that is the correct unit to use here for ergonomics. We don't want user space IMO.

[dfrg]

Thanks, this makes sense to me as well if we consider .fea files to be sources and “design space” to be the canonical variation space for sources.

[behdad]

IMO the coordinates should be in design space.

The rationale is that if the user changes the avar mappings, the variable values here should NOT change! So the current behavior is buggy at best.

[behdad]

We can define that for a .fea file that is not associated with any design-space, the values are in normalized coordinates.

[rsheeter]

Design coords are notably mostly for having simple translation to normalized. Historically these units have been an endless source of confusion.

Why not just say the values are always normalized? - simple, clean, and an AT can still present some alternate interface and generate or alter fea if it really wants.

[behdad]

It just makes it much harder to write them by hand, if that's a supported scenario.

[davelab6]

Authoring Tools handling ergonomics isn't a panacea, given community passion for writing text file versions of fea, designSpace XML, etc

[rsheeter]

I agree it would make it somewhat harder to handwrite.

Todays main source formats tightly couple user and design units. Do we know how that will change for avar2? Is it certain design:normalized will remain a simple mapping?

Sometimes being design units sounds guarranteed to cause confusion unless which unit you are using is very clearly specified when you write your fea?

I believe, though I can't seem to find I citation, that I have heard it claimed a fea file is meant to be more of an assembly for features. Introducing a new unit that is source specific seems out of line with that idea. We could abandon that idea ofc, but lets make sure we do so intentionally if we do.

EDIT: https://github.com/adobe-type-tools/afdko/pull/1350#discussion_r620445162

I think Just wasn't at the meeting where we all agreed that FEA should be regarded as assembly-equivalent, to be emitted by programs, and he's still trying to make it easy for humans to write. ;-)

[rsheeter]

https://github.com/adobe-type-tools/afdko/pull/1350#discussion_r636008312 specifically says user coords. Presumably it did not consider avar2 complications.

[behdad]

adobe-type-tools/afdko#1350 (comment) specifically says user coords.

Yeah, but as per https://github.com/harfbuzz/boring-expansion-spec/issues/94#issuecomment-1604770895 I think that's just wrong.

[rsheeter]

adobe-type-tools/afdko#1350 (comment) specifically says user coords.

Yeah, but as per #94 (comment) I think that's just wrong.

Under avar2 I agree, just highlighting because OP says it is unclear what unit is intended.

[behdad]

adobe-type-tools/afdko#1350 (comment) specifically says user coords.

Yeah, but as per #94 (comment) I think that's just wrong.

Under avar2 I agree, just highlighting because OP says it is unclear what unit is intended.

No. I mean it's already broken. If user tweaks the avar1 mapping, the values here should not change meaning. Otherwise they all have to be readjusted. That breaks the desirable condition that designer can tweak avar mappings at the end of design work freely, without needing to modify the masters.

[rsheeter]

That breaks the desirable condition that designer can tweak avar mappings at the end of design work freely, without needing to modify the masters.

Ty, I was not aware that was a goal. Then clearly design or normalized is correct. Edit: chat discussion supports the claim people want to handwrite so supporting only normalized seems less appealing.

Adam (here) and Simon (here) both assert normalized is unpleasent for the user so our options are down to:

1. design units
2. design and normalized with explicit indication of which

I'm currently not aware of any person or tool that particularly benefits from option 2. That would lead me to believe design units is the correct choice?

[behdad]

1. design units
2. design and normalized with explicit indication of which

I'm currently not aware of any person or tool that particularly benefits from option 2. That would lead me to believe design units is the correct choice?

+1.

A redundant way to allow .fea files to be self-contained is to have declaration of axis design min/default/max optionally.

[Lorp]

Maybe I am missing something, but are we essentially discussing how to use arbitrary surfaces as the boundaries in design spaces where features are triggered? In a simple 2D case we might want a diagonal line to decide things, and we’d perhaps implement it primitively as a staircase; and we could extend that to multiple dimensions. Alternatively we could use avar2 to control an additional axis specifically intended for triggering features, and thereby allow true diagonals and curves. This axis might indeed be tricky to set up in an avar2 table, but the feature code itself would be trivial.

[behdad]

Maybe I am missing something, but are we essentially discussing how to use arbitrary surfaces as the boundaries in design spaces where features are triggered?

Not really. We're discussing what space the numbers should be specified in.

In a simple 2D case we might want a diagonal line to decide things, and we’d perhaps implement it primitively as a staircase; and we could extend that trivially to multiple dimensions. Alternatively we could use avar2 to control an additional axis specifically intended for triggering features, and thereby allow true diagonals and curves. This axis might indeed be tricky to set up in an avar2 table, but the feature code itself would be trivial.

Correct.

[Lorp]

~Thanks… In such a case, which is kinda the general case, normalized is the only choice.~ Edit: Scratch that, it’s no problem for a feature-triggering axis to have its own designspace coordinates.

[skef]

In terms of axis positions, Adobe is generally pursuing a design in which the designer can use the units they think appropriate to the situation, picked with a suffix, and with design units available if the designspace document is available, but not otherwise. If there is no suffix then user coordinates are assumed. So if a given context requires normalized coordinates, that' will be supported in our grammar.

That doesn't resolve the rest of this issue, however.

Requiring normalized coordinates in this case makes it easy to implement but just puts the onus on the designer, raising the question of what advice to give them. So, is there viable straightforward advice, or do we need additional infrastructure to help with this (e.g. some sort of tie-breaking notation, if that's relevant).

[skef]

Incidentally, on the recurring "assembly language" metaphor: Even if almost all development switched entirely to higher-level tools, and it is far from obvious this will happen anytime soon, our situation would be more analogous to assembly language in the late 80s rather than assembly language of the past 20 years. It took many developer-millennia to reach the point where almost no programmer needs to look at or think about assembly, and that's just not how things work with font tools. If it were, we wouldn't be having this conversation about the fea syntax and higher-level tools would be generating tables directly or targeting ttx XML or whatever.

We expect that many font engineers will be doing at least some work directly in feature files, even if the bulk of (e.g.) pairwise kerning data is auto-generated. And even those that don't may have to look at those files sometimes, so support for generating files that aren't awful to read is also important.

[rsheeter]

@skef ty for the update, that's very helpful as we build out fea-rs.

Would it be possible to share the unit suffixes you are thinking of, even if they are tentative/subject to change? - I need variable fea nowish and I would like to have it built as close as possible to where you expect final variable fea syntax to land.

Other than units do you expect significant change from the variable fea sketch given in https://github.com/adobe-type-tools/afdko/pull/1350?

[behdad]

Currently with the proposed syntax, .fea would need input from the fvar table to be compiled, since we would need to know the axis indices. One way to resolve that, and the issue raised here, is to require axis specifications at the top of .fea file, which will determine the axis order as well as the design range of the axes to be used in the rest of the .fea file.

[skef]

@behdad Whether axis specifications are needed from the start of a feature file depends on whether, in a VF-first workflow, we're thinking that the designspace file is going away, which is not what we at Adobe are currently imagining, in no small part because that file does a number of things that would be awkward to replace. So: Do conditions move to the feature file? Probably, although perhaps not without continuing to have backwards support for them in the designspace file. Do design unit mappings move? Almost certainly not.

Instead there's a larger question here about the order in which tables are built. We're currently thinking that fvar (at least) is probably built first along with cff or glyf/loca. Those tables have never depended in any important way on feature file content. So, you start by getting your basic path data sorted out and then work up from there.

This does mean that some tables may need to be reworked during the build process. Maybe you know some stuff about what goes into hhea early on and only learn other stuff later.

[behdad]

Whether axis specifications are needed from the start of a feature file depends on whether, in a VF-first workflow, we're thinking that the designspace file is going away, which is not what we at Adobe are currently imagining, in no small part because that file does a number of things that would be awkward to replace. So: Do conditions move to the feature file? Probably, although perhaps not without continuing to have backwards support for them in the designspace file. Do design unit mappings move? Almost certainly not.

I was thinking that a snippet is generated from the .designspace file and injected into the .fea file. It already is the case that kerning and anchors need to be generated from UFO and injected into the final .fea file, so I see it fit in that model.

[rsheeter]

I would prefer the feature file just assume the compiler can translate units. Whether information came from a .designspace or .glyphs or is out of scope for fea, it just trusts there will be a compiler that knows about unit translations.

No need to duplicate the information into fea and inevitably have people hand write it such that it doesn't match other parts of source and no need to dictate how the feature compiler is informed of unit translations.

In a complete coincidence this would work very well for fontc: today we call fea-rs with (glyph order, feature file), tomorrow we call it with (glyph order, unit conversion information, feature file).

[skef]

@rsheeter

As far as your narrow question, my current notes have "u" for user units, which as I said would be the default if omitted. Design units would be "d" and normalized units would be "n". We also may have keywords "min", "max" and "def"/"default" as alternatives for -1n, 1n, and 0n respectively.

Whether this is the full list depends on something I've been meaning to follow up on, and I guess this is as good a place as any. @behdad can you expand on your comment here? I'm not sure I yet understand the difference between, e.g., normalized userspace and normalized designspace.

[behdad]

Whether this is the full list depends on something I've been meaning to follow up on, and I guess this is as good a place as any. @behdad can you expand on your comment here? I'm not sure I yet understand the difference between, e.g., normalized userspace and normalized designspace.

normalized userspace would be before avar mapping. That said, I retract my comment. I don't think that's a useful unit to have.

[skef]

In that case, the only additional unit set that I've considered is one that goes from 0 to 1 or perhaps 0 to 1000, which would be analogous to "n" units except that it takes the detail of where the default is out of the picture. I'm still on the fence as to whether those would be useful.

Anyway, I think "u", "d" and "n" cover all of the most common cases, and if we were to add something it wouldn't get in the way of what your tools do in the meantime. Making "u" optional should make things largely compatible with current ad-hoc practice.

[behdad]

I respect your decision as the owners of the format. But I like to reiterate why having user-space is a bad idea IMO, let alone being the default:

They are very error-prone. The variations in a .fea file are based on the master designs. If the designer later changes the avar mapping of their designspace, the features variations should NOT change. That's never the correct behavior.

I'm not convinced that user-space coordinates have any use in the .fea file. I'm open to be convinced otherwise.

[skef]

I only have user coordinates as the default in my notes because I was under the impression that current ad hoc VF-first support in feature files (in, e.g. fonttools) used user units, and that seemed to provide the best continuity. We can revisit that. Maybe there shouldn't be a default at all.

In terms of not providing them in the first place, maybe they're an attractive nuisance in feature files but if so people will learn that lesson by hanging themselves with the rope we give them. In thinking through these cases myself it seems like different contexts call for different units and I'm not convinced there aren't some contexts where they aren't appropriate.

[skef]

On @rsheeter 's broader question about scope, the answer is "yes" but I'm sorry, we're just not there yet on the details. I have a pile of notes from internal meetings that have tried to synthesize the higher-level-tool-oriented ideas from issue 1350, current practice, and feedback from internal users of the FDK. Turning this pile into a proper specification means that we need to implement it and make some use of it. Once we get to that point we hope to release something like a beta implementation and an RFC for the grammar changes and then work to integrate the feedback we receive.

I understand that Adobe started working on this much later than we ideally should have. I also understand that from the outside it's not evident that we're doing much of anything. Unfortunately, a large technical debt has stood in the way of doing anything VF-first using the FDK. Although Adobe is a large company the font tools group is a modest shop. We make many of our tools available as open source but Adobe also a foundry and the FDK is the lower level of our own practice. In updating the grammar to support VF-first workflows we want to ensure it can still be used by the same folks to build fonts somewhat like they have been.

Until recently the core of the FDK has been a cleaned-up version of C code from the 90s (or earlier) with custom, limited, and somewhat flakey container code. makeotfexe [edited] is a tool that takes a Type-1 font and turns it into a CFF and then builds its shaping tables. This is not a great foundation for VF-first. A couple years ago, as a contractor, I moved the parser from pccts/Antlr 1 to Antlr 4. Some time ago we got all of the FDK C code compiling together as a single C-style C++ unit, so that we could start making the further changes we needed. Then we got things turned around so that makeotf's replacement starts with an already-built CFF and used the (frankly better) code from tx to build the CFF. Shortly after that we added what was needed to do the same with a CFF2.

Literally today I pushed updates to this branch so that GPOS, GDEF, and "otl" (what deals with scripts, langauges, features, lookups) are rewritten in modernish C++ with proper object lifecycle. This finally puts us in a position to move forward with implementing the GPOS-side of the VF first workflow.

(This work currently lives at https://github.com/adobe-type-tools/afdko/tree/addfeatures for the curious.)

I don't currently know how much longer this will take. can say with great confidence that we're well, well past the half-way mark, for whatever that's worth. There are months to go still, but not years.