googlefonts / fontc

Where in we pursue oxidizing (context: https://github.com/googlefonts/oxidize) fontmake.
Apache License 2.0
85 stars 14 forks source link

master values should be ot-rounded *before* computing deltas in order to match fontmake #1043

Closed anthrotype closed 3 weeks ago

anthrotype commented 1 month ago

take https://github.com/SorkinType/Gelasio/tree/main/sources/Gelasio.glyphspackage and run through ttx_diff.py

You'll notice MVAR has one delta related to "stro" tag (strikeout offset or position) which differs by 1 unit when compared with fontmake's

--- /Users/clupo/oss/fontc/build/default/fontc.MVAR.ttx 2024-10-16 12:21:15
+++ /Users/clupo/oss/fontc/build/default/fontmake.MVAR.ttx  2024-10-16 12:21:15
@@ -22,7 +22,7 @@
         <NumShorts value="0"/>
         <!-- VarRegionCount=1 -->
         <VarRegionIndex index="0" value="0"/>
-        <Item index="0" value="[5]"/>
+        <Item index="0" value="[4]"/>
         <Item index="1" value="[8]"/>
       </VarData>
     </VarStore>
\ No newline at end of file

The reason for this is that fontmake builds TTFs for each master and then uses varLib to build MVAR among other tables, and the global metrics values as stored in the master TTFs are already rounded to integers (using otRound) so the input to varLib are ot-rounded integers, not floats.

Whereas fontc keeps the global metrics value as floats (the sources allow to store these as floats even though basically all these metrics are going to end up being rounded to int16 or uint16 when serialised to OpenType) and then it generates the deltas from these float metrics, and only at the end it ot-rounds the deltas themselves.

In this particular case, Gelasio doesn't actually contain floats for strikeout offset; in fact it doesn't specify any strikeout offset, so it is the GlobalMetrics::populate_defaults() method that is introducing the floats, here:

https://github.com/googlefonts/fontc/blob/37143a0c39535aaf8eaa42be53fa028bd6746cc3/fontir/src/ir.rs#L698-L701

Now, in order to match fontmake we could either

1) change ir::GlobalMetrics so that it stores integer values instead of f64, and do an ot_round() for all incoming values (from any frontends, not just glyphs or ufo) as soon as they get set, eg

- HashMap<GlobalMetric, HashMap<NormalizedLocation, OrderedFloat<f64>>>
+ HashMap<GlobalMetric, HashMap<NormalizedLocation, i32>>

1) keep f64 in GlobalMetrics and only perform to ot_round() only for the global metrics values that come from glyphs and ufo sources (not any frontends), be them user-defined or those introduced by the populate_defaults() method; this way we would match fontmake for .glyphs and DS+ufo sources, but not also force other frontends to round global metrics sources before deltas get computed.

anthrotype commented 1 month ago

Ok so, I tried to see if I could instead make so that ufo2ft and fonttools did not round the global metrics master values before computing the deltas, but nothing actually changed, I was still seeing the same diff.

it turns out the issue was not that fontc doesn't round before computing deltas... but rather that fonttools does the rounding within the delta computation itself, see https://github.com/fonttools/fonttools/issues/2213

This allows for rounding error from one set of deltas to be accounted for when computing subsequent deltas, reducing the total error from accumulating unbounded as number of masters grows, to always being limited to 0.5.

So this is the same as https://github.com/googlefonts/fontc/issues/235

If I apply the following patch, then ttx_diff.py for Gelasio.glyphspackage yields identical outputs for MVAR.

diff --git a/fontir/src/variations.rs b/fontir/src/variations.rs
index ff43b3a5..49b94046 100644
--- a/fontir/src/variations.rs
+++ b/fontir/src/variations.rs
@@ -21,6 +21,24 @@ use write_fonts::{

 use crate::error::VariationModelError;

+trait BankerRound<U, T = Self> {
+    fn banker_round(self) -> U;
+}
+
+impl BankerRound<f64> for f64 {
+    #[inline]
+    fn banker_round(self) -> f64 {
+        self.round_ties_even()
+    }
+}
+
+impl BankerRound<kurbo::Vec2> for kurbo::Vec2 {
+    #[inline]
+    fn banker_round(self) -> kurbo::Vec2 {
+        kurbo::Vec2::new(self.x.round_ties_even(), self.y.round_ties_even())
+    }
+}
+
 const ZERO: OrderedFloat<f32> = OrderedFloat(0.0);
 const ONE: OrderedFloat<f32> = OrderedFloat(1.0);

@@ -171,7 +189,7 @@ impl VariationModel {
     ) -> Result<Vec<(VariationRegion, Vec<V>)>, DeltaError>
     where
         P: Copy + Default + Sub<P, Output = V>,
-        V: Copy + Mul<f64, Output = V> + Sub<V, Output = V>,
+        V: Copy + Mul<f64, Output = V> + Sub<V, Output = V> + BankerRound<V>,
     {
         if point_seqs.is_empty() {
             return Ok(Vec::new());
@@ -234,7 +252,8 @@ impl VariationModel {
                         })
                         .fold(initial_vector, |acc, (other, other_weight)| {
                             acc - *other * other_weight.into()
-                        }),
+                        })
+                        .banker_round(),
                 );
             }
             model_idx_to_result_idx.insert(model_idx, result.len());
anthrotype commented 4 weeks ago

Fixed by #1070

anthrotype commented 4 weeks ago

PR #1070 only really fixed #235 as we are now correctly rounding deltas as they get computed. However that hasn't technically also fixed this issue, because fontc still passes float master values to VariationModel for some stuff for which fontmake instead passes already-rounded integers. It's not just global font metrics (e.g. stroke offset in Gelasio MVAR). I am now noticing other off-by-one diffs elsewhere which I think are related and haven't been fixed by #1070 (some have surfaced after that).

For example, fontc_crater reports that after #1070 got merged, the GDEF table diff in https://github.com/chankfonts/Teachers-fonts got -26.929% worse off.

Image

I tracked this down to the fact that there's a composite glyph "ordfeminine" which contains a transformed component "a" scaled by 0.75. The composite inherits anchors from the component via propagate_anchors.rs, and the anchors (x,y) positions are passed on as floats by fontc to its VariationModel, whereas fontmake (ufo2ft's markFeatureWriter) is rounding them off (via otRound) before computing the deltas.

I subsetted the source file to only contain the minimal reproducer, three glyphs "a", "ordfeminine" (and "acutecomb" to trigger mark feature), here's the test file: Teachers-Italic.glyphs.zip

This is how the propagated "top" base anchor appear in fontc' build/anchor_ir/ordfeminine.yml (note how the values are un-rounded floats):

glyph_name: ordfeminine
anchors:
- kind: !Base top
  positions:
    ? - - wght
        - 0.0
    : x: 282.75
      y: 691.25
    ? - - wght
        - 1.0
    : x: 291.75
      y: 689.5
    ? - - wght
        - 0.75
    : x: 289.5
      y: 689.5

... whereas this is the ufo2ft-generated build/default/debug.fea where you can see the same anchor's master values rounded off (this become the input to feaLib to build the variable GPOS/GDEF):

feature mark {
    lookup mark2base {
        pos base ordfeminine
            <anchor (wght=400:283 wght=700:290 wght=800:292) (wght=400:691 wght=700:690 wght=800:690)> mark @MC_top;
    } mark2base;
} mark;

The rounding (otRound) appears to be performed in ufo2ft markFeatureWriter (not in glypshLib propagate_anchors.py) when the anchors are serialized to FEA ast: https://github.com/googlefonts/ufo2ft/blob/d15ef8f6f235804733c6e2ea5f01f39aafb92f45/Lib/ufo2ft/featureWriters/markFeatureWriter.py#L37-L38

If I apply the following patch to fontbe/src/features/marks.rs so that the anchors get rounded before being sent to the VariationModel for computing deltas, then I get back to zero diff in Teachers-Italic's GPOS/GDEF (not just the subsetted test font but the whole source as well):

diff --git a/fontbe/src/features/marks.rs b/fontbe/src/features/marks.rs
index cc5d4bd3..b49ae3d4 100644
--- a/fontbe/src/features/marks.rs
+++ b/fontbe/src/features/marks.rs
@@ -600,9 +600,11 @@ fn resolve_anchor_once(
         .positions
         .iter()
         .map(|(loc, pt)| {
+            let x: i16 = pt.x.ot_round();
+            let y: i16 = pt.y.ot_round();
             (
-                (loc.clone(), OrderedFloat::from(pt.x as f32)),
-                (loc.clone(), OrderedFloat::from(pt.y as f32)),
+                (loc.clone(), OrderedFloat::from(x as f32)),
+                (loc.clone(), OrderedFloat::from(y as f32)),
             )
         })
         .unzip();
anthrotype commented 4 weeks ago

oh, it looks like also master glyph coordinates get otRounded in fonttools before being sent off to VariationModel, see:

https://github.com/fonttools/fonttools/blob/7a0062a718e33cf4900e1a5b005c9b649ea6d964/Lib/fontTools/varLib/__init__.py#L319-L322 https://github.com/fonttools/fonttools/blob/7a0062a718e33cf4900e1a5b005c9b649ea6d964/Lib/fontTools/ttLib/tables/_g_l_y_f.py#L389

Whereas fontc is not doing the same as far as I can tell: https://github.com/googlefonts/fontc/blob/fd1fc6ab8e3a724d24b4672364a87129e85c3250/fontbe/src/glyphs.rs#L226

That may account for additional diffs in gvar..

EDIT: I was wrong, see below https://github.com/googlefonts/fontc/issues/1043#issuecomment-2446509670

anthrotype commented 4 weeks ago

Same thing for HVAR: fontc is not rounding the advance widths before passing them on to VariationModel to compute the deltas, whereas fonttools always works with rounded values (the master TTFs' hmtx tables). This probably surfaces less often because, although source formats allow witdhs to be either integers or floats, font developers don't actually set the glyph widths to floats. Maybe only when a master was in turn an interpolated instance re-inserted as a master.

anthrotype commented 4 weeks ago

it looks like also master glyph coordinates get otRounded in fonttools ... Whereas fontc is not doing the same as far as I can tell

self-correction. fontc is already rounding the glyph coordinates before computing deltas, because it first converts the IR's BezPaths into interpolatable write_fonts::SimpleGlyphs (which internally use i16 for x and y coordinates, otRounded) and then it extracts lists of kurbo::Points, which internally use f64, to send off to VariationModel for delta computation. So we're good as far as rounding for the glyph outlines and the respective variations.

anthrotype commented 3 weeks ago

for reference, this is where master values for anchor positions get otRounded in ufo2ft featureWriters, when the VariableScalar is constructed:

https://github.com/googlefonts/ufo2ft/blob/d15ef8f6f235804733c6e2ea5f01f39aafb92f45/Lib/ufo2ft/featureWriters/baseFeatureWriter.py#L431-L432

kerning values are "quantized" to 1 by default (which simply means otRound'ed) here: https://github.com/googlefonts/ufo2ft/blob/d15ef8f6f235804733c6e2ea5f01f39aafb92f45/Lib/ufo2ft/featureWriters/kernFeatureWriter.py#L515 https://github.com/googlefonts/ufo2ft/blob/d15ef8f6f235804733c6e2ea5f01f39aafb92f45/Lib/ufo2ft/util.py#L573