IR alternative: frontends drive backend directly

[madig]

While reading https://github.com/googlefonts/oxidize/blob/main/text/2022-11-14-why-ir.md#motivation, I had a random thought:

What if instead of an IR, you have a builder-pattern-like backend? A source frontend would load the source in whatever format, do post-processing like applying corner components and then call BackendBuilder::new().add_glyphs(...).set_opentype_fields(...) and so on. This would avoid having yet another format, even if just intermediate. But maybe it would be worse, I don't know!

[rsheeter]

In this model is the builder meant to be building the final binary form directly? Including doing things like figuring out optimal packing for layout?

[madig]

Yes. The low level stuff happens in the builder. The frontend turns the higher level source into the basic stuff representable by OpenType, and a builder API may be enough. Or not, I have not tried it.

[behdad]

Yes. The low level stuff happens in the builder.

Well, how would the builder work then. The IR is a way for the builder to work, no?

[madig]

In a weird way, the builder API would be the IR. You give it a hashmap of locations to a hashmap of kerning pair values and it generates the variable GPOS and GDEF tables for you.

The idea is that most source formats are very similar and differ more in the ✨ they enable in design applications than any fundamental way, so doing the magic processing in the frontend and pushing dumb buttons in the backend might be one way to do things.

[madig]

There's precedent in https://github.com/fonttools/fonttools/blob/main/Lib/fontTools/fontBuilder.py in a way, but maybe stuff can be made slightly higher level so the call order doesn't matter.

[behdad]

You give it a hashmap of locations to a hashmap of kerning pair values and it generates the variable GPOS and GDEF tables for you.

That's exactly how fontTools.otlLib works as well. But to do optimizations and other processing, you need an intermediate representation.

[rsheeter]

the builder API would be the IR

I agree :) Let's chase that thought.

We'd quite like incremental compilation. I suppose we're going to need the builder to write down it's state, restore it, and to be able to figure out what changed vs incoming sources and only do the subset of the work that is needed.

Next up, we'd quite like to not include the same shapes over and over again so lets insert an optimization pass for that.

I feel like we're ending up pretty much where https://github.com/googlefonts/oxidize/blob/main/text/2022-11-08-font-compiler-ir.md ends up?

[rsheeter]

Closing based on the "we ended up in the same place" argument. If I misunderstood and this is more different than I'm imagining please reopen for further discussion.

[belluzj]

An IR is also very nice for writing unit tests, especially if it comes with a DSL to write small bits of IR as test inputs/outputs

[rsheeter]

especially if it comes with a DSL to write small bits of IR as test inputs/outputs

At time of writing I expect we'll support a binary form and a textual form, tentatively bincode and toml respectively. Should be easy enough to write fragments in the textual form.