search

unified-font-object / ufo-spec

The official Unified Font Object specification source files.
http://unifiedfontobject.org
170 stars 30 forks source link

TrueType instructions in UFO #93

Closed moyogo closed 3 years ago

moyogo commented 5 years ago

Could we add a lib keys to hold fontTools TTX instructions?

This would be like #42 but for TrueType instructions. It would be useful to have a standard way of storing bytecode TrueType instructions in UFOs, especially in the case of extracting UFOs from TTFs and being able to compile that back into TTFs. Or this could be useful when processing UFOs in a standard way with tools that can compile TrueType instructions.

It could use the following structure: In the glyph’s lib:

In a font’s lib:

moyogo commented 4 years ago

The component.lib key for useMyMetrics, overlap and round should be public.truetype instead of public.htic since those are TrueType component flags.

htic seems to:

It doesn’t seem to do anything for overlap.

@jenskutilek does this merge request https://gitlab.com/sev/htic/-/merge_requests/6 that has been merged in htic resolve the DELTA[CP][123] roundtripping? Regarding the PUSH instructions, it would be safer if these were preserved. I think this is a requirement when JMPR (Jump), JROT (Jump Relative On True), JROF (Jump Relative On False) are used.

jenskutilek commented 4 years ago

Just a small update. From what I heard from Severin, the creator of htic, he sees the status of htic as bugfix-only, he doesn't plan to add any new features. E.g. jump target label support or other nice features would apparently require a complete rewrite.

I tried to add verbatim roundtripping for PUSHB, PUSHW, NPUSHB, and NPUSHW instructions and was not entirely successful (push argument size is preserved, but the first push instruction always gets moved to the beginning of the program).

Forcing htic to not optimize the code is kind of like forcing it not to do what it does best. I came to think that maybe it would be better to keep htic as a separate optimization step, and use a straight assembly to bytecode compilation like fontTools already provides. But maybe with a streamlined syntax that looks a bit more like htic? (Don't use comments for counting the push arguments, dropping empty brackets). Maybe a simpler compiler makes implementing optional point labels instead of indices feasible? That would make the hinting code more robust in case the outlines change slightly or paths get reversed.

BTW my current bytecode to htic decompilation code is in my fork of extractor. I could have a go at writing the compilation step as well.

jenskutilek commented 4 years ago

Would additional hashes need to be stored to check the validity of the assembly code? In addition to the TrueType outline, they may also be invalidated by

For the PS outline, the same HashPointPen could be used as for the TrueType outline. The hash function for the source code could only be defined when that source format is standardized.

If a hash is present, and doesn't match the referenced item, the hinting must be discarded. Absence of the hashes would mean that the hinting is to be considered valid. If an error occurs during compilation (e.g. missing point identifier), the hinting for the glyph must be discarded.

moyogo commented 3 years ago

Just a small update. From what I heard from Severin, the creator of htic, he sees the status of htic as bugfix-only, he doesn't plan to add any new features. E.g. jump target label support or other nice features would apparently require a complete rewrite.

This, among other things like having an extra dependency, is a deal breaker. I really think we should go for something that already works and is actively maintained. We can use htic on top of it, when that makes sense, in UFO editors and consumers.

Would additional hashes need to be stored to check the validity of the assembly code? In addition to the TrueType outline, they may also be invalidated by

  • changes in a PostScript outline from which the TrueType outline has been derived
  • changes in TrueType hinting source code from which the assembly has been compiled, and which may be standardized some day

The TrueType outline hash must be checked before compiling the TTF. If the PostScript outlines change or their TrueType conversion changes, if TrueType instructions hash is checked and the instructions are discared if there’s no match. The tools that write or change the TrueType hinting source code should generate the hash every time.

Maybe the lib["public.truetype.instructions"]["instructionList]" can be None or empty so tools that store hinting sources in a better format can generate it at build time, while still using the lib["public.truetype.instructions"]["id]" hash to check the TrueType outlines match.

jenskutilek commented 3 years ago

Related PR for calculating the hashes: https://github.com/fonttools/fonttools/pull/2005

benkiel commented 3 years ago

Agree that htic looks great, but is unnecessarily blocking this

moyogo commented 3 years ago

Actually we still need to address the composite glyph component flags (overlap, round, useMyMetrics) mentioned in https://github.com/unified-font-object/ufo-spec/issues/93#issuecomment-650106369 or https://github.com/unified-font-object/ufo-spec/issues/93#issuecomment-650253676.

From the discussion here is what it could look like:

<key>public.objectsLib</key>
<dict>
  <key>bANWxQTe5a</key>
  <dict>public.truetype.flags</dict>
    <key>useMyMetrics</key>
    <true/>
    <key>overlap</key>
    <false/>
    <key>round</key>
    <true/>
  </dict>
</dict>

Technically the useMyMetrics flag should only be used for one component. The OT spec doesn’t explicitly restrict it to one component but it is understood that only the sidebearing of only one component can be used.

The overlap flag must be only set for the first component, this is explicit in the OT spec. This is more specific to the glyph than to the component. There is also a OVERLAP_SIMPLE flag for contour glyphs, it also applies explicitly only to the first flag byte of the glyph. We should probably have a single flag in UFO that can be use for the composite glyph component OVERLAP_COMPOUND flag or the simple contour glyph OVERLAP_SIMPLE. Should we have a glif.lib["public.truetype.overlap"] key instead of a component.lib key through objectsLib?

moyogo commented 3 years ago

I'll open a separate issue, since what remain is about TrueType flags instead of TrueType instructions.

xorgy commented 3 years ago

@moyogo Is there a tracking issue or venue for composite glyph component flags? If there's not something wrong with it, I'd like access to USE_MY_METRICS in the glif, basically just how @jenskutilek suggested. I like the fallback behaviour but I can imagine ways that this would confuse people if they weren't aware of it.

moyogo commented 3 years ago

@xorgy The component flags issue was #162 and the pull request #170 has been merged.

See https://unifiedfontobject.org/versions/ufo3/glyphs/glif/#publictruetypeoverlap and https://unifiedfontobject.org/versions/ufo3/glyphs/glif/#standardized-object-lib-keys.