Port to 3.2.0

[LaurenzV]

Status	Commit message	HB Link	Comments
⚪️	[sanitize] add a maximum recursion depth for sanitize.	Link
⚪️	[subset] Only sanitize recursion depth in COLR.	Link
⚪️	[subset] reuse colrv1 max nesting depth constant for sanitize.	Link
⚪️	[colr] Simplify sanitize	Link
⚪️	[sanitize] In lookup sanitize don't try to access subtable 0 if it isn't there.	Link
⚪️	[array] Add default methods again	Link
⚪️	[map] Allow storing classes in the hashmap	Link
⚪️	[test] Silence unused-variable erros in test	Link
⚪️	[test] Fix nullptr_t scope issue	Link
⚪️	[array] Add constructor taking std::nullptr_t	Link
⚪️	[array] Mark constructors constexpr	Link
⚪️	[subset] inputSequence could be empty	Link
⚪️	[subset] keep features that have FeatureParams and the tag is "size"	Link
⚪️	Merge pull request #3301 from harfbuzz/hashmap-classes	Link
⚪️	[array] Fix a couple constexpr warnings	Link
⚪️	[subset] avoid writing out duplicate extra glyph names in post table	Link
⚪️	[meson] Require 0.55.0	Link
⚪️	[ci] Split sanitizers job	Link
⚪️	[ci] windows-2016 image is going away in a bit	Link
⚪️	[subset] layout_features filtering fix	Link
⚪️	[util] Undo treat as single-paragraph text provided on cmdline	Link
⚪️	[util] Add --single-par	Link
⚪️	3.1.2	Link
⚪️	[subset] Make cmap4 packing more optimal.	Link
⚪️	[subset] Don't pad glyphs when using long loca.	Link
⚪️	[subset] further optimize cmap4 packing.	Link
⚪️	[iter] add hb_concat (a	Link
⚪️	[subset] Improve sharing of Ligature subtables.	Link
⚪️	[subset] add all_links () to object_t.	Link
⚪️	[iter] fix forward implementation in hb_concat().	Link
⚪️	[iter] add specialized implementation for hb_concat forward when iterators are not random access.	Link
⚪️	Revert "Remove direct link to dwrite from meson scripts"	Link
⚪️	Revert "Revert "[hb-directwrite] Don't load dwrit.dll dynamically""	Link
⚪️	[directwrite] Set unsafe to break flag	Link
🟢	[tests] Add tests for platform shapers	Link
⚪️	[subset] Fix warning about uninitialized use of end_cp.	Link
⚪️	[subset] Actually fix end_cp unitialized warning.	Link
⚪️	[subset] MATH: don't serialize coverage table when iterator is empty	Link
⚪️	[CONFIG] Add HB_CONFIG_OVERRIDE_H to simplify config overrides	Link
⚪️	[buffer] Merge and rename clear_glyph_flags()	Link
🟢	[buffer] Rename find_min_cluster	Link
⚪️	[subset] COLR : only include glyphs after COLR closure	Link
⚪️	[repacker] create repacker output buffer after final length is known.	Link
⚪️	[repacker] Move all overflowing roots to a new space simultaneously.	Link
⚪️	[repacker] add a maximum number of roots that can be moved in one iteration.	Link
⚪️	[TINY] Don't disable VORG table in any config profile	Link
🟢	[CONFIG] Add HB_NO_VERTICAL	Link
🟢	[CONFIG] Fix HB_NO_VERTICAL build without HB_TINY	Link
🟢	[GPOS] Disable vertical advance if HB_NO_VERTICAL	Link
🟢	[ft] Disable vertical funcs if HB_NO_VERTICAL	Link
🟢	Update language system tag registry to OT 1.9	Link	some of those changes already existed, probably because a more recent version was already downloaded when running this
⚪️	[repacker] Improve vertex priority packing.	Link
⚪️	[repacker] add test case for using max priority to resolve an overflow.	Link
⚪️	[subset] (Chain)ContextSubst glyph_closure fix	Link
⚪️	[repacker] Clear distance and position cache when assigning a new space.	Link
🟢	[buffer] Inline revers_range() / reverse()	Link
🟢	[buffer] Add ::reverse_groups()	Link
🟢	[buffer] Add foreach_group()	Link
🟢	[ot-layout] Port foreach_grapheme to buffer foreach_group	Link
🟢	[buffer] Add optional merge_clusters arg to reverse_groups()	Link
🟢	[buffer] Fix reverse_group() to reverse() at the end	Link
🟢	[morx] Reverse graphemese	Link
⚪️	[subset] reset the visited lookup count before each closure iteration.	Link
⚪️	[subset] Don't double count visit lookups in closure_lookups	Link
⚪️	3.2.0	Link

[RazrFalcon]

as far as I can tell we don't support coretext/directwrite/uniscribe, so this can be skipped?

Yep.

[buffer] Merge and rename clear_glyph_flags()

Hmm... we don't have unsafe_to_break_all, but we do have unsafe_to_break. And the change in hb-ot-layout-gsub-table.hh already does what we do in src/ot/substitute.rs AlternateSet::apply. So just ignore.

we don't provide such granular access to configuring rustybuzz right?

Yes, everything is enabled by default. And all deprecated code is removed already, just to clarify.

[buffer] Inline revers_range() / reverse()

I don't think it affects us at all. We don't really use #[inline] in rustybuzz and rely on the complier to figure it out. That change is just C++ shenanigans.

[RazrFalcon]

@LaurenzV We don't have --single-par flag. I think we should just ignore it in gen-shaping-tests.py.

[LaurenzV]

Cool!

[LaurenzV]

@LaurenzV We don't have --single-par flag. I think we should just ignore it in gen-shaping-tests.py.

Okay, I will change that.

[LaurenzV]

Removing the flag seems to impact lookupflag tests:

../../harfbuzz/builddir/util/hb-shape --features="test" --no-clusters --no-glyph-names --no-positions --font-funcs=ot ../../harfbuzz/test/shape/data/aots/tests/../fonts/lookupflag_ignore_attach_f1.otf --unicodes=U+000A,U+000B,U+0015,U+000D,U+0016,U+0017,U+001D,U+001A,U+000A

[15|21|22|23|29]

 ../../harfbuzz/builddir/util/hb-shape --features="test" --no-clusters --single-par --no-glyph-names --no-positions -
-font-funcs=ot ../../harfbuzz/test/shape/data/aots/tests/../fonts/lookupflag_ignore_attach_f1.otf --unicodes=U+000A,U+000B,U+0015,U+000D,U+0016,U+0017,U+001D,U+001A,U+000A

[10|15|21|22|23|29|10]

The second one, so with single-par, is what rustybuzz produces.

[LaurenzV]

Hmm but then it doesn't make sense that it worked before. Will look into it more.

[RazrFalcon]

We shouldn't remove this flag when calling hb-view, but when generating Rust test. So this difference is expected.

To my understanding, this flag reads the string until the first new-line. So we should implementing this logic in our tests, preserving the flag.

[LaurenzV]

Ah oops, it's introduced here: https://github.com/harfbuzz/harfbuzz/commit/720ab0883

I missed it. Shouldn't be too hard to add.

[LaurenzV]

Alright so I think what it does is if this flag is not enabled the string will be split by newlines and then each line gets processed separately, while in the other case the newline is treated as part of the input.

For example:

builddir/util/hb-shape /System/Library/Fonts/Supplemental/AppleGothic.ttf "$(printf 'harf\nbuzz')"
[gid80=0+522|gid73=1+565|gid90=2+352|gid78=3+340]
[gid74=0+562|gid93=1+555|gid98=2+526|gid98=3+526]

builddir/util/hb-shape --single-par /System/Library/Fonts/Supplemental/AppleGothic.ttf "$(printf 'harf\nbuzz')"
[gid80=0+522|gid73=1+565|gid90=2+352|gid78=3+340|gid0=4+1000|gid74=5+562|gid93=6+555|gid98=7+526|gid98=8+526]

I tested it manually and it works with my implementation. I hope it's okay the way I wrote it.

[LaurenzV]

I'll also change it in the shape method in the tests.

[LaurenzV]

I manually verified that both, with and without --single-par the tests pass.

[RazrFalcon]

I'm fine having it in examples/shape, but it feels rudimentary in tests. It doesn't seems like any of the tests, even in master, actually rely on it. And we run strip() on the string anyway.

Let's remove --single-par from tests args parser. And strip it during tests generation. It should work just fine I think.

[LaurenzV]

Okay! By the way in such cases do you prefer if I revert commits and force push, or should I rather commit undo's manually?

[RazrFalcon]

Just a normal commit would be fine.

[LaurenzV]

Done.

[RazrFalcon]

All good. Thanks!