Support color fonts

[RazrFalcon]

Emojis can be stored in multiple ways, but we should at least support bitmap one.

[LaurenzV]

I will try to look into this. Although I'm not sure what I'm getting myself into. xD

[RazrFalcon]

Although I'm not sure what I'm getting myself into.

Very bad time 😄

I've mentioned main issues with text handling here recently: https://github.com/RazrFalcon/tiny-skia/issues/1#issuecomment-2009124094

Also, just a heads up, but "bitmap fonts" are a bit of a mess of a term. What we need is Emojis support, which in TrueType terms means:

• sbix (Apple, basically a PNG)
• bloc+bdat (legacy 1bit, parsed by ttf-parser, but not decoded)
• EBLC+EBDT (legacy 8bit, parsed by ttf-parser, but not decoded)
• CBLC+CBDT (Google, basically a PNG)
• COLR (kinda a mini SVG, v0 is supported, v1 is in progress, had to be rendered somehow)
• SVG (aka SVG, but technically a subset/superset, we need a custom parser for it)

As you can guess, the first step would be to write a TrueType glyph rasterization library. Well, unless you plan to stick with sbix/CBLC+CBDT.

Overall, I would say the step number 1 would be to untie the current implementation from operation strictly on path outlines. Meaning we have to do layout purely on glyph ids and then fetch/render/cache required glyphs.

I have a lot of ideas, but barely any time. For me, this is the next big resvg update. But it's a beefy one and I have no time. Therefore feel free to ask any questions.

PS: oh yeah, embedded SVG fonts + WOFF support is also kinda part of it. After all, SVG is mostly about text and nothing else.

[LaurenzV]

I looked a bit into it. I agree that in the long term, it would probably be best to have text just contain a number of glyphs as well as their position, and then just render them correspondingly. Eventually, I plan to implement text selection for svg2pdf, and for that it would make things a lot easier if I just have to deal with positioned glyphs, instead of having to copy paste basically the whole layout logic. But this would probably take a bit of timeto implement.

What do you think if we try this is a "stopgap" approach until we have something better: Right now, an OutlineCluster only contains a tiny skia path. Instead, we could change it so that the OutlineCluster contains a group with some children. These children would be:

• Path nodes for outlined glyphs
• Image nodes for raster image glyphs
• Group nodes for SVG glyphs

And then we basically just render the group as is. I'm not sure if I'm missing something. But I've managed to change the current implementation so that it works with Path nodes and it seems to work fine, so next I could look into whether it actually works with bitmap emojis. If you think that this approach is worth pursuing for now.

[LaurenzV]

Regarding SVG, when you refer to having to write a custom parser I presume that you are referring to the fact that some elements must be ignored according to the specification (e.g. text, stylesheets, etc.)? But I'm really not sure if it's worth it to write a custom parser for that... Maybe we can instead add options to usvg to ignore such elements? Not sure. But for now, it should be fine to just use the existing usvg implementation, no?

EDIT: Nevermind, I should've kept on reading... There is more stuff that we need to support.

[LaurenzV]

But the more I think about it, maybe it's indeed best to first rewrite the current code to contain positioned glyphs which are then subsequently turned into paths, and then worry about how to best deal with emojis... Will try to look into it.

[RazrFalcon]

If I recall correctly, the two main issues with TrueType SVG are: CSS 3/4 features which we do not support (namely computed values) and an ability to reference other glyph (aka <use xlink:href="#glyph14"/>, which are obviously impossible right now). Meaning we have to improve CSS support and add a custom xlink resolver (like we do with images) first.

As for the "next step" - it doesn't really matter to me. The reason I would suggest starting with separating layout and outlines is because outlines can be freely scaled, but bitmap fonts can't. Meaning we have to either get the bigger bitmap from the font and downscale it or somehow dynamically choose which bitmap size to render. PS: yes, fonts with the sbix table (aka Apple Emojis) contain multiple bitmap sizes for each glyph.

[LaurenzV]

I'll keep you posted! I have some ideas I will try out.

[LaurenzV]

So we have glyph layouts now, so once I find the time I will try to work on this (for now just bitmap emojis) in the next few days/weeks.

But my question is: Do you know any fonts that I can use to test all of the legacy bitmap formats? I can use Apple Color Emoji to test sbix, and I think older version of Noto Color Emoji use CBLC + CBDT. But any idea for where I can fonts for the older ones?

Basically, I'd like to have one font for each of the formats that ttf-parser can return. Any idea where to search for those?

[LaurenzV]

Or would you accept a PR that implements this just for PNG bitmaps for now?

[RazrFalcon]

Well, that the trick - there are none. The only sbix font I know is Apple Color Emoji. Some versions of Noto Color Emoji do use CBLC + CBDT. And that's about it. Here are some EBDT+EBLC fonts: https://github.com/RazrFalcon/ttf-parser/pull/121

Let's stick with PNG for now.

[LaurenzV]

So I'm currently stuck with the following:

Let's say we have the following SVG:

<svg id="svg1" viewBox="0 0 200 200" xmlns="http://www.w3.org/2000/svg"
     font-family="Apple Color Emoji" font-size="32">
    <title>Emojis</title>

    <path id="crosshair" d="M 20 100 L 180 100 M 100 20 L 100 180"
          stroke="gray" stroke-width="0.5"/>

    <text id="text1" x="100" y="100">😀😁😂🤣</text>

    <!-- image frame -->
    <rect id="frame" x="1" y="1" width="198" height="198" fill="none" stroke="black"/>
</svg>

my current implementation renders it like this:

Basically, I'm setting "the cursor" at the position (100, 100), and then I'm aligning this with the bottom of the picture and I render the image at font size. However, in Chrome it looks like this:

So for some reason the emoji reaches a bit below the vertical axis. But I'm not really sure where this shift is coming from and how to calculate it... Any ideas?

[RazrFalcon]

ttf_parser::RasterGlyphImage has x/y positions. Probably them.

[LaurenzV]

Nope, in this case all glyph raster images have x and y set to zero...

[LaurenzV]

My best guess would be that it is somehow aligned to the descender of the font? But I don't see this being described anywhere...

[RazrFalcon]

Hmm, Apple emojis are rendered "below" the baseline in Firefox and Safari as well, so we definitely have to account for something, but I have no idea for what.

rustybuzz seems to be producing the correct output as well:

> cargo run --example shape -- '/System/Library/Fonts/Apple Color Emoji.ttc' '😀'
u1F600=0+800
> hb-shape '/System/Library/Fonts/Apple Color Emoji.ttc' '😀'
[u1F600=0+800]

[LaurenzV]

Yeah I really have no idea... The width seems to be just fine, there really is just some shift on the y-axis for some reason.

[LaurenzV]

Would it be okay for you if for now I just try to find a magic value that at least works with Apple Color Emoji and Noto Color Emoji? I will also add some tests to make sure it also works for different font sizes. I'm not sure what else I could do.

[RazrFalcon]

Not ideal, but good enough.

[LaurenzV]

Welp... using a magic value that works for Apple Color Emoji does not work for Noto Color Emoji. 🥲 So yeah... I guess we're stuck for now.

[RazrFalcon]

I was expecting you hardcode an offset per font family. I might look into it next weekend.

[LaurenzV]

Ah... Yeah I guess this could be a last resort option.

[RazrFalcon]

Can you also publish your changes so I could test them locally.

[LaurenzV]

Yes, sorry, I'll try to clean it up and upload it within the next few days.

[LaurenzV]

See #735.

[yisibl]

Also, just a heads up, but "bitmap fonts" are a bit of a mess of a term.

I suggest changing the title of this issue to Support for Color fonts (including Emoji), this site does a good job of explaining what color fonts are.

Because color fonts are not just emoji, it is a broad font technology, emoji is just one of its usage scenarios, other scenarios including COLR v0 are also widely used in font icons.

Color font is also a term used in CSS: https://drafts.csswg.org/css-fonts-4/#color-font-support

<color-font-tech>= [color-COLRv0 | color-COLRv1 | color-SVG | color-sbix | color-CBDT ]

[yisibl]

SVG fonts are pretty much an abandoned format in browsers (Chrome has never supported it and doesn't intend to), and COLRv1 is an alternative technology. I would suggest that consideration could be given to lowering the priority of this format implementation in resvg.

[Offirmo]

Hi mates, amazing that you're working on it! How about focusing at having NotoColorEmoji work? (Open Font license)

[LaurenzV]

SVG fonts are pretty much an abandoned format in browsers (Chrome has never supported it and doesn't intend to)

Are you sure? At least on my Computer Chrome can handle emojis from Twitter Color Emoji, which as far as I know is an SVG font... But maybe it depends on the OS. But anyway, at least in my PR it wasn't much effort to support that, so it seems like the more "obscure" features of SVG fonts that deviate from standard SVG might not be used as commonly. So we might as well just add it.

How about focusing at having NotoColorEmoji work? (Open Font license)

AFAIK the new vesions for Noto Color Emoji use COLRv1 which currently isn't supported yet upstream. I don't know if there are any particular difficulties in implementing this or whether someone just needs to find some time to finish it, maybe @RazrFalcon can say more about that.

But the older versions of Noto Color Emoji use bitmaps, which should work with that PR.

[yisibl]

Are you sure? At least on my Computer Chrome can handle emojis from Twitter Color Emoji, which as far as I know is an SVG font... But maybe it depends on the OS.

See MDN SVG fonts:

The functionality was removed from Chrome 38 (and Opera 25) and Firefox postponed its implementation indefinitely to concentrate on WOFF.

You can try ttx -l some_font.ttf to see what tables are available for the Twitter Color Emoji font.

[LaurenzV]

Oh. This is talking about the font element in SVG. Yes, this is deprecated, but I don't think there are any plans to implement this, anyway. I was talking about the SVG table in OpenType fonts, which use SVGs as glyph descriptions. And I think those are supported by web browsers.

[RazrFalcon]

@yisibl The SVG Font element and TrueType's svg table are two different things. No one supports the first one and it was deprecated in SVG 2. But TrueType SVG fonts are pretty common, sadly.

COLRv0 will be supported soon, thanks to @LaurenzV. COLRv1 is in progress by me https://github.com/RazrFalcon/ttf-parser/pull/133 (for a while now...).

[yisibl]

Sorry, I misremembered. Please ignore the MDN link and description.

It's true that Chrome doesn't implement OT-SVG(See https://issues.chromium.org/issues/40336440), though, and it doesn't look like there are any plans to do so at the moment, since COLRv1 replaces OT-SVG with some extent.

I don't have much of a problem with implementing OT-SVG in resvg, if it's not complicated and doesn't have a significant performance disadvantage, otherwise, I don't see much point in it.

OT-SVG Test Page: https://pixelambacht.nl/lapislegit/

[RazrFalcon]

Fixed by https://github.com/RazrFalcon/resvg/pull/735