Text rendering

[RazrFalcon]

Text rendering is not supported and not planned. This is an absurdly complex task and the Rust ecosystem doesn't provide basically any libraries to implement this.

We need:

• [x] Font parser: ttf-parser.
• [x] Text shaper: rustybuzz or all-sorts.
• [ ] Font database with a font fallback mechanism.
• [ ] High-quality glyph rasterization library. Like ab_glyph_rasterizer, but with FreeType level of quality.
• [ ] Code in tiny-skia that ties it all together.

[yisar]

I think woff2 is better than ttf.

[alerque]

@yisar WOFF2 is a file format organization and compression scheme to optimize transmission of fonts over the internet for embedding in websites. The font payload you get after transfer and decompression in the end is the same. These days whether packaged as TTF, OTF, or WOFF2 what you are getting in terms of shapes and font functionality is identical. And hence you need a TTF parser to handle WOFF2 ;-) And WOFF2 doesn't make much sense for local font rendering, it's more CPU time to optimize away something that isn't even a bottleneck when working with local font files.

[Brooooooklyn]

Maybe https://github.com/servo/font-kit is also suitable for tiny-skia, I've used it 3 years ago: https://github.com/rust-canvas/rust-canvas/blob/master/src/canvas/context_2d.rs#L226

[RazrFalcon]

@Brooooooklyn font-kit is a wrapper over system libraries, which is no go. I want a pure Rust solution.

[kettle11]

Take a look at ‘fontdue’ by @mooman219

It may meet your requirements: https://github.com/mooman219/fontdue

[RazrFalcon]

@kettle11 fontdue is just a rasterizer right now. We need a lot more.

[mooman219]

@kettle11 this is kinda a small rundown of my opinion of the situation

It may meet your requirements: https://github.com/mooman219/fontdue

Fontdue is a no-std raster and layout tool for non-shaped text over a small subset of font features found in glyphbrush/ab_glyph/rusttype. This feature subset we all support isn't freetype grade for reasons mentioned below. In terms of runtime performance, it's much faster than glyphbrush/ab_glyph/rusttype, all of which use the raster from font-rs. It does exact coverage anti aliasing just like font-rs, so the quality is the same.

This is an absurdly complex task and the Rust ecosystem doesn't provide basically any libraries to implement this.

This is fairly accurate. The problem is that overall, the crates mentioned above (no-std crates with nice APIs) in rust are very limited in their scope over the domains they cover. Freetype font engine grade text includes a whole slew of hard problems. A non exhaustive list of features we're generally missing include:

• Gzip decompression for woff if you really care about that
• RGB Subpixel anti aliasing/blending/positioning
• Font config (including fallback)
• Stem darkening
• Shaping to any degree
• Color emoji/color in general
• Variation axis transformations
• BiDi
• CJK locale behavior

The big one here is shaping, which in pure rust is immature and greatly influences the design of implementing some of these features like BiDi. Any serious work from any no-std font-engine can really start once that's settled in my opinion, so we're basically hoping that either rustybuzz or all-sorts mature. The rest of the features aren't nearly as insurmountable by comparison.

A quick note: Pathfinder actually does a ton of these things like stem darkening, RGB AA/Positioning, color emoji, and such, but I'm assuming that's out of the question given its kinda hard to integrate with and also on the gpu.

[mooman219]

I wanted to add linebreaking to that list as well. It's wildly more complex than the spec implies. The unicode linebreak annex defines the bare minimum you should handle, but is not freetype grade in the slightest. There's a ton of nuance and opinions in the space and across platforms that add/remove line break opportunities, handle trailing whitespace, etc.

This feels like making a working group territory, but also there's not a ton of people that actually care about pure rust solutions :'( . I think the xi zulip is the most concentrated group so far, but there's not a big push for no-std solutions there. I'd be happy to break anything out of fontdue / work on a small tool for one of these features if you see an opportunity. I really want to see great no-std rust text in the future.

[RazrFalcon]

@mooman219 Afaik, freetype does only rendering. It doesn't do layout in any way. At least on linux, it's up to pango/Qt.

Also, Skia has it's own paragraph implementation on top of harfbuzz. This what flutter use (or plan to use).

PS: Why do you care about no-std solutions?

[mooman219]

@mooman219 Afaik, freetype does only rendering. It doesn't do layout in any way. At least on linux, it's up to pango/Qt.

Whoops, I'm conflating pango in with freetype again for no good reason. s/pango/freetype where it matters in my original comment.

Also, Skia has it's own paragraph implementation on top of harfbuzz. This what flutter use (or plan to use).

Yep! I've read a lot about it the past few weeks which is why I wanted to mention linebreaking as a problem that still needs to be solved in pure rust, even if it's just cloned from Skia.

PS: Why do you care about no-std solutions?

This is mostly a personal opinion. I think that a huge benefit of having the ecosystem in pure rust is going the next step to being no-std. Fontdue for example is used in some no-std environments including a small kernel and on consoles, and there's really no reason to not be no-std for the majority of the stack as far as I can tell, unless you're aware of something significant? Font config / fallback behavior may be system dependent, but that can be optional. The only thing preventing allsorts from being nostd is that it depends on a std gzip library for woff.

[RazrFalcon]

no_std is kinda an abstract term. fontdue uses alloc, so it's not pure no_std in my opinion. ttf_parser is, because it doesn't use heap at all.

In this sense, tiny-skia also can be no_std, because the only thing from std I'm suing is Vec.

[mooman219]

No-std is a well defined term since it's a crate level attribute. I like to think alloc is in the spirit of no-std.

Alloc is reasonably free to add on most no-std environments. Requiring all the threading/networking/file system/etc components is really heavy. Making tiny-skia no-std and just needing alloc would be great for some of the common no-std environments like consoles

[bschwind]

I've been using fontdue for rasterization and basic layout, both in a work project and a personal project. I don't have any complicated text layout or shaping issues to handle, but I've been very pleased with the results so far. Someone with sharper eyes than mine may be able to spot issues, I'm somewhat of a layman when it comes to seeing slight issues with text rendering.

[RazrFalcon]

@bschwind You have a very large text. It will always look good. The problem is in the tiny one.

To test shaping, you can simply try rendering Arabic text.

To test the layout, you can use a mix of Arabic and English.

[bschwind]

You have a very large text. It will always look good. The problem is in the tiny one.

Ahhh okay, today I learned :) Makes sense though!

I'm sure Arabic text or Arabic + English wouldn't turn out well currently. As mooman219 mentioned, it just does basic rasterization and layout at the moment.

[mooman219]

Anyway,

High-quality glyph rasterization library. Like ab_glyph_rasterizer, but with FreeType level of quality.

I think it would be productive to make an exhaustive list of what this means and see if we can those on the radar of the existing pure-rust rasters.

[RazrFalcon]

@mooman219 All I want to see, as I've mentioned a while ago, is a detailed comparison between different libraries. Here is PNG generated by fontdue, here is one by freetype, etc. As a library author, you should sell your library first. Performance is great, but this in not what I care about.

Font's rendering is one of the areas that I really don't want to go into, because it's too subjective.

[SimonSapin]

High-quality glyph rasterization library. Like ab_glyph_rasterizer, but with FreeType level of quality.

I’d expect this (and parsing as necessary for this) to be in scope for tiny-skia eventually. Everything else (shaping, layout, bidi, fallback, …) could be built separately on top, though that doesn’t mean smooth integration can’t exist.

Compare with Cairo v.s. Pango (and pangocairo integration).

[RazrFalcon]

@SimonSapin I actually haven't looked into how Skia renders glyphs yet, so I'm not sure. It clearly depends on freetype, so I assume it uses it for rasterization (or maybe just for outlining). It also has it's own minimal TrueType parser for some additional info I guess.

[dhardy]

Barely worth a mention here, but KAS-text implements:

• BiDi layout
• line breaking
• very primitive rich text

The library vaguely follows the relevant Unicode specifications but isn't fully compliant.

Caveats: the API won't suit all users, likely significant redesign is needed for freetype-quality and performance.

[RazrFalcon]

@dhardy I don't know when I would have time to implement text support, but it would probably be implemented from scratch using ttf-parser + rustybuzz + fontdb + ab_glyph_rasterizer/fontdue/own renderer. I.e. a pure Rust solution. Any non-Rust dependencies are forbidden.

[dhardy]

@RazrFalcon the only non-Rust dep in KAS-text is HarfBuzz, which is optional (and probably will be replaced soon).

In any case, I was more pointing out another "piece" for the Rust font ecosystem, for anyone here interested.

[RazrFalcon]

@dhardy What about font-kit?

Sure. I'm still not sure what API I want, but it probably will be low-level enough to be used in resvg. Basically, I need a very good font fallback support (which I don't have) and an ability to modify position of each glyph (each glyph should also have some metadata left, so I could match it with an original string).

[dhardy]

You're right, font-kit is used as a crutch to find fonts. Still need a native-Rust fontconfig. It may be worth trying the recent rust_fontconfig crate.

KAS-text does include the string index with each glyph, but no, I wouldn't recommend resvg depending on kas_text.

[RazrFalcon]

rust_fontconfig

Nice. I didn't know about this crate. I've actually wanted to write one myself, but fontconfig is an untested mess, so I've ended up writing fontdb. Also, looks like rust_fontconfig doesn't support fontconfig's disk cache, which will make it ultra-slow. This is the main problem with pure Rust font db: system libs doesn't scan fonts each time.

[rzerres]

@dhardy I don't know when I would have time to implement text support, ... You have been pretty active and progressed with ttf-parser, rustybuzz and fontdb. How about the missing peace of a glyph raserizer?

I'm asking, since OrbTK as a pure rust UI toolkit did fully integrate tiny-skia for 2D rendering. It is still lagging a freetype grade solution for text rendering and handling. This is needed to support i.e multiline text blocks. So I'm highly interested in any progress in this front.

[dhardy]

@rzerres it's done (couple of weeks ago). It appears I didn't enable the feature for docs.rs so you'll have to build the docs locally with the raster feature. See Cargo.toml. You still need some driving code; see text_pipe.rs.

Do let me know how you get on; I'm open to design changes. It's not FreeType grade; a couple of major omissions are support for embedded objects (e.g. emojis) and scalability to larger texts (possibly by stitching together "paragraphs"; I think some redesign is needed but haven't thought too much about it). Font fallback support is also not complete.

[dhardy]

@rzerres I added doc to the raster module: https://docs.rs/kas-text/0.3.3/kas_text/raster/index.html

swash may (eventually) be a another option in this space, though currently omits text layout.

[rzerres]

Thanks @dhardy. Right now I'm quite busy. But will have a look. Did you got feedback from @RazrFalcon conserning his needs to get a crate that links to resvg and `tinyskia'?

[RazrFalcon]

@rzerres resvg will probably forever use just rustybuzz. SVG text layout it too complicated and SVG-specific. Not much you can delegate to a separate library.

The scope of text support in tiny-skia is just a singe-line text layout for now. No paragraphs and other complex features.

[rzerres]

@RazrFalcon sadly i do know :) For OrbTk we are hunting for a feasable solution to support multiline TextBoxes. Sure a little bit of markdown parsing would be much appreciated. I started to just allow newlines.... but then ... selection and cursor handling is starting to get a nightmare. The deeper you think about it, the more the generic solution is needed.

[saona-raimundo]

I am sorry if this is out of the scope of this issue, but as far as I could understand, femtovg implements a few things that have been mentioned here (using ttf-parser, rustybuzz, unicode-bidi and unicode-segmentation).

[RazrFalcon]

Sure, there are multiple libs like this. I simply haven't had time to look into this. Mainly because I don't need it myself. I'm still not sure what scope text support in tiny-skia should have.

[UE2020]

Have you looked into swash? I tried out the demo and was very impressed with the quality, couldn't distinguish it from FreeType rendering in Chromium. Afaik swash supports both hinting and sub-pixel anti-aliasing.

[Speykious]

Have you looked into swash? I tried out the demo and was very impressed with the quality, couldn't distinguish it from FreeType rendering in Chromium. Afaik swash supports both hinting and sub-pixel anti-aliasing.

Damn, I tried it too and it's really impressive. I think I'm gonna try to use tiny-skia in combination with swash for my project. I've seen some mentions of how the difficult part of rendering text is when it is small, so I tried with font size 8 in some parts of the demo and it looks like this:

[saona-raimundo]

Adding references to solutions that come up, cosmic-text says that they implemented their own Font fallback.

Font fallback is also a custom implementation, reusing some of the static fallback lists in browsers such as Chromium and Firefox. Linux, macOS, and Windows are supported with the full feature set. Other platforms may need to implement font fallback capabilities.

[rice7th]

@saona-raimundo Yes, Cosmic-text is probably the way to go, since it is the most complete and the highest quality font rendering library in pure Rust out there. I think there isn't much choice if not to use it ¯_(ツ)_/¯

[vikigenius]

+1 for cosmic-text, in a short time it has become extremely popular and seems to be almost becoming the de-facto solution for Rust.

[notgull]

I've created a piet-tiny-skia crate. Among other things, it adds support for text rendering via the cosmic-text crate.

Even though piet-tiny-skia is released under the MPL, if someone wanted to port the text rendering code from there to this crate I would be happy to allow it.

[RazrFalcon]

@notgull Great! I still plan to implement my own text layout library for tiny-skia one day.

[rawhuul]

Is there any plan for implementing text rendering in tiny-skia, anytime soon??

[RazrFalcon]

Nope. Maybe in a couple of years.

[rawhuul]

Nope. Maybe in a couple of years.

I've read the thread and as you're very serious about implementing your own text rendering system, I would like you to look at stb_truetype.h. I hope you find it useful.

[RazrFalcon]

stb_truetype.h simply renders glyphs. It doesn't do layout. So until I write a layout library, or at least exact the one one form resvg, tiny-skia would not have any text.

[rawhuul]

OK, got it!

[rice7th]

@RazrFalcon fontdue supports complex text layout. Actually, even Cosmic text itself has a layout engine, though not as complete (no ellipsize or wrapping)

[RazrFalcon]

@rice7th fontdue does not support a complex text layout. cosmic-text does, via rustybuzz, but it's still designed for UI and I want a more fancy solution. Essentially, I want NSAttributedString from Apple's CoreText. And Rust has nothing even close to it.

[rice7th]

@RazrFalcon whoops I mixed the libaries 😵‍💫 Anyways, I still think that cosmic text is advanced enough for 75% of the use cases. It would still be great to have some good enough text support in the meantime rather than no text support at all.

[bennoinbeta]

@RazrFalcon What do you imagine under a NSAttributedString Rust implementation? How should/could the API & internal architecture look like?

I'm currently experimenting with a concept somewhat akin to NSAttributedString, aiming for rich text handling. The first design goal is to build something that takes in a (english) text & attributes and outputs a tiny-skia-path. Does something similar already exist?

It's a early Proof of Concept and based on usvg’s text_to_paths implementation, thus I’d appreciate insights on several design choices:

1. IntervalTree (like rust-lapper) for attribute management—efficient for overlapping attributes?
2. Structuring attributes—partial struct for direct access or a HashMap for flexibility?
3. Tokenizing text into TextFragment, WordSeparator, Linebreak, etc., beforehand for simpler transformation application, like text layout, emoji, etc. handling

and ofc any thoughts in general.

Thanks 🙌

Code: https://github.com/dyndotart/monorepo/blob/58-dyn-69-attributed_string-crate/crates/attributed_string/src/lib.rs

[RazrFalcon]

@bennoinbeta Yes, the idea is to extract usvg text handling code into a separate crate and make a nice, high-level API around it. The problem is that recently I had no free time, so I never actually looked into it. Therefore I cannot comment much.

The main issue with the current usvg implementation, and yours as well, is that the library should not care about outlines or rendering in general. It should accept text + fonts + styles and produce a list of glyphs with positions (aka layout). It might provide utilities to outline or even render glyphs, but it should be a separate step. This is needed to properly implement bitmap fonts, aka emojis.

Another usvg, or rather SVG, limitation is the lack of paragraphs. Aka multi-line text. That's a very complicated topic on its own and I have no expertise here. Especially when hyphenation is involved.

A yet another usvg limitation is the lack of glyphs caching. Which is a must.

Also, the API should provide a way to distinguish between logical and visual indexing. To properly support BIDI text. usvg does handle it internally, sans a couple of bugs, but it would be hard to provide a decent and fast high-level API for it. Think of it as character-to-glyph index and vise-versa mapping. In RTL text, the first glyph is the last character.

And so on and so on... It would be nice to have a library like this, but it's an extremely hard task.