Text rendering

[RazrFalcon]

Text rendering is not supported and not planned. This is an absurdly complex task and the Rust ecosystem doesn't provide basically any libraries to implement this.

We need:

• [x] Font parser: ttf-parser.
• [x] Text shaper: rustybuzz or all-sorts.
• [ ] Font database with a font fallback mechanism.
• [ ] High-quality glyph rasterization library. Like ab_glyph_rasterizer, but with FreeType level of quality.
• [ ] Code in tiny-skia that ties it all together.

[bennobuilder]

@RazrFalcon No worries, I'll give it a try as I've a bit free time right now (but not the expertise in Rust and especially text layout, shaping, .. so don't expect too much). But learning by doing :) The current iteration offers the following API:

        let text = String::from("Hello, world!\nשלום עולם!\nThis is a mix of English and Hebrew.");
        let attrs_intervals = vec![
            AttrsInterval {
                start: 0,
                stop: 10,
                val: Attrs::new()
                    .font_family(FontFamily::Monospace)
                    .font_weight(400)
                    .font_size(24.0),
            },
            AttrsInterval {
                start: 10,
                stop: text.len(),
                val: Attrs::new()
                    .font_family(FontFamily::Serif)
                    .font_weight(400)
                    .font_size(12.0),
            },
        ];

        let mut attributed_string = AttributedString::new(
            text,
            attrs_intervals,
            AttributedStringConfig {
                bbox: Vec2::new(100.0, 100.0),
                ..Default::default()
            },
        );

        // Tokenize the text (String) in logical intervals for further processing
        attributed_string.tokenize_text(&mut fonts_cache);

        // Apply the layout like linebreaks, letter spacing, ..
        attributed_string.layout();

        // Create tiny-skia-path
        let path = attributed_string.to_path(&mut fonts_cache);

Right now, it's early work in progress and mostly a mixture of concepts from usvg and cosmic-text. It's also still bound to tiny-skia-path but its possible to make it optional later and instead iterate over the glyphs as needed (like in to_path() implementation).

Would appreciate your thoughts on the current API and (maybe) implementation as I've no plan what I'm doing and am figuring it out on the go..

Implementation details: Initially the text is divided into spans (tokenize_text (1)), each corresponding to an attribute interval for potential caching and determining direction (ltr or rtl). Each span contains shape tokens like TextFragment, WordSeparator, Linebreak, .. which encapsulate glyphs. This assigns glyphs a logical context for subsequent processing in the layout (2) step. Idk whether I'm on the right track and whether this approach makes any sense and is efficient & performant in any means.

Thanks 🙌

[dhardy]

@bennoinbeta the attribution is vaguely similar to my FormattableText excepting that it doesn't allow custom trait impls and doesn't require font face lookup be done in advance (an optimisation for when the result will be drawn many times).

How do you plan on handling overlapping start..stop regions and uncovered regions? I solved that by having defaults for everything and only encoding the start. It looks like your system doesn't allow merging attributes from overlapping regions anyway?

... this is not the right place to discuss this type of detail however.

[bennobuilder]

@dhardy Yeah probably not the right place @RazrFalcon ?

How do you plan on handling overlapping start..stop regions and uncovered regions?

I'm using rust-lapper and added a function(divide_overlaps_with()) to merge overlapping intervals (See: https://github.com/sstadick/rust-lapper/pull/23 , not merged yet though)

  self.attrs_intervals.divide_overlaps_with(|overlaps| {
            let mut merged_attrs = Attrs::new();
            for &attrs in overlaps.iter() {
                merged_attrs.merge(attrs.clone());
            }
            return merged_attrs;
        });

For uncovered regions I structured the Attrs struct in a way that all attributes are optional and made them only accessible via getter methods in which I apply the defaults (if None). However, I actually don't like this approach but couldn't figured out a better way in the Rust ecosystem yet.

Thanks for sharing FormattableText, I'll have look. Does it maybe already accomplish what I'm trying to build. Like a more abstract library positioning glyphs (supporting layout, ..) with an API like the AttributedString one from Apple than e.g. cosmic-text? Thanks 🙌

[dhardy]

@bennoinbeta the kas-text library is designed to handle text layout from an str (optionally attributed) to positioned glyphs, with some (optional) helpers towards rendering. It covers roughly the same ground as cosmic-text, but older, and more designed for explicit cache control (stateful), so not the simplest API.

[RazrFalcon]

@bennoinbeta Not being much of an expert here as well, I would try to clarify what resvg needs:

1. Static layout and not dynamic. All we care about is a heavily styled text layout. We don't care about caching, performance, content bounds resize and any kind of user input. It will not be used to render text at 60Hz. Which leads to a very different design.
2. Stateful API (I hope I use this term correctly). We create an AttributedString, call the layout() and get all the info we need (positions, glyphs, attached styles (like a gradient in SVG), underlines, etc). No need for a callback mess like NSLayoutManager.
3. Ability to post-process the layout. For example to implement text-on-path. To do so, we need a quite low-level access to the final layout.

Overall, I appreciate your effort, but I'm not sure I would be able to use it. Text support is such a fundamental feature that I would rather write it myself (one day I hope) to understand all the intricacies. Especially if we account all the missing features in usvg, like shape layout, bitmap fonts, caching, proper font fallback, embedded fonts, glyphs rasterization (aka SVG and COLR fonts), etc. It would be nice to offload it to some true text expert, maybe as a part of a bigger project, but that's highly unlikely. There are probably like 5 people who know how the text works.

I'm not even really sure what I want from a AttributedString-like library myself, therefore I cannot judge your implementation either.

[bennobuilder]

@RazrFalcon Understood, and thanks for the insights :) I'm hopeful you'll either find a skilled collaborator for the text layout or manage to allocate some time for it eventually. Looking forward to an expert-crafted abstract AttributedString-like library. In the meantime, I'll use my hacky and not so thought through implementation, given the lack of alternatives. cheers :)

[nicoburns]

I've just implemented an example demonstrating integration of parley with tiny-skia using skrifa to do scaling and hinting. Emoji rendering is currently missing which is:

• Partly a limitation of skrifa (bitmap emoji formats). This can be worked around be using swash instead.
• Partly a limitation of tiny-skia. There are missing gradient type(s) (we need both Radial and Conic/Sweep) for full COLRv1 support.
• Partly just a matter of writing integration code to implement skrifa's ColorPainter trait on top of tiny-skia (need code to manage clips and transforms). This would immediately enable COLRv0 fonts and would also enable COLRv1 if the gradients types are implemented.

[notgull]

I've learned to appreciate tiny-skia's smaller API. I'd be fine if text handling was entirely absent here.

[rawhuul]

yeah @notgull let it do one thing and do it well

[jsprog]

@rawhuul I won't hurt adding text support behind a feature flag

[PerryDesign]

Would this provide a reason to implement?

https://github.com/googlefonts/fontations

[RazrFalcon]

No. fontations is still an alpha. And we already have rustybuzz + ttf-parser. That's not the problem. What is missing is a layout library.