Basic rich text support

[rparrett]

What problem does this solve or what need does it fill?

I'd like to be able to render some text, but have some bits of that text show up in a different color and have text layout stuff still generally work.

e.g. A particular word in this sentence.

It's currently possible to approximate this in the UI in some cases by throwing multiple text elements in a flexbox. With text in a 2d scene, I was able to hack something together with Changed<CalculatedSize>, but it isn't pretty. In both situations, it seems like there's some inconsistency in positioning due to kerning (maybe).

What solution would you like?

This seems like a bit of a slippery slope towards something I definitely couldn't take on myself, so I think it would be great to start with color only.

UnityUI supports an html-like syntax with a few non-nested tags and I think adding support for at least a "color" tag would be doable.

e.g.

A particular <color=#ff0000ff>word</color> in this sentence

Perhaps this would be an opt-in feature with a new bool in TextStyle e.g. rich: true, or a separate RichText2dBundle.

What alternative(s) have you considered?

Godot's RichTextLabel uses a similar syntax with non-nestable square-bracketed tags related to "bbcode"

Adding this as a bevy_richtext crate could be possible, but I assume that would involve a bunch of undesired code duplication from bevy_text / bevy_ui.

Additional context

My specific use case for this is a typing game where I want to

• Have floating centered Text2d above various game objects that contain a word for the player to type
• Color a portion of that text differently as the player types it
• Additionally position other elements relative to the dimensions of that text, even if it changes

Here's an illustration of what I'm after: https://imgur.com/a/7xdwY1G

[arekbal]

I wouldn't consider parsing some specialized syntax as "basic". The syntax layer could/should be implemented in user-space on top of some stylized basic "text chunks/segments" that each have style attached and could be easily rearranged, restyled.

I would like to see "fuller" document-like support: text wrapping, word spacing+alignment, text selection, caret, system commands(copy, cut, paste, select all). If I were to be able to write blog posts in markdown to then preprocess them to bevy consumable asset and deploy it as wasm... that would be cool. ;)

The list could go further... multiplatform url routing support, data binding, templates. It's easy to imagine future/feature but somebody has to implement this. ;)

[mockersf]

for reference, the styled text doc from unity: https://docs.unity3d.com/Packages/com.unity.ugui@1.0/manual/StyledText.html

I did a quick, naive and incomplete implementation of that: https://github.com/vleue/bevy_rich_text

• Changing a styled text often is slow: it delete all child entities, re-split the string and re-create child entities. there could be an optimisation here by only deleting/recreating the part of the text that changed
• When changing the style this way, the text is dancing. It may not be an issue if it doesn't change often, like for some information text with a few words that needs to be highlighted, but if part of your gameplay requires style to change it's definitely noticeable

[rparrett]

That's really cool. Seems like a great generalized implementation of what I ended up doing. (For the text in the my UI layer, anyway). Seems like it would be plenty fast for most of the use cases I can imagine.

In my specific case, I was able to get away with just spawning two children and reusing them, as I'm only ever highlighting the left side of some portion of my text.

The "text dancing" is what I called "inconsistency in positioning" in the original issue. It's a bit distracting in my case, as changing-text-all-over-the-screen is the core gameplay. In my non-ui implementation, I attempted to remove the text dancing by rendering a Color::NONE copy of the whole text and positioning the highlighted and non-highlighted portions relative to the left and right edges of the whole text. That ended up giving me a consistent width for the label, but the dancing remained.

I'm not sure if it's a kerning thing, an antialiasing thing, or something else but it hasn't bothered me enough to dive into the internals just yet.

[mockersf]

I'm not sure if it's a kerning thing, an antialiasing thing, or something else but it hasn't bothered me enough to dive into the internals just yet.

I think for the UI part, this is due to stretch doing some rounding. I tried some debug prints with a monospaced font to reduce number of reasons this could happen: format is :

text                                               -> calculated width  | global_transform.translation.x

Lorem i                                            -> 229.27734  | 114.75
psu                                                -> 98.26172   | 278.5
m dolor sit amet, consectetur                      -> 949.8633   | 802.5

Transform of the last part should be 229.27734 + 98.26172 + 949.8633/2.0 => 802.47071, so a difference of -0.03

Lorem ipsum dolor sit amet, conse                  -> 1080.8789  | 540.5
cte                                                -> 98.26172   | 1130
tur                                                -> 98.26172   | 1228.25

Here, 1080.8789+98.26172+98.26172/2.0 => 1228.27148, so a difference of +0.02

Not sure how much such tiny differences really have an impact, but it may be enough to jump from one pixel to the next one

There may be an issue with how the text size itself is calculated as there are small variations (with a non monospaced font) on the cumulated calculated size depending on where I split the string, but it's even smaller (in the order of ±0.0003)

In my non-ui implementation

Could you give me a link to this so that I could check?

[tigregalis]

I think we should take advantage of glyph_brush_layout's API in this case.

Currently the Text component has made an assumption that there is exactly one section only, but it probably wouldn't take too much effort to make use of more than one section, but this would be a breaking change.

glyph_brush_layout has sections:

// Layout "hello glyph_brush_layout" on an unbounded line with the second
// word suitably bigger, greener and serif-ier.
let glyphs = Layout::default().calculate_glyphs(
    fonts,
    &SectionGeometry {
        screen_position: (150.0, 50.0),
        ..SectionGeometry::default()
    },
    &[
        SectionText {
            text: "hello ",
            scale: PxScale::from(20.0),
            font_id: FontId(0),
        },
        SectionText {
            text: "glyph_brush_layout",
            scale: PxScale::from(25.0),
            font_id: FontId(1),
        },
    ],
);

bevy\crates\bevy_text\src\text.rs

pub struct Text {
    pub value: String,      // <--- combine these into effectively pub sections: Vec<(String, Handle<Font>, TextStyle)>?
    pub font: Handle<Font>, // <--- I'm not sure if Changed<Text> query would detect changes to these?
    pub style: TextStyle,   // <--- alignment needs to be split out again however,
}                           // as alignment is a block-level consideration rather than a section-level consideration

bevy\crates\bevy_text\src\draw.rs

pub struct TextStyle {
    pub font_size: f32,           // <--- section level styling (along with font)
    pub color: Color,             // <--- section level styling
    pub alignment: TextAlignment, // <--- block level styling
}

pub struct DrawableText<'a> {
    pub render_resource_bindings: &'a mut RenderResourceBindings,
    pub position: Vec3,
    pub scale_factor: f32,
    pub style: &'a TextStyle,                  // <-- combine into Vec<PositionedStyledGlyph> perhaps
    pub text_glyphs: &'a Vec<PositionedGlyph>, // <-- or Vec<(TextStyle, Vec<PositionedGlyph>)>
    pub msaa: &'a Msaa,
    pub font_quad_vertex_descriptor: &'a VertexBufferDescriptor,
}

bevy\crates\bevy_text\src\glyph_brush.rs

impl GlyphBrush {
    pub fn compute_glyphs<S: ToSectionText>(
        &self,
        sections: &[S], // <--
        bounds: Size,
        text_alignment: TextAlignment,
    ) -> Result<Vec<SectionGlyph>, TextError> {
        let geom = SectionGeometry {
            bounds: (bounds.width, bounds.height),
            ..Default::default()
        };
        let section_glyphs = Layout::default()
            .h_align(text_alignment.horizontal)
            .v_align(text_alignment.vertical)
            .calculate_glyphs(&self.fonts, &geom, sections);
        Ok(section_glyphs)
    }

bevy\crates\bevy_text\src\pipeline.rs

    pub fn queue_text(
        &mut self,
        id: ID,
        font_handle: Handle<Font>,
        fonts: &Assets<Font>,
        text: &str, // <--- sections: &[Text]?
        font_size: f32,
        text_alignment: TextAlignment,
        bounds: Size,
        font_atlas_set_storage: &mut Assets<FontAtlasSet>,
        texture_atlases: &mut Assets<TextureAtlas>,
        textures: &mut Assets<Texture>,
    ) -> Result<(), TextError> {
        // ...
        let section = SectionText {
            font_id,
            scale: PxScale::from(font_size),
            text,
        };
        // ...
        let section_glyphs = self
            .brush
            .compute_glyphs(&[section], bounds, text_alignment)?;

bevy\crates\bevy_ui\src\widget\text.rs

            match text_pipeline.queue_text(
                entity,
                text.font.clone(),
                &fonts,
                &text.value, // <-- &text.sections?
                scale_value(text.style.font_size, scale_factor),
                text.style.alignment,
                node_size,
                &mut *font_atlas_set_storage,
                &mut *texture_atlases,
                &mut *textures,
            ) {

I'll have a play and see if I can get something working. But if anyone else wants to work on it, please do.

[tigregalis]

I've got something working, and afaict it works well. I'll tidy up, add a change log entry, fix the examples (and make sure they work) and submit a PR.

The new API is as follows:

// Component
pub struct Text {
    // pub value: String,             // moved to TextSection
    // pub font: Handle<Font>,        // moved to TextSection
    // pub style: TextStyle,          // moved to TextSection
    pub sections: Vec<TextSection>,   // +
    pub alignment: TextAlignment,     // +
}

pub struct TextSection {              // new
    pub value: String,                // +
    pub font: Handle<Font>,           // +
    pub style: TextStyle,             // +
}

pub struct TextStyle {
    pub font_size: f32,
    pub color: Color,
    // pub alignment: TextAlignment,  // moved to Text
}

It might make sense to either:

1. spread the members of TextStyle into TextSection, or
2. move font from TextSection into TextStyle

Example below:

The "Score: " and the "10" have different fonts, font sizes, and colours

fn setup(
    commands: &mut Commands,
    mut materials: ResMut<Assets<ColorMaterial>>,
    asset_server: Res<AssetServer>,
) {
    // Add the game's entities to our world
    commands
        // ...
        // scoreboard
        .spawn(TextBundle {
            text: Text {
                sections: vec![
                    TextSection {
                        font: asset_server.load("fonts/FiraSans-Bold.ttf"),
                        value: "Score: ".to_string(),
                        style: TextStyle {
                            color: Color::rgb(0.5, 0.5, 1.0),
                            font_size: 40.0,
                        },
                    },
                    TextSection {
                        font: asset_server.load("fonts/FiraMono-Medium.ttf"),
                        value: "".to_string(),
                        style: TextStyle {
                            color: Color::rgb(0.9, 0.3, 1.0),
                            font_size: 60.0,
                        },
                    },
                ],
                ..Default::default()
            },
            style: Style {
                position_type: PositionType::Absolute,
                position: Rect {
                    top: Val::Px(5.0),
                    left: Val::Px(5.0),
                    ..Default::default()
                },
                ..Default::default()
            },
            ..Default::default()
        });
    // ...
}

fn scoreboard_system(scoreboard: Res<Scoreboard>, mut query: Query<&mut Text>) {
    for mut text in query.iter_mut() {
        text.sections[1].value = scoreboard.score.to_string();
    }
}

[arekbal]

https://makepad.dev/ has currently best rust text rendering right now IMO. It's MIT licensed. https://github.com/makepad/makepad It's not perfect, still a work in progress. But considering how "immature" bevy text is... using makepads code as a baseline, might help a little... I hope. Still learning bevy myself. I will have a look at what I think it is missing in making text better, whatever that means.

[arekbal]

@tigregalis +1 Area for further improvement: Text editors are usually the main example of flyweight pattern usage. I believe and hope there would be more and more fields incoming into text style. Flyweight pattern here would let text section structure stay smallish while letting the style expand in size. I mean all "Style" fields + structrues should be sooner or later designed this way as an indirection(handle?).

[rparrett]

@mockersf I ripped the relevant code out and threw it here: https://github.com/rparrett/bevy-test/tree/tower-slot-label-dance

[mockersf]

@mockersf I ripped the relevant code out and threw it here: https://github.com/rparrett/bevy-test/tree/tower-slot-label-dance

Couldn't find any obvious issue, there is a rounding error of ±0.00003 but that seems to be due to f32 precision...

good news is #1245 fixes this completely, I ported your example to it to test

[rparrett]

I consider this resolved, as engine modifications are probably no longer needed to implement this sort of thing.

Thanks, everyone!