Autodesk: Text - Update from suggestions

[erikaharrison-adsk]

Description of Proposal

Update text proposal according to suggestions.

• Add fontType to TextStyle schema.
• Add more detail about the three text plugins, and we will provide default plugins.
• Add a new build configuration "--textplugin" to control if the text plugins will be built together with USD. By default it is disabled.

Supporting Materials

Previous PRs:

• https://github.com/PixarAnimationStudios/OpenUSD-proposals/pull/38
• https://github.com/PixarAnimationStudios/OpenUSD-proposals/pull/24

Link to updated proposal

Contributing

• [X] I agree to and accept the Supplemental Terms.

[PierreWang]

We have an ASWF meeting last month, talking about this proposal. There were several questions and suggestions proposed in the meeting. I would like to answer the questions here.

Q: Language attribute may be required, so that the implementation will know what language the script is. A: We can decide the script from the unicode of the character. Because the font contains a limit set of charsets, the charset of the character may not be supported by the current font. In that case, we may display the character using a default character (such as a blank rectangle), or use font substituion to find another font which can display the character. So I think we don't need to add the language attribute.

Q: It might make sense to split it up into separable bits so we could land text as an object before styling etc. A: In our design, the TextStyle is a separate schema from the SimpleText or MarkupText. It can be applied to one or several text primitives.

Q: And if harfbuzz is used, does that mean it becomes a dependency on usd/hydra? A: Yes, if you build USD together with the default text plugins, harfbuzz will be a dependency. So we have added the build configuration "--textplugin". By default the config is off, so the plugins will not be built, and FreeType and harfbuzz are not required. If you turn on this config, the default text plugins will be built together with USD.

Q: The "alt-text" solution if we can not support the character. A: Sure if a character is not supported, we need to display a default character, or we will use font substitution to find another font which can support the character.

Q: I think authoring extents might be something to require. A: Sure. I will add the part how to get extents for a text primitive in the proposal.

Q: I was wondering if this could be considered just a "extra annotation" of a prim, like a "drawn bbox" could be, instead of adding new prims for texts. A: Text itself can be a separate object in the scene. So it worth to add a new prim for text.

Q: Simple question. Would one be able to interact with text in this proposal. Such as highlighting when hovering? A: Yes. In selection or highlighting, it works like the other prims. Whether you can hit the text at the rectangle of a character or at the stroke, depends on the implementation.

Q: Why we need material binding for text? A: The rendering of a character is something to get the opacity of each pixel. Different rendering technique may have different way to generate the opacity. So we use material binding to allow the user to give custom shader for the getOpacity function.

[PierreWang]

Sorry the proposal may be updated again next week. I will add a section about extents.

[meshula]

Could you say a few words about the following?

• harfbuzz; is it possible to make it optional? i.e. provide a fallback layout based on elementary glyph metrics and kerning?
• accessibility considerations. I'm not an expert here, but I sense that we might want to think about it. @dgovil is this something you have experience with?
• material binding: It seems a bit confusing to merely say that one should bind a material in order to see something rendered. This makes sense, but any of the default materials like PreviewSurface are inappropriate in that they are all about previewing surfaces. One can imagine that people will on day one want materials that implements "closest" sampling for bitmap style fonts; a filtered sampler for typical font atlas purposes, and an SDF based shader for fancy rendering. My intuition is that we should provide either one Uber-text material, or three specialized ones. To avoid branching, I lean to three.
• bitmap fonts. sdf fonts take a while to generate at good quality, and people may wish to provide atlased fonts directly. I think we should explore a bitmapped atlassed font scheme, potentially as a schema. This would involve a texture reference for the atlassed font, coordinates into the atlas per glyph, basic metrical information, and a kerning table. There are a spectrum of use cases, ranging from elementary labeling to bidi multilingual annotations, and a variety of font technologies, and I think we should accommodate the fact that rendering text is not just the whole enchilada or nothing :)

[dgovil]

Q: Language attribute may be required, so that the implementation will know what language the script is. A: We can decide the script from the unicode of the character.

I'm not sure this is accurate. You have various issues like Han unification that prevent a script being truly inferable, but outside of that it's generally not an issue.

That aside, my point was about language vs script detection.

I do think a language consideration would make sense.

Perhaps it would make sense to have it be a separate proposal that is associated with this one. I think it could be useful to separate it from accessibility as well since it could be shared across multiple domains. Imagine a scenario where you have text in your scene but want to present different text per language as well.

In effect what I'd propose is that the USD file have a stage level metadata (and possibly prim level with Spiff's new proposal) to specify language. I'd also propose that this be allowed to be a purpose on attributes for when it differs from the primary language.

Something like

#usda 1.0
(
     language = "en_ca"
)

def foo {
     string text = "Colours are awesome"
     string text:en_us = "Colors are awesome, but the letter U is not"
     string text:fr = "La couleur est géniale"
}

The benefit here is that you can also specify translations in separate USD files and layer them in.

Another reason I'd advocate for language tags is that a theoretical screen reader could enunciate words differently based on the language specified.

[spiffmon]

I really like the multiple language encoding idea, @dgovil . Perhaps not as germaine for labels on parts, but for anything more conversational or longer, it'll be useful to be able to intentionally provide non-auto-generated translations.

And I'd advocate skipping layer metadata entirely for the "default encoding" tag, and go directly to an API schema; in multi-user environments, you'll likely see avatars being referenced in from many different parts of the world, I'd imagine?

[meshula]

Moved discussion on languages to https://github.com/PixarAnimationStudios/OpenUSD-proposals/issues/49

[meshula]

Ok, busted out the conversations into distinct issues. @PierreWang I didn't make an issue for extents since you said you have new work incoming, so I'll anticipate the discussion is picked up when that's ready.

[PierreWang]

@meshula @dgovil @spiffmon Thank you for your suggestions. I am currently working on the line style proposal. I will have a deep think about your suggestions in the text proposal after the line style proposal updated.

[PierreWang]

Could you say a few words about the following?

• harfbuzz; is it possible to make it optional? i.e. provide a fallback layout based on elementary glyph metrics and kerning?
• accessibility considerations. I'm not an expert here, but I sense that we might want to think about it. @dgovil is this something you have experience with?
• material binding: It seems a bit confusing to merely say that one should bind a material in order to see something rendered. This makes sense, but any of the default materials like PreviewSurface are inappropriate in that they are all about previewing surfaces. One can imagine that people will on day one want materials that implements "closest" sampling for bitmap style fonts; a filtered sampler for typical font atlas purposes, and an SDF based shader for fancy rendering. My intuition is that we should provide either one Uber-text material, or three specialized ones. To avoid branching, I lean to three.
• bitmap fonts. sdf fonts take a while to generate at good quality, and people may wish to provide atlased fonts directly. I think we should explore a bitmapped atlassed font scheme, potentially as a schema. This would involve a texture reference for the atlassed font, coordinates into the atlas per glyph, basic metrical information, and a kerning table. There are a spectrum of use cases, ranging from elementary labeling to bidi multilingual annotations, and a variety of font technologies, and I think we should accommodate the fact that rendering text is not just the whole enchilada or nothing :)

• Yes. A fallback layout should be provided.
• Sorry I am not sure about what do you mean by accessibility. For the other two comments let's discuss in the related conversation pages.

[meshula]

By accessibility, I mean (1) language affordances, as proposed by Dhruv, and (2) beyond language tagging, an "alt text" version for readability. For example, a long block of mark up heavy text might be illegible without a mark up renderer, so an "alt text" version for human readability might be the same text without formatting. An another example might be a tag to indicate text that is meant for other modalities such as text to speech.

[PierreWang]

illegible without a mark up renderer, so an "alt text" version for human readability might be the same text without formatting. An another example might be a tag to indicate text that is meant for other modalities such as text to speech.

I understand. In my implementation the default behavior without a markup parser is to directly use the whole text string as a plain string. An "alt-text" is a better solution.