search

mapbox / DEPRECATED-mapbox-gl

Issue-only repository for discussion of Mapbox GL (DEPRECATED)
https://www.mapbox.com/mapbox-gl/
21 stars 5 forks source link

Complex Text Rendering #4

Closed mikemorris closed 7 years ago

mikemorris commented 8 years ago

What is the state of text rendering in Mapbox GL?

We currently do not render scripts that require bidirectional support or complex text shaping correctly in mapbox-gl-native or mapbox-gl-js. This ticket will track adding proper support to both projects.

Broken Arabic labels example courtesy of @mushon

What's missing?

We need to add the following functionality for proper text rendering:

In terms of scripts affected by this, Hebrew requires bidi support, Indic scripts like Hindi can require complex text shaping, and Arabic requires both bidi and complex text shaping support. Additionally, implementing the Unicode line-breaking algorithm should improve support for cases like smarter line breaking in Chinese.

How do we currently handle fontstack fallbacks?

Currently, the Protobuf-encoded "glyph tiles" we create with node-fontnik are a composited "fontstack" with missing glyphs in fonts higher in the stack being filled in by glyphs from fonts further down the stack and we therefore end up with a combined Helvetica, Arial Unicode fontstack with per-glyph fallbacks in rendered text.

Fontstack Coverage
"Helvetica" Latin
"Arial Unicode" Latin, Arabic
"Helvetica, Arial Unicode" Helvetica Latin, Arial Unicode Arabic

Why will this not work for complex text shaping?

Because shaping tables are specific to a font file, to apply shaping properly we will need to work exclusively with glyphs from a single font. Instead of using "fontstack" glyph tiles, we will need tiles which contain all the glyphs in a given range for a single font. This approach should also limit glyph atlas duplication for multiple fontstacks with a common fallback.

How will we do this?

We will first need to segment each label into text runs (splitting words into individual segments, and splitting Arabic text segments from numerical segments for example) with the Unicode bidi algorithm. Then, for each segment, we will attempt, with each font in the fontstack until a match is found, to shape the text segment with a single font's shaping table and check whether all characters in the shaped result can be rendered by that font (using a glyph coverage file). If coverage is incomplete, we will fall back to the next font in the stack.

(It's possible we could check glyph coverage first, but the necessary glyphs may change after shaping, and the glyph coverage check would have to be repeated. We should test performance to determine whether a possibly inaccurate initial coverage check is faster than redundant shaping passes for fonts lacking glyph coverage.)

Example

For the fontstack "Open Sans, Arial Unicode", no glyphs change when shaped with Open Sans/gsub.sfnt - do all characters in résumé exist in Open Sans.coverage.json? NO? Missing é? Reshape with Arial Unicode/gsub.sfnt, then check if all characters in résumé exist in Arial Unicode.coverage.json

Once a font with matching coverage has been determined, we can request glyph tiles from a single font containing the necessary glyphs, like Arial Unicode Regular/0-255.pbf.

How will we get/use these "shaping tables"?

Shaping tables are contained in font files as GSUB (glyph substitution), GPOS (glyph positioning) and KERN (kerning) tables, which can be read by the FreeType function FT_Load_Sfnt_Table. We will need to extract these tables from from uploaded font files, then request them from the client through an API. We've started work on extracted shaping tables but it isn't quite functional yet.

To use these shaping tables, we will need to pass them into HarfBuzz for mapbox-gl-native, or an emscripten port for mapbox-gl-js. I'm not sure if HarfBuzz currently has an interface for reading raw shaping tables (it generally works with full font files). If this interface doesn't currently exist, we'll need to add it.

Resources

Universal

C++

JavaScript

/cc @mapbox/gl

lucaswoj commented 7 years ago

Closing this ticket as part of an effort to merge this repo into the mapbox-gl-js repo. Work on this project is nearing completion. Status updates and conversation will continue at https://github.com/mapbox/mapbox-gl-js/issues/3708

lucaswoj commented 7 years ago

The initial phase of support for right-to-left and Arabic script is now complete.

The primary tracking issue for the remaining complex text challenges is now: https://github.com/mapbox/mapbox-gl-native/issues/7774

chuckhacker commented 7 years ago

Great!!!

johnnybegood7 commented 7 years ago

Any consideration of using Graphite2 to handle this on top of FT?

ChrisLoer commented 7 years ago

@johnnybegood7 Our assumption is that if we were adding support for fonts that used Graphite, it would be through the Harfbuzz wrapper of Graphite (see discussion in https://github.com/mapbox/mapbox-gl-native/issues/7774).

AbdulrhmanBazrto commented 7 years ago

was this problem solved for android SDK ? :)

zugaldia commented 7 years ago

@3bo0o0odee We integrated ICU to support bidirectional text layout and Arabic text shaping with https://github.com/mapbox/mapbox-gl-native/pull/6984 which is available in the Android SDK 5.x series. The remaining complex text work is ongoing and tracked on mapbox/mapbox-gl-native#7774.

MahdiAstanei commented 7 years ago

Is there any body to create a step by step fix for android? can we use Mapbox on android and USE for Persian or any Arabic countries?

kkaefer commented 7 years ago

@MahdiAstanei Upgrading Mapbox to the latest released version (currently 5.1) will include this fix. You don't have to do any special configuration.

mushon commented 7 years ago

Thanks for making progress there but what about supporting this at Mapbox Studio and supporting the static (bitmap) tiles? We really can't design like this… @ChrisLoer can you give us an update?

ChrisLoer commented 7 years ago

@mushon The plan right now is that Studio support won't come until we've integrated RTL text into the core library. We are exploring using web assembly as a way to integrate more "native" code into GL JS -- any solution we come up with there will probably include RTL text. I know that "just enable the plugin in Studio" would be a more immediate solution for you, and that's still an open discussion, but it's not our current plan. An alternative short term solution we've discussed is having Studio preview raster tiles generated by api-gl, which would include the shaping support.

When you say static (bitmap) tiles, do you mean tiles generated by api-gl? They should already be fixed.

mushon commented 7 years ago

@ChrisLoer I would really appreciate at least a browser plugin that would give us something to work with in studio until you implement a more robust solution. As for the bitmap tiles, you are right, they are processed correctly both for static images and for leaflet. It's just that the static image interface is inconsistent as it currently shows the wrong preview to what it actually generates.