Multiple copies of invalid characters and access time for a glpyh

[forthy42]

The next issue I want to attack:

When you render an invalid character, freetype-gl only caches it for the actual code point, instead of caching it once for a given font face.

This is because freetype-gl uses the Unicode code point and not the glyph index (which is 0 for all invalid characters). Given that freetype-gl removes the font library after caching the glyphs, it doesn't have the lookup table available, and a direct map from code point to glyph is reasonable.

However, the next issue that pops up is that the search within the glyphs is linear, because they are all just in an append-only vector, unsorted. That's not too bad for small fonts with few glyphs each, but for longer CJK texts, this would be too slow.

I suggest to have a double indirection: First stage is indexed with codepoint>>8, second stage is 256 code points (codepoint & 0xFF), if used. Most (non-latin) fonts also use some relatively dense ranges of Unicode, so that can be space efficient. And that gives a way to solve the invalid character problem: If we detect an invalid character (glyph index=0), we can map it to U+FFFD. If U+FFFD is already rendered, we point to the same glyph, otherwise we render it, store the glyph information in U+FFFD, and redirect there.

The actual glyphs could be stored as now, just add a quick index table to access them much faster. Bonus: The indirection only needs to be the index into the glyph vector, no full pointer needed.

[rougier]

While I understand the problem, I'm not sure to get every details of your solution. Would you have an implementation to play with ?

[forthy42]

I've checked in a branch with the suggested changes

https://github.com/forthy42/freetype-gl/tree/invalid

Generating the font .h file doesn't work yet... but most of the examples work. demos/benchmark before/after: Computing FPS with text generation and rendering at each frame... Number of glyphs: 4452 FPS : 717.54 (1794 frames in 2.50 second, 3194480.7 glyph/second) FPS : 768.69 (1922 frames in 2.50 second, 3422205.7 glyph/second) FPS : 767.99 (1920 frames in 2.50 second, 3419086.5 glyph/second) FPS : 766.99 (1918 frames in 2.50 second, 3414629.4 glyph/second) FPS : 744.97 (1863 frames in 2.50 second, 3316610.9 glyph/second)

Computing FPS with text rendering at each frame... Number of glyphs: 4452 FPS : 1647.58 (4119 frames in 2.50 second, 7335046.7 glyph/second) FPS : 1635.43 (4089 frames in 2.50 second, 7280942.1 glyph/second) FPS : 1650.55 (4127 frames in 2.50 second, 7348269.1 glyph/second) FPS : 1635.37 (4089 frames in 2.50 second, 7280656.7 glyph/second) FPS : 1644.08 (4111 frames in 2.50 second, 7319460.3 glyph/second)

after:

Computing FPS with text generation and rendering at each frame... Number of glyphs: 4452 FPS : 1125.37 (2814 frames in 2.50 second, 5010131.7 glyph/second) FPS : 1201.38 (3004 frames in 2.50 second, 5348565.8 glyph/second) FPS : 1199.32 (2999 frames in 2.50 second, 5339366.3 glyph/second) FPS : 1201.57 (3004 frames in 2.50 second, 5349368.8 glyph/second) FPS : 1194.61 (2987 frames in 2.50 second, 5318390.8 glyph/second)

Computing FPS with text rendering at each frame... Number of glyphs: 4452 FPS : 1632.69 (4084 frames in 2.50 second, 7268742.4 glyph/second) FPS : 1657.96 (4145 frames in 2.50 second, 7381233.9 glyph/second) FPS : 1645.44 (4114 frames in 2.50 second, 7325510.1 glyph/second) FPS : 1656.47 (4142 frames in 2.50 second, 7374591.7 glyph/second) FPS : 1646.22 (4116 frames in 2.50 second, 7328993.1 glyph/second)

[forthy42]

This experimental repository contains two further fixes:

1. The ...load_glyphs function should iterate while i < strlen(codepoints), not the number of utf8 code points, as i is the character index into the string, not the number of code points.

2. The kerning needs to be added in pairs, i.e. old, new and new, old. The existing glyphs also need their kerning tables to be updated, not just the new glyph its kerning table filled.

Anyways, I'll also update the kerning structure to use a similar approach (similar, because an array of floats does fine, no need for pointers in the second stage). I guess that most kerning tables are within one language glyph set, so close together in the Unicode space, too.

[rougier]

Wouah, speed difference is huge actually. Well done. Any idea why the font.h doesn't work?

[forthy42]

Yes, dumping the font+atlas needs a few more changes, then it will work again.

[forthy42]

Ok, the new format needs one indirection in the generated font.h, so that NULL pointers can be expressed.