Improvement for Asian language

[GoogleCodeExporter]

For Asian language, such as Chinese, the common used character is about 6K 
(total amount exceed 10K). So the current implementation of texture font is not 
enough. Potential improvement could be:
1. Use hash table for glyph fetching.
2. Create glyphs on the fly.  It may introduce some streaming mechanism that a 
background thread keep reading from the input stream and render all glyph to 
the atalas
3. Texture font may contain multiple atalas now, so texture font : texture 
atlas is n:n relationship (currently is n:1). Reference counting on atlas may 
help.
4. Vertex buffer should sort all glyph mesh according texture id.

I'm now porting freetype-gl to my game engine, and intent to solve the problems 
mentioned above. 

In the attach file is the code of hash table of glyph. It's tested but without 
comment ( interface is straight forward ). A generic hash table may be better, 
but it's normally a key-value pair and take more memory. For thousands of 
glyph, it's a bit waste. I hope it may help.

Original issue reported on code.google.com by wangyong...@gmail.com on 6 Feb 2012 at 5:04

Attachments:

• glyph-dict.h
• glyph-dict.c

[GoogleCodeExporter]

1. That's great ! I intended to implement such a hash table but was still 
searching for the best solution.

2. "Create glyphs on the fly", what do you mean by that. Currently, glyph are 
created when they are not present within the atlas.

3. The problem with multiple atlas is that you cannot render a vertex buffer 
with a single call,  can you ? another solution could be to use a 3d texture 
where each layer would be an atlas but I don't know how common is 3D texture on 
old and new graphic card.

4. What would be the advantage of doing this ?

I will look at your dict code and see how to use it. Maybe we'll need some 
benchmark at some point, especially for Chinese text.

Nicolas

Original comment by Nicolas.Rougier@gmail.com on 6 Feb 2012 at 7:40

[GoogleCodeExporter]

>> 2. "Create glyphs on the fly", what do you mean by that. Currently, glyph 
are created when they are not present within the atlas.

Sorry, please forget about it. It's not the issue of freetyp-gl. Briefly,  in a 
real-time game, I wanna seperate glyph creation from the main thread to 
minimize the impact to the frame rate. 

BTW, I found that texture_font_load_glyph load face and done face each time. 
For latin language it's OK. With Asian language, you have to call this function 
many times. Each load face may trigger disk I/O, and consume about 0.009 
seconds. For a game that runs in 60FPS, it eats half time of one frame. I 
should be able to decide the time fo done face.

>>3. The problem with multiple atlas is that you cannot render a vertex buffer 
with a single call,  can you ? another solution could be to use a 3d texture 
where each layer would be an atlas but I don't know how common is 3D texture on 
old and new graphic card.
>>4. What would be the advantage of doing this ?

No, you can't render all text in a single call if multiple atlas involved. 
Because glyphs of text won't override each other, you could sort the triangles 
of these glyphs according texture id. Triangles that use same texture are 
rendered together. It could reduce draw calls as much as possible.

I will post a few demo about this later. 

Original comment by wangyong...@gmail.com on 9 Feb 2012 at 6:36

[GoogleCodeExporter]

I have played around with freetype-gl for a few days, and finally I use it in 
my game. I have made a lot of changes and optimizations to the source. I should 
submit a demo to demonstrate the idea I mentioned about background loading and 
massive glyphs management of Asian language. But my code is written in C++ and 
introduce a few platform dependence such as atomic operation and threading. So 
maybe I don't have time to do this.

Here I will talk about some ideas that I have implemented. I hope it could 
help, and your suggestions are welcome.

1. I keep FT_Library and FT_Face until I really do not need it. Each call of 
FT_New_Face may cost about 1/3 frame time of a 60 FPS game. My game has a text 
field that players could input any characters, so the program may keep 
generating new glyphs. 

2. I use hash table as container of glyphs. It's bad to find in 1000+ glyphs 
with a vector. BTW: The glyph-dict.c that I submit above has a fatal error. I 
have fixed it and the new source is attached below.

3. For Chinese,  hinting is unnecessary ( and is tough to do this for 1000+ 
glyphs on the fly... ) because all glyphs are box. 

4. For Chinese, a common paragraph of size of 20+KB, may contain 1500+ glyphs 
and consume several 512x512 atlases. I add a next pointer to texture_atlas_t 
and make texture_font_t::atlas an intrusive linked list. I also store in 
texture_glyph_t::id the pointer of the atlas which the glyph belongs to. When I 
fill indices buffer, I will keep the vertices that reference the same texture 
stay aside (note that because characters never override each other, so you can 
render them in any order). With this way I could minimize the texture switch 
and draw calls.

5. Distance transform is good for big font. It just take a little atlas and 
scale with no cost. But generating edt bitmap is slow.

Original comment by wangyong...@gmail.com on 7 Mar 2012 at 6:13

Attachments:

• glyph-dict.c