Full Unicode support?

[GoogleCodeExporter]

-------------------------------------
vv.Text(a, u'VisVis - ВисВис')

Warning: Cannot draw character 1042!
Warning: Cannot draw character 1080!
Warning: Cannot draw character 1089!
Warning: Cannot draw character 1042!
Warning: Cannot draw character 1080!
Warning: Cannot draw character 1089!
-------------------------------------

Why partial unicode support? I mean utf-8.

IMHO this is also matplotlib big problem. User can't easily (or by no means) 
adopt text other then ASCII or math symbols.

I still use gnuplot when I need annotated text in my language, or text other 
then ASCII

Original issue reported on code.google.com by klo...@gmail.com on 27 Jun 2011 at 9:33

[GoogleCodeExporter]

The problem is that there is no way (at least not an easy one) to draw system 
fonts in OpenGL. What usually happens is that a package draws its own glyphs 
using a texture, or using lines. 

I can tell you that I spend a lot of time to get the current font system of 
visvis off the ground.

So visvis has a 1024x1024 texture in memory that contains the glyphs of a 
subset of the Unicode table. And a subset thereof is also available in bold. If 
we would support the full Unicode table, we would need an enormous texture! 

There might be other ways, such as drawing the text on an invisible canvas 
using the backend GUI toolkit, capturing that text as a texture and then draw 
that. This is not easy to get right in a cross-platform cross-backend setting.

If you really want this feature, I welcome you to try and implement it. We 
would be glad to have full Unicode support, and we would be glad to help you 
implement it in visvis!

Original comment by almar.klein@gmail.com on 27 Jun 2011 at 11:08

• Added labels: Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

Original comment by almar.klein@gmail.com on 27 Jun 2011 at 11:09

• Changed title: Full Unicode support?

[GoogleCodeExporter]

OK, thanks for your reply

I understand it's only one little aspect of your package offerings.
I forgot that there is no font setting so you must use textures

OTOH, thanks for making possible simple LaTeX style entering while not
requiring existence of LaTeX software. Great feature

About me implementing such colossal things, I'm afraid I don't have such
knowledge, I'm just scripter :)

Original comment by klo...@gmail.com on 27 Jun 2011 at 11:26

[GoogleCodeExporter]

> About me implementing such colossal things, I'm afraid I don't have such
knowledge, I'm just scripter :)

We are all just scripters :)  Don't let the size of a task fair you. It might 
not be an easy task, but it can be done (I think); just take it step by step.

Nevertheless, I understand if you're too busy or don't want to dedicate too 
much time for this. 

Original comment by almar.klein@gmail.com on 27 Jun 2011 at 8:45

[GoogleCodeExporter]

On Mon, 27 Jun 2011 20:45:48 +0000

I have no problem to try, if you really think that random outsider can
do it :)

I see that fonts file is in SSDF format (implemented by you as well as
SSDF Python parser)

There are 3 font types: mono, sans and serif, each with 14 arrays or
various sizes and types.

Now what to do with those arrays? :)

Original comment by klo...@gmail.com on 28 Jun 2011 at 10:18

[GoogleCodeExporter]

I mean, I expected creating additional table or even modifying current
one or similar, as doing OpenGL tricks I sure can't do that as I know
nothing about it

Original comment by klo...@gmail.com on 28 Jun 2011 at 10:25

[GoogleCodeExporter]

Well, only adding new glyphs won't help much. I mean, you could add the ones of 
your language, but then up comes someone else who want his characters included 
too.

Having full Unicode support would require a completely new approach. The 
easiest (as I think now) would be to let the backend draw the characters 
somehow.

> doing OpenGL tricks I sure can't do that as I know nothing about it

Ah, ok. Well then, I'll leave this issue open; maybe someone will have a go 
with it some day, or maybe I will do it myself when the time is right.

> I have no problem to try, if you really think that random outsider can do it 
:)

That happens a lot in open source. I would still check your code of course, and 
correct it if I find that necessary :)

Original comment by almar.klein@gmail.com on 28 Jun 2011 at 10:56

• Changed state: Accepted
• Added labels: Priority-Low
• Removed labels: Priority-Medium

[GoogleCodeExporter]

Yes, you are right. Recreating Unicode table as a whole or implementing
it in parts in bad idea to say at least.

OK, hope someone with more coding experiance jumps in here with same idea
(before you, maybe :)

Original comment by klo...@gmail.com on 28 Jun 2011 at 11:09

[GoogleCodeExporter]

I haven't really looked into this in depth, but I just saw that the PIL has an 
ImageFont module 
(http://www.pythonware.com/library/pil/handbook/imagefont.htm).  The 
documentation seems to imply that it supports unicode, though it isn't 
explicit.  Presumably we could render text to an image and then use the image 
as a texture.

If we're trying to get away from PIL, we might be able to do something similar 
with freetype-py (http://code.google.com/p/freetype-py/).  I note that the same 
author also has freetype-gl (http://code.google.com/p/freetype-gl/), which I 
think does what we want, but in C.  Maybe it's worth talking with him about 
developing freetype-gl-py.

Original comment by rschr...@gmail.com on 2 Jun 2012 at 3:10

[GoogleCodeExporter]

Yes, I think that in any case it comes down to have the glyphs drawn to a 
texture and then use that. In addition this results in much better looking 
text, especially for larger fonts.

I want to move away from PIL, yes. I'm porting visvis to Python3, and PIL is 
not really available in Py3k (on Windows there are some projects that have 
ported it though). For me this seemed like a good moment to get away from PIL. 
I've started a project called imageio (imageio.readthedocs.org) (based on code 
started by Zach Pincus) as a replacement. It will get most its functionality by 
wrapping the FreeImage library. If you're interested, you could join the 
discussions, we're kind of defining the API right now.

The freetype-py seems interesting, it requires an library, but that is 
something we can deal with (we have the same problem in imageio). I'm a bit 
less enthusiastic about freetype-gl, as its build in c, and would I suspect 
cost a lot more work.

The initial idea that I had in mind was to have the backend draw the font for 
us. I would start with QT (PySide and PyQt), and me or someone else could add 
functionality for the other backends, and we could leave the current 
implementation as a fallback.

However, having something that's independent of any backend (freetype-py?) 
could offer some significant advantages, also considering offline rendering etc.

Original comment by almar.klein@gmail.com on 2 Jun 2012 at 10:05

[GoogleCodeExporter]

It looks like freetype-py may be all we need.  Check out 
subpixel-positioning.py in the examples directory.  Note:
1. You probably need to change the font file passed TextureFont in line 212, 
unless you happen to have Arial sitting around in your current directory.
2. You can change the text in line 213 to a unicode string, and it Just Works.
3. You should comment out the last 3 lines of TextureFont.load() in 
texture_font.py.  Otherwise the character spacing gets wonky when you change 
the font size.

To use it as is, we'd need several files from the examples directory, in 
addition to the freetype module itself.  Given that it's released under the new 
BSD and it's rather small, it's likely easier to distribute freetype-py with 
visvis, rather than require it as a dependency.  We could add a test to see if 
FreeType exists on the system, and fall back to the current mechanism if it 
doesn't.

Original comment by rschr...@gmail.com on 5 Jun 2012 at 2:25

[GoogleCodeExporter]

This is on the top of my list of things to do for visvis. But I'm currently 
already too busy. I hope to make some time for this in the coming weeks.

Original comment by almar.klein@gmail.com on 8 Jun 2012 at 2:41

[GoogleCodeExporter]

I've gotten things mostly working with freetype for the text rendering.  Since 
I did it without being very careful, I've put it in another clone (and in a 
branch in that clone, because I'm still confused by mercurial): 
http://code.google.com/r/rschroll-freetype/source/list?name=freetype.  It's 
actually working better than I was expecting (see attached images), but you may 
want to rewrite it more carefully anyway.

The big issues that I'm aware of:
- The text rotation and letter spacing are not implemented.
- Other parts of the code access attributes of BaseText that I had assumed were 
private.  I've put little stubs in to prevent errors, but either these need to 
be added for real or access needs to be curtailed.  Right now, this includes 
_vertices1, _vertices2, and _color.  But I'm also not sure that _deltax and 
_deltay are what is expected.
- I'm using the shader class that came with freetype-py, since I wanted to 
minimize the number of possible breakage points.  But to avoid duplication, we 
should probably be using visvis's shader class instead.
- I'm just assuming freetype is available.  It'd probably be better to have a 
fallback if it's not.  This could be the current mechanism, or we might be able 
to pickle a few TextureFonts and a TextureAtlas with the main characters from a 
few fonts.
- I've written a basic font manager based around shelling out to Fontconfig's 
fc-match (since I couldn't find any pure-python bindings).  I don't know what 
to do if Fontconfig isn't available (or, for that matter, what we'd do 
instead).  Perhaps ship a few basic fonts, so we know where they are.
- The font manager is a global.  Previously, I think there was one attached to 
each figure.  I didn't have a good reason for changing it.

Once you get a chance to look it over and give it some thought, let me know how 
you'd like to proceed.

Original comment by rschr...@gmail.com on 11 Jun 2012 at 12:44

Attachments:

• freetype.png
• subpixel.png

[GoogleCodeExporter]

Wow, these screenshots look awesome... I'm impressed :)  and I want it! I'll 
check out your code and figure out a plan to get it in the visvis base repo. 
Thanks for this work!

Original comment by almar.klein@gmail.com on 11 Jun 2012 at 8:23

• Changed state: Started

[GoogleCodeExporter]

Thanks.  I can push it to a branch in the main repo, if you want to work 
on it there.  Or I could give you push access to my clone and we can 
work on that.  Just let me know which you prefer.

Original comment by rschr...@gmail.com on 11 Jun 2012 at 3:46

[GoogleCodeExporter]

I'll first have a proper look at the code. I probably want to design some base 
class(es) and subclass that for the freetype method and the old method as a 
fallback.

Can you give me a small overview of what your code does? I saw something about 
an atlas, and a shader to do subpixel stuff. I assume you use freetype to 
create a texture of some sort. A texture for each piece of text? And what 
happens if you dont use the subpixel shader (on OpenGL < 2)?

Thanks.

Original comment by almar.klein@gmail.com on 12 Jun 2012 at 9:44

[GoogleCodeExporter]

I can try, but most of the code was copied directly from freetype-py, so 
my understanding is limited.

In my mind, there are three levels of the code.  The lowest level is the 
freetype bindings, provided by freetype/__init__.py and freetype/ft*.  I 
have no idea what they're doing, and I hope I never have to find out.

In the middle level are freetype/shader.py and freetype/texture_font.py. 
  The former is a shader class, and can probably be replaced by the 
existing shader class in visvis.  The latter is more interesting, with 
three classes:  TextureAtlas keeps a texture onto which glyphs can be 
drawn.  As I understand it, you ask it for a region of a given size, and 
it finds a free region and lets you draw in it.  TextureFont uses an 
atlas (possibly shared) to draw glyphs on an as-needed basis.  Thus, we 
can have all of unicode available without needing it pre-rendered. 
TextureFont is queried for specific glyphs, which are returned as 
TextureGlyphs, which are essentially structs with releveant info.

Then, at the top level, BaseText uses these things to make a texture 
mapping to display the glyphs in the correct order.  (I think the basic 
idea is the same as before, though the implementation is different.) 
This is the only part that I modified, from the subpixel-positioning 
demo in freetype-py.  There is some trickiness going on with the shader 
to produce subpixel anti-aliasing.  First, inside TextureFont, Freetype 
is rendering in LCD mode, using the subpixels to produce better 
horizontal resolution.  Each glyph is assumed to be aligned with the 
pixel grid.  When the glyphs are assembled into the text, though, they 
are not forced onto the pixel grid.  The vertices are aligned with the 
pixels, but the offset is stored in a vertex attribute.  This is used in 
the shader to interpolate the drawn glyph.  This iterpolation works on 
the subpixel level, so it doesn't produce too much blurriness, as we're 
essentially working at 3x the pixel resolution.

All of this subpixel stuff is only happening in the horizontal 
direction.  Therefore, I snap Text objects to the pixel grid.  This 
isn't as easy to do with Labels, which have an external position.  If 
you try giving a Label a non-integer vertical position, you'll see 
significant blurriness.

This all produces beautiful looking text, so long as it's horizontal and 
black-and-white.  I haven't thought too much about what to do in other 
cases.  It looks like Freetype has the ability to render rotated text, 
so we could add rotated glyphs to the atlas.  But I wonder if it would 
be better to give up the crispness in this case, render textures without 
the subpixel coloring, and rotate the vertices appropriately. 
Alternatively, we could render whole strings at once when they are 
rotated, but not save them for future use.

I assume it breaks.  I don't know exactly how badly at the moment.  At 
the very least, you'll get funny letter spacing, since the glyphs will 
all be pixel-grid-aligned, and lots of colored fuzziness if the whole 
thing isn't aligned to the pixel grid.  But I don't think this is even 
checked for in the current code, so it could crash.

Original comment by rschr...@gmail.com on 13 Jun 2012 at 12:14

[GoogleCodeExporter]

The description is fairly accurate in fact. The subpixel shader is only needed 
if you want very precise pixel positioning.  Glyph is rendered on the pixel 
grid in the texture and it is displaced using the subpixel shader. It is nit 
really necessary but it is nicer visually. There are also some other "tricks" 
(like discarding vertical hinting) that can be found at 
http://www.antigrain.com/research/font_rasterization/.

As for rotated text, I did not try, but it would require to compute the cosine 
of  the rotation angle I would say.

Hope that helps.

Nicolas

Original comment by Nicolas.Rougier@gmail.com on 14 Jun 2012 at 4:51

[GoogleCodeExporter]

I dont fully understand (but I did not properly check the source yet). So 
here's a bunch of questions :)

So you use freetype to render to a texture, which has a 3x higher resolution in 
the x-direction? And in the shader you apply the subpixel stuff to fake higher 
resolution by using the individual colors?

And this subpixel stuff only works on LCD monitors, and only in the 
x-direction, right? So if we do rotated text, the extra resolution would be at 
an odd angle with respect to the text, making it probably useless.

The anti-aliasing is already done by freetype? 

@Nicolas: thanks for joining the discussion. How do you render text in glumpy? 
I saw some modules for text in the source code, but I did not see any freetype 
stuff, nor was it mentioned as a dependency.

  Almar

Original comment by almar.klein@gmail.com on 15 Jun 2012 at 9:03

[GoogleCodeExporter]

Glyphs are rendered either in an ALPHA texture or an RGB one. RGB is needed 
only if you want LCD subpixel rendering (freetype) and subpixel positioning and 
then the shader is needed.

The subpixel shader is in charge of moving a texture glyph (that is aligned on 
the pixel grid in the texture) to the proper place. If you do not  do that, 
then you would need a glyph for each fractional place of a letter. But this is 
not really necessary if you can live with sub-optimal positioning. Note that 
when you ask freetype to render a whole text, it will use subpixel positioning. 
The problem for me was then to have equivalent quality with only one glyph in 
the texture.

Subpixel stuff only work on LCD monitor and only in the X-direction, you're 
right. Anti-anlias is done by freetype and be discarded if needed (see 
freetype-py examples).

In glumpy, I'm using this library but I've not yet finished integrating it (and 
it does depends on freetype-py).

If you want to remove freetype dependencies, you might want to look at:
http://digestingduck.blogspot.fr/2009/08/font-stash.html

but this would require to translate the library from C to python.

Hope to see you at Euroscipy !
Nicolas

Original comment by Nicolas.Rougier@gmail.com on 15 Jun 2012 at 10:56

[GoogleCodeExporter]

Thanks. Given that we cannot really use subpixel rendering for rotated text, 
lets forget subpixel for now and see if we can get it into the equation for 
horizontal text later.

So freetype produces an ALPHA texture with an antialiased glyph. I assume this 
is at exactly the screen resolution? So to draw this correctly, you'd have to 
blit it to the screen one-to-one, right? This means we'd have to ask freetype 
for a new glyph at every change in position or rotation.

Where I'm going with this is that I would like to acquire a texture for a piece 
of text only once, and then be able to re-use it. In other words, I think it 
would be bad for performance if we would have to ask freetype to render the 
text for every frame-update if the user interacts with the figure. 

So maybe we can render the text at 2x or 3x the resolution (not anti-aliased) 
and do the anti-aliasing in the shader. Do you think we can obtain good quality 
with this? Or maybe freetype is so fast (if we ask it to render the whole text) 
that re-rendering on each frame does not matter?

Ah, you're going to Euroscipy? Would be nice to talk to you there! (But I hope 
to introduce this text rendering in visvis before then :))

  Almar

Original comment by almar.klein@gmail.com on 15 Jun 2012 at 11:44

[GoogleCodeExporter]

I've got an initial version of the FreeType font manager running in the trunk. 
You need to uncomment a line at the bottom of text/__init__.py to enable it. 

It seems there are still some problems with the sizing and alignment. Will look 
at it further next week. If you want to have a go at it, be my guest.

Original comment by almar.klein@gmail.com on 21 Jun 2012 at 10:01

[GoogleCodeExporter]

Ok, I've got the text rendering working with FreeType. Still need to write some 
code to detect whether the FreeType system is available and nicely fall back if 
it is not.

I completely ditched the LCD stuff, because it makes things a lot simpler and 
easier to generalize. Apart from that, LCD has its downsides, because you want 
to turn it off when the user does *not* have an LCD screen or for instance when 
a screenshot is made.

What I did now is that the text is actually rendered at a higher resolution 
(~2.5x right now) and I do a bit of antialiasing in a shader. This already 
looks pretty good. Not as crisp as your screenshot, but good enough IMO. If we 
do full screen antialiasing later, we dont need that anymore.

How strong do you feel about this subpixel LCD thing? If you like, we can try 
and think of a way to introduce it back for horizontal text. 

Original comment by almar.klein@gmail.com on 27 Jun 2012 at 8:56

Attachments:

• visvisTextScreen1.png

[GoogleCodeExporter]

Oh, and this one

Original comment by almar.klein@gmail.com on 27 Jun 2012 at 8:57

Attachments:

• visvisTextScreen3.png

[GoogleCodeExporter]

Looking good!  My only complaint is that the pipe has a varying darkness 
in the the subpixel positioning example.  But that's nitpicking.

While the LCD subpixel stuff is very pretty, I'm not all that attached 
to it.  I implemented it originally simply because that's what was in 
the freetype.py example, and I wanted to change as few things as 
possible in getting it working.

One thing to note is that the freetype font manager depends on 
fontconfig as well.  I think we can count on finding this on Linux 
systems, but I don't know about Mac or Windows.  It we keep things as 
is, we should add a check for the fc-match command before we choose the 
freetype text methods.

(Now that I'm looking at that part of the code: I just learned that 
subprocess.check_output() was a 2.7 addition.  If we want compatibility 
with previous versions, we should use subprocess.Popen().communicate().)

Original comment by rschr...@gmail.com on 28 Jun 2012 at 5:11

[GoogleCodeExporter]

About the varying darkness: I think this can be improved by tweaking the 
antialiasing parameters. However, I dont think that it can be completely 
solved, because its intrinsic to this not-aligning-to-the-grid approach.

About fontconfig: I want to ship with visvis a sans, serif and mono font (which 
should be free/libre of course). Where fontconfig is available, we can support 
any available font. Otherwise I dont think its a problem if the fonts are 
limited to what we ship with visvis.

I will soon do some testing on Windows to get all stuff and the fallback 
mechanism sorted out.

Original comment by almar.klein@gmail.com on 28 Jun 2012 at 8:22

[GoogleCodeExporter]

I've done some work to get it running on Windows. I've included the FreeSans 
font, which is a font that has *a lot* of unicode chars. Other free fonts that 
I tried often lacked math symbols, or had a too restrictive (copyleft) license. 

I also added some code to obtain a serif and mono font on Windows by simply 
checking if c:/windows/fonts/cour.ttf and times.ttf exist.

Good news on visibility: I made an improvement to the anti-aliasing; previously 
the mini-textures for the glyph were very tight around the character, leaving 
no space for neighboring pixels to join in the aa. You could see that in the 
vertical pipe for instance. The images now look prettier, and I think there's 
less difference in darkness; only sharp vs blurry. Also when the text is moved 
or scaled the text "flickers" much less.

Have to figure out how to get the FreeType lib for Windows users. Would be two 
times ~600kb (for 32/64 bit). I dont like the idea of shipping them with visvis 
so much. I could make an ino-installer to make installation easy. But I'd have 
to check how that works out with freezing an application and distributing it on 
a PC that does not have FreeType installed...

Original comment by almar.klein@gmail.com on 29 Jun 2012 at 10:34

[GoogleCodeExporter]

I've done some more work on this. For Windows users there is now an easy 
installer on the visvis website. And the FreeType renderer is also freeze-safe 
(i.e. you can freeze apps and distribute them to others without them requiring 
to have the FreeType lib installed.)

Original comment by almar.klein@gmail.com on 1 Aug 2012 at 10:12

[GoogleCodeExporter]

Original comment by almar.klein@gmail.com on 16 Aug 2012 at 12:20

• Changed state: Fixed
• Added labels: Milestone-Release1.7