[Question] Old and new screenshot differences

[schmittl]

Just updated to the latest version and noticed that some of the new screenshots are rendered differently compared to the old ones from 2012.

Diffs can be viewed here https://github.com/rougier/freetype-gl/commit/09c90cd79d029c833ce7985d4d84e041d54800b0

For example here is a zoomed side-by-side comparison of subpixel.png (left is old and right is new version)

To me the old left version looks kind of like ClearType subpixel rendering.

I am just curious and confused - Is it just different filtering / changed implementation or an entirely different reason? Perhaps someone can help me understand and provide some insight about what or why it has changed over time, thanks in advance!

[rougier]

It looks like the right screenshot is not using LCD filtering (at the freetype level). How did you obtain the two screenshots, from the same freetype binary ?

[adrianbroher]

Both my local installation and the test environment have subpixel hinting disabled for patent reasons. This is not related to the implementation.

[rougier]

But the right screenshot looks like LCD was active. @schmittl can you confirm ?

[rougier]

Ok, just realized the diff you posted comes from github diff. So yes, the difference come from @adrianbroher (or maybe me) rendering without LCD subpixel. You should obtain the left screenshot on your machine if you have LCD subpixel filtering.

[schmittl]

Thanks for the clarifications!

Yes, both screenshots are from the github diff, sorry about the confusion.

But locally I can only reproduce the right version. Using freetype 2.7 and my ftoption.h contains

#define TT_CONFIG_OPTION_SUBPIXEL_HINTING  2

#define FT_CONFIG_OPTION_SUBPIXEL_RENDERING

Am I missing something else?

[rougier]

Could you post the image rendering the subpixel demo ?

[schmittl]

Here is the whole image, which to me looks like the right version from above

[rougier]

Can you try playing with https://github.com/rougier/freetype-gl/blob/master/texture-font.c#L463

(For example, you can replace FT_LCD_FILTER_LIGHT with FT_LCD_FILTER_LEGACY, see https://www.freetype.org/freetype2/docs/reference/ft2-lcd_filtering.html for other options)

[schmittl]

Hm, I tried all enum options for FT_Library_SetLcdFilter, but unfortunately they all render like the image above.

Setting the values of self->lcd_weights at https://github.com/rougier/freetype-gl/blob/master/texture-font.c#L228-L232 to non-sense values at least affects the output: Setting all to 0x00, makes the text disappear Setting all to 0xFF, makes the text appear bold

Using reasonable values like 0x00 0x55 0x56 0x55 0x00 or 0x08 0x4D 0x56 0x4D 0x08 or 0x10 0x40 0x70 0x40 0x10 all produce the same output again like above.

[schmittl]

Here is a zoomed image of the texture atlas, with unmodified code: (I used CodeXL to inspect the texture)

Which looks like subpixel rendered glyphs to me. So I am wondering why they are not rendered like the left image from the first post?

[rougier]

I think at some point I rewrote the blending in subpixel.c to be able to colorize the text even if it was specified as RGB. Maybe in the process I get rid of the LCD aliasing. Could you compare the display function in subpixel.c as you did for the picture ?

And by the way, how do you produce the diff link ?

[schmittl]

Could you compare the display function in subpixel.c as you did for the picture ?

Sorry can you please elaborate, I don't think I understand what you want me to do.

And by the way, how do you produce the diff link ?

This link? https://github.com/rougier/freetype-gl/blob/master/texture-font.c#L228-L232 I just edited the # part manually

[rougier]

Sorry, I was talking about this link

[schmittl]

Ah, just went to the commit section https://github.com/rougier/freetype-gl/commits/master and simply clicked the commit. The rest of the magic is done by github.

[schmittl]

Perhaps this commit is relevant https://github.com/rougier/freetype-gl/commit/d252d165df5e3e0442c01db58b00f4b84f7a27b2 If you scroll down to the text-buffer.c diff there are some commented lines regarding blending.

Edit: the blending was originally changed in https://github.com/rougier/freetype-gl/commit/87cbe450842da434e3aa573ac32f138a85c424da it seems to me, perhaps you remember why?

[rougier]

Yes, it was to allow for colorizing the font. When using LCD, the texture is RGB and it needs special processing (blending) to display it in another color. Maybe this is incompatible with LCD filtering.

[schmittl]

Ok, thank you very much for the explanation! So if I understand correctly, right now the color in the texture atlas is discarded when blending. Which means there is no subpixel rendering, but still subpixel positioning?

Since the subpixel demo is not using a colorized font, would you consider this a bug then?

[rougier]

Yes, it is a bug.

[schmittl]

I decided to investigate a bit and here is what I found. This commit https://github.com/rougier/freetype-gl/commit/1dc7fefa3884a3a33b9b6089150e967647ff0434 from 2011 initially added support for subpixel rendering.

Here is a diff showing the code changes compared to master (I simplified the blending a bit): https://github.com/schmittl/freetype-gl/commit/52beca9e41c569822ad3482217a26a398f7ea1bd

With this code you can successfully replicate the original subpixel screenshot.

Unfortunately I think that the subsequent changes made to blending caused it to become wrong. Let me try to explain..

Take a look at the old gamma demo screenshot from the screenshot diff link https://github.com/rougier/freetype-gl/commit/09c90cd79d029c833ce7985d4d84e041d54800b0:

Notice that white text on a black background on the top side shows working subpixel rendering. But black text on white background has no subpixel rendering! Because the blending is done wrong.

Another example is the old ansi demo screenshot:

You can see varying degrees of subpixel rendering depending on the color of the text. The white text on the blue background on the right side clearly has subpixel rendering, while the black text has no visible subpixel rendering and gray text is in between.

The following link derives and explains alpha blending for subpixel rendering, if you scroll down to section 6. Rendering http://fsrv.dyndns.org/mirrors/dmedia-tutorials-textrendering1/index.html

This is basically how blending is implemented in the commit mentioned in the beginning. The key takeaway is that text color must not affect the background while blending, or it will be wrong. The approach has the limitation though that the color for the text is set via glBlendColor, which makes it very inflexible compared to per vertex coloring.

Not sure if it is correct, but it works at least for the subpixel demo. To make it not break the other demos a new shader for the subpixel rendering would be needed I think.

A different solution I can think of would be to implement blending solely in the fragment shader which could use per vertex coloring for the text. But for correct blending that would require a fixed background, because you cannot sample the framebuffer from the fragment shader.

Perhaps there are other solutions and please correct me if I am wrong. I guess the upside to all this, that no one noticed it until now. :smile:

[rougier]

I now remember this post and this is what I used to (try to) implement the blending. I'm not sure there is a solution that would work for every situation and that's a bit annoying. I wonder how general text rendering (no OpenGL) copes with this.

[schmittl]

Yes, I would be interested in that as well.

Was just reading the freetype docs here https://www.freetype.org/freetype2/docs/reference/ft2-lcd_filtering.html and have another question about gamma correction.

Each of the 3 alpha values (subpixels) is independently used to blend one color channel. That is, red alpha blends the red channel of the text color with the red channel of the background pixel. The distribution of density values by the color-balanced filter assumes alpha blending is done in linear space; only then color artifacts cancel out.

In OpenGL blending happens after the fragment shader. My understanding is that for subpixel rendering the gamma correction would need to be applied after the fragment shader and after blending. This implies that fragment shader output needs to be in linear space for blending to work. Gamma correction would need to happen afterwards e.g. by post-processing or by using glEnable(GL_FRAMEBUFFER_SRGB);

Thoughts?

[rougier]

But what kind of control do you have on GL_FRAMEBUFFER_SGRB ?

[schmittl]

None. If I understand correctly you are stuck with whatever gamma value (2.2?) it uses.

The more flexible alternative I can think of is to use post-processing. The idea is to first render to a offscreen framebuffer object, which has a texture as big as the framebuffer dimensions attached. After rendering and blending to this offscreen framebuffer, you apply gamma correction to the whole texture attached to it with a simple fragment shader. And finally you can display the texture in the default framebuffer.

This way the blending will happen in linear space, but you can control the exact gamma value in the shader. Basically a two pass rendering if I am not mistaken. (I only read about framebuffer objects yesterday so my explanation may not the best)

Edit: And of course there may be yet other options to handle gamma correction, I was just wondering when it needs to be applied for subpixel rendering

[rougier]

Two pass rendering might be overkill in most application. I can understand if you're doing for example a text editor, but most of the time the current solution should be "good enough". However, it might be worth modifying the subpixel demo with a dedicated shader (and explanation of the problem in the source)

[schmittl]

I totally agree that the current solution is probably good enough. I currently need to research all this, so my main interest is to find the theoretical correct way and then figure out practical solutions for it.

Anyway thanks for the replies, really appreciated!

[schmittl]

FYI regarding blending, in core OpenGL 4.5 there is a feature new called texture_barrier (maybe also available via extensions). See https://www.opengl.org/wiki/Memory_Model#Texture_barrier

Here is the important part:

The third thing this functionality changes is that you are permitted to perform a single read/modify/write operation between a texel fetch and a framebuffer image under the following conditions:

Each fragment shader reads only from a single texel from the image being written to. Specifically, the texel that that particular fragment shader will write to. This is easily done via texelFetch(sampler, ivec2(gl_FragCoord.xy), 0)

This enables you to safely sample the target framebuffer and do the blending entirely in the fragment shader. It is perfect for doing correct blending with arbitrary backgrounds.

There are two drawback I can see - firstly if you have overlapping pixels in a single draw call, you will again get undefined behaviour. If this is a concern you could use two buffer objects to only store non adjacent glyphs and then call glTextureBarrier() between their draw calls.

The second drawback is that it requires modern OpenGL drivers. Unfortunately e.g. my intel driver is stuck on OpenGL 4.0, does not provide the extension and thus I cannot use this feature atm.

[yairchu]

For reference here's the issue that caused this change: https://code.google.com/archive/p/freetype-gl/issues/48

[yairchu]

I think that I found a way how to do proper subpixel rendering with OpenGL.

On my branch implementing it, it looks like this:

It uses two rendering passes. The first rendering pass is for occlusion via per-subpixel-alpha using the color channels (using glBlendFunc's multiplication blending ability), and the second rendering pass adds the text's color.

[rougier]

Do you have also screenshot in different text color ?

[yairchu]

(made by changing the colors in demos/subpixel.c)

[yairchu]

Also tweaked the atb-agg example in my branch so you can interactively play with foreground and background colors. Looks good here so far :)

[yairchu]

Now after I read the tutorial linked above by @schmittl - http://fsrv.dyndns.org/mirrors/dmedia-tutorials-textrendering1/index.html I have some insights. In 6.2 it describes exactly the two-pass solution that I found. And it also explains a better solution at 6.1 (with caveat that all text in vertex buffer must have the same color) which I now understand is what freetype-gl probably implemented - using GL_CONSTANT_COLOR blending with glBlendColor. Looks like at https://github.com/rougier/freetype-gl/commit/e4279a2e9ffae515273fb54cdefd055955bbf18d freetype-gl stopped using this method (last reference to GL_CONSTANT_COLOR was removed), I wonder why?

[rougier]

I don't remember either actually. I probably did the change after having realized LCD rendering could be done only using black ink but I'm not even sure.

[yairchu]

LCD rendering could be done only using black ink

@rougier can you elaborate what you mean? I don't understand..

[rougier]

Subpixel rendering is using a RGB texture and in the earlier versions it means if you encoded the texture in black, you could only display the text in black. At some point, I wanted to be able to use the subpixel rendering but using any color for displaying it. This when I started to investigate how to use a different blending mode.