SDF fonts: possible solution for downscaling

[yard]

Hi guys,

I've been recently playing around with rasterizing the fonts with msdf textures in playcanvas. SDF is well-known for its ability to handle upscaling well, but performs poorly when it comes to downscaled shapes. There are solutions using supersampling of various sorts, but they don't seem to be quite universal and require a lot of adjustments.

An idea: why don't we use alpha channel to store original glyph's pixel opacity and fall back to it when font size is lower than the char size in msdf map? Bitmap fonts look pretty good when chars are rendered at the scale of 1 and below (and thanks to bilinear filtering we even get a poor man's AA).

Looks like alpha channel is not used in msdf calculations in any way, so we can use it to render glyphs as if we had a bitmap font. The threshold can be controlled by material's parameter "enabling" the alpha channel to provide pixel opacity for low font sizes.

P.S. Could you please hint me which tool is supposed to be used to create JSON and PNG files for Playcanvas fonts?

[willeastcott]

@vkalpias or @daredevildave know which tool is used to generate the font files. Guys?

[daredevildave]

Interesting idea about storing the bitmap version in the alpha channel. My original thought was that we'd offer two types of font asset, MSDF and Bitmap. If you needed a small font size you could generate a bitmap font of the size you want. I haven't got around to trying it yet. For MSDF fonts we generate them at 32x32 px. If you think anything smaller than that would work OK scaled down then it's worth trying out.

The tool we use for generating is a modified version of msdfgen. Our branch has fixes for generating glyphs of consistent scale and then we do atlas and json generation in the backend pipeline.

[yard]

Thank you for the hints. Let me try the fallback approach - I will post the results shortly and we can decide if it's worth mainlining.

Btw I've recently encountered this article - http://wdobbie.com/post/gpu-text-rendering-with-vector-textures/. The demo looks particulary impressive; I was very surprised such a sophisticated shader actually does perform quite well in terms of performance. Would be curious to hear your thoughts on this one :)

[yard]

Hi guys,

I've been playing with various tweaks to fonts to make it look better in smaller sizes.

One thing I noticed is that example SDF texture provided with playcanvas engine has non-uniform scaling on chars. While I appreciate smaller symbols (like + or *) might benefit from having more pixels to store SDF values in, it looks like non-uniform scale actually damages text lines as different letters (say, "i" and "A") get different "precision" and look blurry because of that.

I have tried exporting font glyphs at the same scale and noticed an immediate improvement in text quality (not ideal still, but it's WIP anyway) – do you guys have any specific motivation for non-uniformly scaled glyphs?

[daredevildave]

I don't think there was any reason beyond getting more precision on the smaller characters.

[yard]

Ok, so there seem to be 3 parts preventing glyphs from rendering properly: 1) SDF map with non-uniform scale of chars – easy to fix, but need to check the effects on smaller chars 2) Sub-pixel SDF bitmap placement – easy to fix as well, doesn't seem to affect anything 3) Sub-pixel glyph quad placement – trying to work it out in the vertex shader right now. This makes the text to move in "steps" upon geometry/canvas resize, but makes small text look much much better.

I will post my results for your review here shortly.

[yard]

Ok guys, he's the progress I've managed to made.

Changes and notes 1) Set the uniform scaling of glyphs upon MSDF export 2) Made translation of glyphs be pixel-perfect 3) Added bitmap version of the font as alpha channel, so that MSDF texture contains both signed-distance field and bitmap glyphs. 4) Texture has mipmapping enabled with linear filtering.

The results MSDF shader: https://www.dropbox.com/s/uifvxi9gkw6wqw4/sdfs.png?dl=1 Bitmap shader: https://www.dropbox.com/s/k4w57zhgz26shwf/bitmaps.png?dl=1

Conclusion Bitmap fonts look more readable at smaller sizes, while SDF shine at bigger scale.

Current concerns 1) Something still doesn't look correct to me. The bitmap font looks slighly too blurry, but that's because it's almost impossible to have 1:1 correspondence between texels and screen pixels. Maybe that's fine (smaller sizes will have a lot of aliasing anyway) 2) SDF could've looked better – just checking if alpha channel damaged SDF data in any way. 3) As far as I know Safari and some versions of IE premultiply alpha upon loading textures with transparency, which could break SDF data. Do you guys know if this is still true and, if so, if there are any workarounds for this? 4) I feel there is an easy way to switch rasterization mode in shader so that texts of big sizes, but placed far away from camera would use bitmaps rather then sdf eliminating the need to hint that through code. Apart from using derivatives, if there any way to do so?

Looking forward to hearing your thoughts.

[daredevildave]

In your bitmap screenshot are you generating a bitmap for each size or is that one size scaled up (or down)?

[yard]

Hi @daredevildave, That's the same size (being scaled and down) residing in alpha channel of MSDF texture. The texture thus looks like https://www.dropbox.com/s/cqnezaws3hz0lwq/Arial-MSDFA.png?dl=0 (previously it was https://www.dropbox.com/s/o1klbecob1ydu15/Arial-MSDF.png?dl=0)

[daredevildave]

Interesting, my feeling is that bitmaps look better in all cases there. What the maximum font size 2x scale up?

[yard]

Well, you are almost right: the MSDF texture has 32x32 glyphs, making only the last 2 lines benefit from SDF - but that's clearly visible if you look closer at the glyphs: with SDF char edges don't become blurry, while bitmap ones do (just make sure you are looking at 100% zoom in both cases). I can prepare a test with even bigger glyphs, but nothing new is hapenning here and the results are pretty much expected; the point is visual outlook for smaller font sizes.

Not sure I understand your question, specifially "what the maximum font size 2x scale up"?

[daredevildave]

Sorry. The question is: in the bitmap image, what is the maximum scale? 2x?

The reason I ask is that it looks pretty good even at the maximum. And I wonder how much bigger that is that the original.

[yard]

Not really, the max font size is 45x45 (using 32x32 char atlas, giving 1.4 scale ratio) and to my taste this is already slightly too blurry (check out https://www.dropbox.com/s/70uz5udnu1x67i5/bmp-vs-sdf.png?dl=0)

So, two questions that are in the way of getting this working properly:

1. Is texture premultiplication still the issue in Safari? Can it be worked around?
2. Do you have any ideas of how to detect texel-to-pixel ratio in the shader to be able to switch rendering mode automatically (to work around cases when 100px char is placed 100m away from camera and takes 10px of real screen estate)?

[daredevildave]

I see. There's definitely issues with both of them. I think the problem with the SDF one is it seems uneven. I want to play more with the "range" option in the export to see how that affects the results.

1. I'm not sure about this. One for @guycalledfrank.
2. I don't. I'd be a little cautious about automatically switching for things like this. Usually the developer will know if the text is going to be big or small and they can choose, bitmap or sdf.

[yard]

Well, SDF is not a silver bullet for fonts, neither bitmap are. I still feel the approach suggested by http://wdobbie.com/post/gpu-text-rendering-with-vector-textures/ is way more suitable for fonts.

As for #2, maybe there should be choice of 3 then? Like "use sdf", "use bitmap" or "auto"? However, I am not sure if there is any way other than derivates to make auto work...

[guycalledfrank]

About texel-to-pixel ratio: this should be doable with dFdx/dFdy. Basically if you have pixel coordinates, these functions will give you the rate of change per screen pixel (how many texture pixels are there per screen pixel). It requires OES_standard_derivatives, but this one is almost universally supported.

Safari and IE suck at PNGs and not planning to stop. Solution: use DDS (or any other non-web format that you can convert to a simple color array).

[daredevildave]

The MSDF shader already requires OES_standard_derivatives so that shouldn't be an issue.

[yard]

@daredevildave yeah it's just that SDF shader works without derivates as well – and "auto" matching obviously won't. However, it seems ok to just wrap this into a define (just like sampling does).

So, do you guys want me to prepare a pull request with the things discussed above?

[daredevildave]

What does a PR for this involve? Some outstanding questions from me:

• How is the font image with bitmap alpha generated? This would be part of the tool chain and not part of the engine.

• My original thought was we would support 2 types of font asset: sdf and bitmap. Bitmap fonts would need to be generated at specific pixel sizes. Do we need this fallback if you can just create a bitmap only font?

• If the above is yes. What does the transition between SDF and bitmap look like. Does it flick noticeable to a different type?

[yard]

Hi,

The PR basically deals with the following: 1) Amends MSDF shader to take alpha value from alpha channel based on fwidth value of texture sampler (if it's less than X it switches to bitmap font) 2) The text element gets a flag indicating whether to sample font texture as msdf, as a bitmap or decide automatically (based on point 1)

Answering your questions: 1) I was just rasterising glyphs from a given TTF using ImageMagick and setting the font size to be the same as for msdfgen (so that glyphs "match" each other) and them combining the images using ImageMagick's output as alpha channel and msdfgen's one as color channels 2) The fallback is purely for convenience purposes to make fonts look OK in most situations. Surely enough, achieving perfect look might force you to go bitmap-only path, but that's why point 2 in the PR allows you to force bitmap rendering; just generate a bitmap font, set this flag up and enjoy. If you don't want to deal with it and/or want the same font to look good across multiple scales, the auto will serve the best 3) The transition is of course visible, but I don't think it should ever "flick" (unless you are animating elements / resizing a browser window and the size of the glyph changes around critical point – but that can be worked around by either forcing a certain rendering type or amending the animations to deal with slightly bigger / smaller font sizes).

[daredevildave]

OK, makes sense. We can implement the tool-side separately. Sounds like we should do it.

Should the flag go on the text element or on the asset? Or both?

[daredevildave]

Also it would make sense if you could display a full color bitmap font using this too

[yard]

Hey,

As for full-color bitmap – that's a setting for asset for sure (setting this up as a uniform to the shader would make it super easy) – as this doesn't change at "runtime" (here I am referring to screen scaling, moving stuff around etc).

As for the sampling setting – I bet it's better to have in the text element AND in the asset. Basically, you can set the asset up to use bitmap, or MSDF or auto, but for some specific elements fall back to bitmaps for instance – and in this case you definitely don't want to create a separate asset. So, assets holding the default plus elements overriding it (if set) makes the most sense to me.

[willeastcott]

I'm just gonna leave this here:

https://astiopin.github.io/webgl_fonts/