Dynamic font support

[geniuszxy]

I mean, at runtime, load a font file, read some glyhps and render them into a texture. then use it like a normal SpriteFont. This feature is very important for asian users. because we use about 2000 common characters and a large amount of rare characters. Put then into a xnb file is almost impossible (takes very long time to build and gets very large xnb file). I have searched on the internet but no usable solution yet.

[MichaelDePiazzi]

I recently implemented something like this in my game, Final Days. What I did was a bit hacky, but it works. Unfortunately, I can't submit what I've done to MonoGame as it's very much a bespoke solution for my game.

I did a fair bit of research into this to try and find a good solution. But one of my requirements was to have font fallback (i.e. if the font you're using doesn't have the glyph you need, then it falls back to another similar font which does have that glyph). Unfortunately, this is easier said than done. The most practical solution I came up with was to use Windows GDI+ to render the glyphs. But this has it's own issues and I definitely came across a few quirks and inconsistencies. It obviously also only works with Windows.

But once the glyph rendering is taken care of, the rest was fairly straightforward (although it was a bit of work). Basically whenever I needed to display missing glyphs, I'd render those glyphs in the background and update the SpriteFont to include them.

If you'd like more details on how I did this, then I'd be happy to share them and try answer any questions.

[tomspilman]

It obviously also only works with Windows.

That is the biggest issue. We need an portable font rasterizer we can call at runtime. Platform specific ones will not work as they all produce different results... meaning your fonts look different on different platforms. It has to be non-GPL so we can use it on consoles as well.

Once we have a solution for that the rest is all pretty easy.

[MichaelDePiazzi]

That is the biggest issue. We need an portable font rasterizer we can call at runtime. Platform specific ones will not work as they all produce different results... meaning your fonts look different on different platforms. It has to be non-GPL so we can use it on consoles as well.

All I came across was Cairo. There's a C# wrapper for it called CairoSharp. I don't know if it would be suitable for MonoGame though. The licence is LGPL, but apparently it can also be used under the Mozilla Public Licence (MPL). Not sure if that helps? CairoSharp seems to only have LGPL though.

[harry-cpp]

Cairo is licensed under LGPL or MPL, CairoSharp on the other hand is LGPL only. I don't think its a good idea to use a native library for this purpose, its way better if something like Typography could fit our purposes.

On a side note, I'm maintaining a fork of gtk-sharp here, which also includes CairoSharp which uses delegate invoking of functions instead of dllmap so it should be better than that fork, tho it probably misses a few functions compared to it.

[mrhelmut]

Typography looks super cool for being a pure C# solution. Cairo does more than just font rasterizing, and I would be more favorable to a lib that does just what we need here.

I would be pretty curious about Typography performances.

There's a desperate need for a runtime glyph rasterizer in MonoGame. Localizing is such a pain (we ended up creating bitmap font hacks that use only the useful glyphs, but it can get pretty quickly unmanageable with higher word count).

I hear a lot about bmfont from other gamedevs: http://www.angelcode.com/products/bmfont/ I haven't looked into it, but it's open source and maybe the rasterizer can be used separately.

stb_truetype is also worth mentioning but it is known to be super slow.

Also, +100 to anything supporting emojis.

[MichaelDePiazzi]

I did come across Typography in the past and it definitely looked promising. But it didn't seem to do font fallback, so I didn't end up going further with it.

I only brought up Cairo because it was the only cross-platform open source library I came across which seemed to handle font fallback.

But font fallback was just something I specifically needed for my requirements. I'm not sure if it would be required for MonoGame as well. But it was definitely very handy not having to worry about glyphs that were missing from the font you were using.

If you don't need font fallback though, it makes things a LOT easier and there are a few options about (e.g. SharpFont which MonoGame already uses in the Content Pipeline).

[MichaelDePiazzi]

Also, +100 to anything supporting emojis.

Emoji support is tricky as SpriteFont only supports UCS-2 characters (i.e. Unicode U+0000 to U+FFFF), and emojis are outside of this range. I did experiment with a workaround which remapped Unicode characters outside of this range into the private use area (U+E000 to U+F8FF). It worked pretty well, but unfortunately the GDI+ renderer I ended up using only supported UCS-2 as well 😒

[tomspilman]

But font fallback was just something I specifically needed for my requirements. I'm not sure if it would be required for MonoGame as well. But it was definitely very handy not having to worry about glyphs that were missing from the font you were using.

We could do font fallback by specifying multiple fonts like you do in CSS. Then worst case we automatically pick a system font if all else fails.

[tomspilman]

e.g. SharpFont which MonoGame already uses in the Content Pipeline).

We could use SharpFont/FreeType at runtime, but i would love to try some fully managed libraries first. And if we find a good runtime font solution we like we would convert the pipeline to use it for offline rendering as well.

[harry-cpp]

@tomspilman As I mentioned above, https://github.com/LayoutFarm/Typography might be a good solution.

[MichaelDePiazzi]

We could do font fallback by specifying multiple fonts like you do in CSS. Then worst case we automatically pick a system font if all else fails.

For sure. This was something I was seriously considering as well, but shied away from because it meant including multiple font files. But ultimately this solution does give you more control over how glyphs are rendered. And it may also be the only choice on some platforms which may not even provide a font fallback mechanism.

[Jjagg]

As another managed alternative, there's SixLabors/Fonts.

[harry-cpp]

As another managed alternative, there's SixLabors/Fonts.

Nice!

[ghost]

Someone in this thread mentioned Cairo for run-time font rendering. I can vouch for it. I used to use it in my game back when user-interface skinning was a thing and we needed user-selected fonts that could be rendered at runtime. If you write the actual rendering code in C++ - i.e, create a function for measuring text and one for rendering to a bitmap in memory - and P/invoke into that middle-man C++ library, it's very fast.

The issues are that you have to compile that C++ middle-man for all the different platforms you need. And you need to make sure that Cairo (and Pango which we also used) are supported on those platforms.

But if you jump over that hurdle, dynamic SpriteFont loading directly from TrueType font files or even font families installed to the system becomes real easy (The latter at least, not entirely sure about the former).

[stromkos]

I have a working Android version of font replacement, redraw fonts within current bounds, to replace standard chars with color emojis. It adds about 5 seconds to the load time. I am working on a generalized version of this, but you can not mix color fonts with regular fonts. So far this solution will only work for Android and Windows platforms. If someone is willing to work on the ios/other platforms specific code, we could develop a standardized API for runtime fonts.

[rds1983]

Another solution would be to use StbSharp/StbTrueType. Right now, it's MonoGame Sample demonstrates how to dynamically create a SpriteFont: https://github.com/rds1983/StbSharp/blob/c3220a9dae71fd3491d7b5bf8bfe0d1f82f08ea8/Samples/StbSharp.MonoGame.Test/Game1.cs#L44

[mrhelmut]

That's very interesting! StbSharp being a pure C# library without any native dependencies, it's a very good candidate for all platforms.

Your example is quite cool to bake fonts at runtime given a character set. I might very well switch to StbSharp for our own projects.

I have been playing around with Typography, SixLabors/Font, and Cairo lately, and they all are cumbersome to use and with other native dependencies (and most of them use System.Drawing, which is a pain cross-platform wise).

So far, I'm all in for StbSharp.

[rds1983]

It has downside - the font baker API(which is wrapper over stb_truetype) is very simple: https://github.com/rds1983/StbSharp/blob/master/StbSharp/FontBaker.cs I've made it just to check whether the thing works.

[mrhelmut]

I'm going to give it a try, checking performance and font compatibility.

[harry-cpp]

There is one more downside, its unusable on the web port of MonoGame.

[mrhelmut]

Because of reflection and/or unsafe code?

[rds1983]

btw, there's another stb_truetype port to C# made by @KonajuGames: https://github.com/KonajuGames/TrueTypeSharp/tree/master/TrueTypeSharp I haven't reviewed it throughfully. But from the glance it doesnt have unsafe code.

[harry-cpp]

Reflection works, unsafe code does not.

On a related note I finally managed to fix up content manager for web, its now as fast as other platforms.

[mrhelmut]

@KonajuGames 's port / the original TrueTypeSharp doesn't have unsafe code, but is missing 4 years of improvements. I wonder if an updated version would be too much work.

It would be cool to have this working on the web and on consoles.

[mrhelmut]

I've been playing around with the Stb.

Konaju's port has some issues and the original TrueTypeSharp is outdated (and using a less clean rasterizer). Bringing it on par with the latest Stb_truetype sounds like a lot of work since parts of the library went over redesign / refactoring.

I started to try removing the unsafe parts from StbSharp, which sounds very doable but is a lot of work to make sure that everything works as expected.

@rds1983 Do you think it would be possible to add an option to Sichem to produce safe code? The FakePtr from TrueTypeSharp looks like a solution (and is of course very garbage prone, unless it would manage its own memory pool).

[rds1983]

@mrhelmut

Do you think it would be possible to add an option to Sichem to produce safe code? The FakePtr from TrueTypeSharp looks like a solution (and is of course very garbage prone, unless it would manage its own memory pool).

That's awesome idea! I'll definitely give it a try.

[rds1983]

I've added the option to Sichem to produce safe code and tested it out on stb_truetype. And it worked! New safe version of StbSharp is here: https://github.com/rds1983/StbSharp/tree/master/StbSharpSafe I've also added new MonoGame sample that works with StbSharpSafe: https://github.com/rds1983/StbSharp/tree/master/Samples/StbSharpSafe.MonoGame.Test

[mrhelmut]

Wow, that was fast! Thanks.

So far, what I can say about all the options:

• freetype: still the best rasterizer around, but it's a native library (which works well on consoles, though) :+1:
• Typography: incomplete and doesn't support all fonts without crashing with NotImplementedException (but does a pretty fine job when it works) :-1:
• SixLabors/Fonts: doesn't support 32bit characters (e.g. most CJK characters, which makes it pretty useless for dynamic fonts) :-1:
• StbSharpSafe: works very well, but doesn't support hinting (which makes small fonts blurry) :+1:

So I'd say it's a tie between freetype and Stb_truetype... they both have advantages and inconveniences.

[Jjagg]

@mrhelmut I got a PR merged in SixLabors fonts to change the internal character representation to int for full unicode support. I planned to do some more work on it because IMO it looked like the best pure C# solution in terms of planned features and code quality (ImageSharp is awesome). They're just missing importers for the 32-bit formats to support CJK. Still lots of other stuff to do though e.g. color emoji and RTL (last time I checked at least).

[mrhelmut]

@Jjagg Good to know! Might become a relevant alternative at some point!

[willmotil]

Is the goal to be able to load system fonts for missing unicode characters during runtime ? Or rendering like to render emojis and stuff i keep seeing rendering all thru out this page.

Like are you guys talking about replacing spritefont or drawstring ? This is just for loading right ?

Spritefont already can hold as many characters as a texture will fit right. I mean i have 1024 characters of a 65k character font ttf loaded now using nothing but spritefont. It would be nice if i could splice in emojis to it but that would require a second file or something like splicing them in.

But spritebatch drawstring still cant read them.

No one ever changed drawstring to make it read code point pairs or we would already have emoji support as is. It's not even a big fix really.

In case anyone doesn't get what i mean ,,,, in drawstring we need to check for code point pairs. Right now it can only ever read up to u+ffff; which is only 16 bits because of this line. var c = text[i]; Right now that would read anything over u+ffff as garbage.

Something like what the below method shows. basically it needs to check for pairs as it goes and read them as codepoints i suppose then it could just look up emojis as is right now in drawstring, The for loop has to be altered though and i think spritefont has to keep strings instead instead of chars for the gylph dictionary key or just cast everything to utf32 but i don't think changing that would actually be a breaking change.

    public string ConvertSurrogatePairsToCodePoints(string text)
    {
        // A emoji can be represented in 2 different forms
        //    Codepoint
        //    String
        // For Example : SMILING_FACE_WITH_SUN_GLASS Emoji
        //    Codepoint: 0x1f60e 
        //    String: 😎 (which is \uD83D\uDE0E) // <<<< which no one ever changed to make monogame read it right.

        // when drawstring hits a line with a emoji its going to instead read it as uD83D

        // Do this in a loop.
        // What we are actually looping now is character objects.
        // What we need to do is read codepoints which are variable length.
        string result = "";
        for (int i = 0; i < text.Length; i++)
        {
            char A = text[i];
            if (i < text.Length - 1)
            {
                char B = text[i + 1];
                if(Char.IsSurrogatePair(A, B)) // <<< this was never added to drawstring.
                {
                    int val = Char.ConvertToUtf32(A, B);
                    result += "  Pair " + ((short)(A)).ToString("X4") + " " + ((short)(B)).ToString("X4") + " = " + val.ToString("x8");
                    i += 1;
                }
                else
                {
                    result += "  Single " + ((short)(A)).ToString("X4") +" = "+ ((int)(A)).ToString("x8");
                }
            }
            else
            {
                result += "  Single " + ((short)(A)).ToString("X4") + " = " + ((int)(A)).ToString("x8");
            }
        }
        return result;
        // really thats just a character that has to be checked for in drawstring that consists of a codepoint pair so it can be looked up in the spritefonts glyph dictionary.
    }

You can test that by just calling the below line on that method above.

string resultsUtf32 = ConvertSurrogatePairsToCodePoints("😎");

The string holds the chars as pairs so the current drawstring will never ever read them right till we start checking the strings passed in for pairs.

Monogame should probably have a method to for the user that will convert utf8 strings to 16 as well.

[mrhelmut]

I guess that what we would be cool about dynamic fonts, is to be able to bake a SpriteFont at runtime without the need of the content pipeline.

When you are localizing games in CJK languages, it's very impractical to generate a SpriteFont with 22k+ characters. So the idea is to be able to bake a font at runtime by feeding it only the characters that are actually used by the game. You can do that with the pipeline by defining ranges, but being able to that at runtime allows to modify the texts without the need of the content pipeline (and therefore being able to share builds with translators or players and let them test a preview of their translations without having to rely on development tools to bake a font).

The icing on the cake would be to limit the bitmap font resolution to a precise size (for hardware compatibility purpose) and to manage multiple textures/pages if the glyphs can't fit.

[willmotil]

I see ... so this is about run time loading basically. Would this be loading system fonts then to add glyphs or just parsing out needed glyphs to add to a running font. It almost sound like the new font would be saved at runtime as well. How do you know what glyphs you need when say the user is typing in a chat box ? How is that sort of situation handled ?

[mrhelmut]

It's basically reading a ttf file shipped with the game, generating a Texture2D of the glyphs and having all the kerning and glyphs info to manually instantiate a SpriteFont at runtime. No need to have it saved on disk.

For text input, that's another issue, and would likely require to rasterize a full range of characters. (Being able to expand the bitmap font as you go would be awesome, but most probably inefficient.)

[Jjagg]

[...] manually instantiate a SpriteFont at runtime

Rather than (or as an alternative to) creating a SpriteFont at runtime we could use the loaded font data to actually do proper layout. SpriteFont simplifies text layout in favor of reduced memory usage and slightly increased performance.

(Being able to expand the bitmap font as you go would be awesome, but most probably inefficient.)

I think it can be done efficiently if the bin packer used for the texture atlas is kept in memory and the texture for the atlas is large enough so it doesn't need to be resized (much). Actual rasterization is probably the slowest part of adding a character and that's not slower than if you'd add a range of characters all at once. Uploading to the GPU will be slower when uploading separate characters though.

[willmotil]

So basically loading a tff instead of a spritefont then building a new spritefont ?

Im still a bit hazy on this.

What is the major advantage or difference to just loading a couple pre-built spritefonts now and making a new one from them as things are ?

I mean its kina annoying to do it manually atm as isn't built in, but there is setdata for the texture and you can pull out the glyphs and rebuild them as it is. http://community.monogame.net/t/font-spacing-issue/11030/4

You would still have to load the ttf to build from it or to use it directly same as loading a spritefont right ?

[mrhelmut]

Dynamic fonts are cool when you don't want to have to rebuild your game to test (or live edit) texts within your game when working with too huge languages. It is very useful when localizing a game or working with writers (and keeping the bitmap fonts small).

For instance, we are translating our games to Chinese, Japanese, and Korean languages, which can go up to 80k characters if you want to be generic enough to allow translators to have an editable build. It is very impractical to generate a bitmap font of 80k characters, the texture would be too huge and would crash most hardware. So what we do, is to generate fonts with only the characters that are used in the game (which helps to go down to 1-2k characters). The Pipeline allows this, but it is done at build time, and therefore prevent translators to edit their texts and have an in-game preview within us having to rebuild the game.

We could provide a baking tool that embed the content pipeline, but that gets complicated.

(What we actually do, is using BMFont as a CLI tool to re-generate fonts at game startup, which means that we have our own BMFont parser and renderer. BMFont is really cool, it can manage pages and does a very good job at packing glyphs. The problem is that it is an external .exe, and that's touchy to ship on most platforms.)

[willmotil]

So what we do, is to generate fonts with only the characters that are used in the game (which helps to go down to 1-2k characters). The Pipeline allows this, but it is done at build time, and therefore prevent translators to edit their texts and have an in-game preview within us having to rebuild the game.

How does the pipeline tool we use now decode a ttf to make a spritefont do we have a actual algorthim that does the decoding or are we calling some external function ? Can that code be separated and moved into a new framework class file that uses file io ?

Its already possible to edit or create a sprite font at runtime as shown previously.

[mrhelmut]

The pipeline uses freetype, which is a native dependency. It is perfectly possible to use freetype at runtime to make dynamic fonts.

The idea to have a pure C# rasterizer is cool if we want to make sure that it works on every platforms (inc. consoles) without an additional dependency to ship/manage.

[MichaelDePiazzi]

(Being able to expand the bitmap font as you go would be awesome, but most probably inefficient.)

I think it can be done efficiently if the bin packer used for the texture atlas is kept in memory and the texture for the atlas is large enough so it doesn't need to be resized (much). Actual rasterization is probably the slowest part of adding a character and that's not slower than if you'd add a range of characters all at once. Uploading to the GPU will be slower when uploading separate characters though.

Yes it can be done efficiently, and this is pretty much what I am doing in my game to achieve this.

Also, to minimise any delays on the main game thread, I am doing it all on a separate thread. The side effect of this is that missing glyphs briefly appear as question marks (or whatever you want the missing glyph char to be) while they are rendered.

I'm using it to do things like properly render players Steam handles in game (e.g. for leaderboards), and it works quite well. I'm also hoping to eventually use this to support user translations.

[stromkos]

@willmotil

No one ever changed drawstring to make it read code point pairs or we would already have emoji support as is. It's not even a big fix really.

I have already created a prototype in pull #6366. This changes the format of the xnb file making it not backwards compatible.

The main problem with decoding is not UTF-32 but the proper application of Decorators( Sequences of UTF codepoints describing a single glyph.)

The Glyph dictionary has a key of type Char which would be changed to an int to store UTF-32. The major rub is the XML Reader to load the spritefont. It produces an error when reading Extended Characters. This is why I created the new type.

[JoaoBaptMG]

There is a (I think) very good implementation of an online bin packer on SFML's source code: https://github.com/SFML/SFML/blob/master/src/SFML/Graphics/Font.cpp

[mrhelmut]

It uses FreeType, which seems to be the most relevant rasterizer for now.

[rds1983]

I've made a library that allows dynamic creation of SpriteFont: https://github.com/rds1983/SpriteFontPlus @geniuszxy

[willmotil]

The StbSharp dynamic font above i think is made by the same guy as the thing we use now not 100% on that.

Anyways. I added unicode support to the above the project and it just uses a renderer i tossed together to test it.

https://drive.google.com/open?id=19eVSczTid9oqOLjK3LyAyAPn7wsur9mn

Drawstring getting unicode support amounts to a assignment to a int and a single if check the vast majority of the entire loop cycle, which is a negligible performance hit. Basically this is very little if any penalty.

        for (int i = 0; i < text.Length; i++)
        {
            char cs = text[i];
            int charval = cs;  // (1)
            if (char.IsHighSurrogate(cs))   // (2)  be nice to implement this in place we could check manually for the hex code range.
            {
                // essentially we get to skip a character next pass so this is extremely minor as well.
                charval = char.ConvertToUtf32(cs, text[i + 1]); 
                i++;
            }

            // there is really no other downside after the above performance wise.
            if (charval == '\r')
                continue;
            if (charval == '\n')
            {

Ok so jjagg has a wip for unicode support which is like as far as i can tell is just about finished. So i just wanted to point out that it appears getting full unicode support in monogame is approaching imminently.
This issue goes back a long way and the original preliminary discussion is here #4805

I suppose it would be nice to get both at once dynamic font support and proper unicode capability. This route seems to do that.

It also just from initial tests seems to get rid of the horrible looking alpha pixels and bad aliasing (that occurs differently at different point sizes) that the current font loader seems to generate into the texture.

5242 #5272 #6614 all still open and more.

It looks amazingly good under spritebatch i don't have to tinker with settings either to try to clear it up. Not shown in the above image but it is when you run the project.

The above image is of course just a quick hacked in font rendering class right out of a test i was working on for something else at the moment. Its not a very good renderer but it shows the results of StbSharp with unicode support.

[mrhelmut]

I'd like to note that StbSharp is very useful (thanks @rds1983 !), but doesn't support hinting, which makes smaller fonts very blurry compared to what freetype is currently outputing through the content pipeline.

This would mean that dynamic fonts would render differently from standard SpriteFont. We'll have to keep that in mind when moving forward.

[willmotil]

Freetype's output is quite frankly horrible even on small sizes... http://community.monogame.net/t/spritefont-rendering-cant-seem-to-disable-smoothing/11225/9

Look at the J specifically. Unseen really here but the zero was turned into a e

stb has some problems so far but freetype has been in use for a long while and it still hasn't been fixable.

StbSharp small text 15 pixel height which is like an 9 point spritefont. Scaled way up. same fonts. I can make these opaque as the pixels come in but this is how it comes in on its own.

[mrhelmut]

It depends on the font and if it has hinting. A font with good hinting enabled (such as this one: https://github.com/SolidZORO/zpix-pixel-font) will be super crisp with freetype and blurry with StbSharp at low size.

Font without hinting will look bad anyway.

Regarding http://community.monogame.net/t/spritefont-rendering-cant-seem-to-disable-smoothing/11225/9, we could add a parameter to the content processor to tell freetype to not use anti-aliasing.

Also note that anti-aliasing is different from hinting. Hinting are properties defined by the font to give hints to the renderer regarding how some subpixels should be handled at specific size. If the renderer doesn't support hinting, most subpixels are going to be interpolated (and most likely wrong).

[willmotil]

we could add a parameter to the content processor to tell freetype to not use anti-aliasing.

That would probably be a good thing. A option in the pipeline tool would be better but the aliasing if that is really what that's from is just terrible for freetype i don't think anyone would keep it on for long if they had a choice.

Well it doesn't matter to me which is used if we can get rid of that alaising. I think i got off track a bit though can freetype dynamically load the fonts.

[Jjagg]

@mrhelmut FreeType also has an auto-hinter that heuristically hints fonts without the font file containing hinting information.

@willmotil It's unfair to scale up a glyph and then say it's badly rendered because it's not close to what you'd expect for the character. It's rendered specifically to look good when unscaled.