Support custom fonts for TextMeshPro

[gravydevsupreme]

Many games uses TextMeshPro with a font that has a very limited character set. When translating text into another language the translated text likely cannot be shown due to this.

[gravydevsupreme]

Examples Issues:

• 53

• 35

[Cerzs]

Hi @gravydevsupreme just curious, any ETA for this feature? I've found some games with TMP, mostly from MangaGamer.

Ty!

[gravydevsupreme]

It's not really an issue I have looked at it.

The problem with it is that it requires a bit of research on my end on how the tooling behind TMP works.

And since I don't really work with Unity or even it's Editor at all in my day-to-day business, it's not really something I have gotten a chance to look at yet.

Certainly I would like to look at it. but I cannot currently provide an ETA, especially since there are a lot of unknown that I am very uncertain of how much effort takes:

• How to generate the font using their (probably) native library
• How to actually get unity to load it, since these things are usually instantiated in the native part of the engine. (this thread suggest it is possible to load them through asset bundles, which may be the best option http://digitalnativestudios.com/forum/index.php?topic=658.0).

Other potentially relevant reference: http://digitalnativestudios.com/forum/index.php?PHPSESSID=kc1ilgtph5b096bd4j9v9v5fg4&topic=805.0

[gravydevsupreme]

Just a small update on this one:

Today I tried to build the fonts that comes when you install the standard TextMeshPro package into an asset bundle and load them through the plugin. And it did seem to work. At least with that specific version of the engine.

So there may be some progress towards this goal.

But that means I would need to figure out which unicode ranges I would need to include in such a font to support various languages.

Unicode table: https://jrgraphix.net/research/unicode_blocks.php

Essentially I would want ranges for: Japanese, Chinese, Korean, Russian + various uncommon latin characters. (Maybe I am forgetting some).

Frankly, I could use some help with this, because some of these unicode ranges seem huge. For instance is all Hangul characters in the range AC00 - D7AF really used?

And I guess I would have to do some compromising when it comes to the thousands of Kanji characters that exist.

I tried generating a font that included thousands of characters today, but the progress bar never left 0. ^^

[Cerzs]

Hm, maybe doing it in parts? Spanish uses these characters only, for example.

Including various languages into a font maybe would create a very large font, 10+ mb perhaps?

[gravydevsupreme]

So, I got a prototype working after spending way too much time looking for fonts.

I have created two fonts that can be placed in the root game directory and referenced from the config file through the TextMeshPro config option.

• arialuni_sdf (I think in general this supports most characters)
• notosanscjk-regular_sdf (may not support russian)

Place the unzipped font files in the game root and set config OverrideFontTextMeshPro=notosanscjk-regular_sdf

There is a significant chance that this is not going to work in all games and it probably depends on both the version of the Unity Engine that is used and the version of TextMeshPro.

These files are generated using Unity 2018.1 and the TMP that comes with it and I can confirm it works with that ResizeMe game from #69

fonts.zip on mediafire XUnity.AutoTranslator-ReiPatcher-4.7.1.zip XUnity.AutoTranslator-BepIn-5x-4.7.1.zip

These asset bundles can be generated without programming knowledge, but the older the version of unity to target, the more difficult it may be.

I would like to know, if you can tell me, where it works and where it doesn't.

You can see which version of unity is used by right clicking on the game exe file and going to Details. From there, there is a File version and Production version which represents the unity engine version.

[Cerzs]

I've tested with Cartagra, the game just keep freezed. I'll try with other games. Rei does the same thing.

Unity version:

Also getting errors but running anyway.

System.TypeLoadException: Could not load type 'BepInEx.Preloader.RuntimeFixes.UnityPatches' from assembly 'BepInEx.Preloader, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'. at BepInEx.Preloader.Preloader.Run () [0x00000] in <filename unknown>:0

Tested games (Freezed):

• Cartagra
• Free Friends

[gravydevsupreme]

Here's a new version that fixes the freezes (the plugin was using a "new" API not available in these old games):

XUnity.AutoTranslator-ReiPatcher-4.7.1.zip

I can imagine BepInEx not working particularly well on such old games.

Now, just because it doesn't freeze doesn't mean it works.

The problem is just substituted with another, and that is the fact that this asset bundle is not compatible with the game because it was generated by a much newer version of the Unity Editor.

The following message appears in output_log.txt:

This asset bundle was not created with UncompressedAssetBundle flag, expected id 'UnityRaw', got 'UnityFS'

Which likely means that if we can actually generated a bundle that is compatible with this engine, it is going to work.

Such a process would involve:

1. Obtaining an appropriate Unity Editor from https://unity3d.com/de/get-unity/download/archive
2. Creating a new project
3. Installing a TextMeshPro version that is compatible with that Unity Editor
4. Creating a TextMeshPro font asset from a "standard font" (good choice is "Arial Unicode MS Regular" or "arialuni" which I think can always be found in the windows Fonts folder.
5. Creating an asset bundle that contains the created TextMeshPro font asset.

I am not sure how compatibilities look across different versions of Unity when it comes to asset bundles. Maybe an asset bundle generated in 5.x or 4.x will be compatible with 2017 or newer. Or maybe not. Same thing can be said of the actual generated TextMeshPro font asset.

[gravydevsupreme]

This might sound a bit easier than it is though. Because TMP was only included as part of Unity since version 2018. Prior to that it was a separate paid product.

[gravydevsupreme]

Actually, I wanted to try to generate an asset bundle that isn't compressed and see if that works before attempting anything else. But right now my Unity Editor is acting up and crashing whenever I open it...

[Cerzs]

Yep, the game is now working, but as you say, is not compatible with this old version.

[gravydevsupreme]

Tried with an uncompressed bundle. Doesn't work either.

[gravydevsupreme]

Here's a new fonts upload that includes a new font asset bundle that should theoretically work for unity 5.5 - 2017.4. Theoretically as in not really tested.

Still wont work for Cartagra. Frankly, I cannot even get the Unity 4.x Editor to run on my system.

https://www.mediafire.com/file/03f3ru5d8hk3rq9/fonts.zip/file

Got Unity 4.5 working on my system. But it turns out that building asset bundles requires the professional edition. -_-

[Cerzs]

I'm trying to find a game 5.5 or newer, but all that I found is 4.x-5.3

Free Friends 2, for example:

[djlaser]

Initialize engine version: 2018.3.4f1 (1d952368ca3a) plugin worker (Russian translation) from arialuni_sdf

[gravydevsupreme]

Getting this to work for old games (pre-5.5) is pretty much a no-starter unless there is some way to obtain an older version of TMP.

There's several problems here:

• The asset store offers zero revision history on uploaded assets. (I don't even know what to say about this. So absurd)
• The previous way to obtain TMP, which was through the creator's private forum has been disabled. You would previously purchase TMP through that forum. The author did state that if one had previously purchased the asset, you would still be able to access it there.
• I have absolutely no clue how I would obtain these old versions of TMP in any other way than actually contacting the author, which I don't really feel like doing.

And if by some miracle we manage to obtain an old version of TMP, we would also need a professional edition of the old Unity Editor to support pre-5.0.

[gravydevsupreme]

The latest version (4.8.0) now has support for loading external asset bundles.

With that being said, I had trouble getting these bundles to work with anything that weren't based on Unity 2018 or greater where the Unity.TextMeshPro.dll file is present in the managed directory.

The asset bundle loading implementation is probably going to work for any game, but the problem is still generating the correct asset bundle with a compatible asset. I fear that for older version of Unity it might require exact version matches for TextMesh Pro and (potentially) Unity, which is a massive pain.

[gravydevsupreme]

Interesting to see you could get it working in a Unity 2017 game.

When I generate a font I usually use the following settings:

• Source Font File: Arial Unicode MS Regular (arialuni.ttf). Noto Sans CJK might be a better choice for you, although I think arialuni does have most of the important Kanji.
• Sampling Point Size: 40
• Padding: 3
• Packing Method: Fast
• Atlas Resolution: 8192 x 8192 (yes, that's a big texture, but I am trying to include many languages as well)
• Character set: Uncode Range (Hex) 0000-052F,2E80-2EFF,3000-4DBF,4E00-9FFF,AC00-D7AF
• Render Mode: SDF16 / Distance Field 16

Usually takes round about half an hour to generate.

[gravydevsupreme]

When it comes to the GUIDs of TMP, I think they can be separated into:

• 2017 and earlier
• 2018 and later

This blog post describes how these GUIDs were modified during that transition.

Although if that was the only issue I would kind of have expected the Unity 2017 asset store TMP / and the "loose" TMP to be compatible, which they clearly are not. This is really my main concern because such arbitrary differences are going to be difficult to handle, if it's supposed to be done in an automated fashion.

After simply generating a TMP asset bundle for 2018.1, I was kinda surprised to simply see it working in a 2018.4 game, since I assume the package manager TMP should suffer from the same kind of problems (minor differences in data format expectations).

Generally I would say that the best approach is to simply target the specific version of Unity / TMP that a game uses.

It's very cool to see you got it working with Emotion Creators.

[djlaser]

5.6.2f1 and 5.6.4f1 Dont work TMP output_log.txt output_log2.txt

[gravydevsupreme]

There isn't really that much I can do about this.

I do not expect the fonts I have included in the release section supports any version of TMP where there is not a dll called Unity.TextMeshPro.dll in the Managed folder.

The problem is that I am unable to build the font without access to the editor tools for the version of TMP used in that game. And those old versions of TMP are not free and not even available for purchase anymore.

[djlaser]

There isn't really that much I can do about this.

I do not expect the fonts I have included in the release section supports any version of TMP where there is not a dll called Unity.TextMeshPro.dll in the Managed folder.

The problem is that I am unable to build the font without access to the editor tools for the version of TMP used in that game. And those old versions of TMP are not free and not even available for purchase anymore.

Thanks for the answer. The app is very good but why the there is no in Diskord canal on him.

[gravydevsupreme]

This plugin is developed primarily in relation to various ILLUSION games. As such it is primarily discussed on Illusion Soft and Koikatsu! discords. I'm scrubbing around on those discords regularly.

There is no discord for this plugin specifically, as I don't really want to maintain it nor do I really feel like it is warranted.

[djlaser]

Hello dear gravydevsupreme!

Please write instructions or make video  how to generate TextMeshPro fonts and it is desirable to how to collect the generated script assestsbundle

Sincerely, djlaser That's what I did and then did Fonts1.zip

[Cerzs]

I was wondering, some games includes .ttf fonts in their .assets, is not possible to redirect the TMP to that .ttf file?

[gravydevsupreme]

I'm afraid not. TMP requires assets that are build specifically for it. You can construct such assets from .ttf files using its editor tools. But as I've mentioned before you generally need the correct version of the tooling to be able to do that. Tooling, which for old versions, is not available for purchase anymore.

[Cerzs]

This may not be useful, but TMP textures can be read/saved with a Photoshop NVIDIA plug-in.

The bad news: You have to modify the symbols you don't need for the character you want. Example: * = á.

Also, this may break the font slightly.

[gravydevsupreme]

It also looks like that texture is built based on the actual characters used in the game (as opposed to unicode ranges). So replacing any of them would cause that character to be missing where it was previously used.

I'm not sure I am ready to jump down the rabbit hole of manually manipulating textures like this.

If I was extremely desperate to get this game translated, then my approach would probably be to contact the author of TMP (which I am pretty sure is just a single person (at least back then)) and ask him how much it would cost to get access to the archived versions of the TMP editor/runtime tooling. (of course, with the risk of still not being able to get it working after paying)

This is of course still assuming that the Unity version is greater than 5.x. If not you need access to a professional edition of the Unity editor as well.

[nathan60107]

Hi, I just want you to know the solution at https://github.com/bbepis/XUnity.AutoTranslator/issues/57#issuecomment-562344581 work on the game Skul. Its file version is 2019.4.8.30749, greater than 2018. So it works correctly as expected.

The solution helps me a lot. I think you should put it at release and TextMeshPro=arialuni_sdf should be a default config. Or at least, put a link at README.md to let the user know if their game use TextMeshPro and their game cannot display translation text correctly, they can try this solution to solve it. This issue always happens in the area which the native language is not English.

Thank you for create XUnity.AutoTranslator.

[gravydevsupreme]

It certainly isn't the most visible part of the documentation, but it is documented.

Although it can certainly be daunting to find anything in that README.

[nathan60107]

In Phoenotopia Awakening Demo, OverrideFontTextMeshPro=arialuni_sdf has no effect. I put arialuni_sdf in every folder of that game. And debug console just tell me:

[INFO][XUnity.AutoTranslator]:  Attempting to load TextMesh Pro font from asset bundle.
[ERROR][XUnity.AutoTranslator]:  Could not find the TextMeshPro font asset: arialuni_sdf

Do you have any idea about that error message?

[KinkyCollars]

arialuni_sdf-u55to2017 notosanscjk-regular_sdf_u2018plus arialuni_sdf_u2018plus arialuni_sdf_u2019 arialuni_sdf_u2018

Which one knows the Hungarian characters? Melyik tudja a magyar ékezetes karaktereket?

[hungchips]

Hello any one, can you help me convert font .ttf to sdf? (My country is VN). All font in topic not support Vietnamese in TextMessPro. my font .ttf download is link https://www.mediafire.com/file/ogzv8rtagmd5l9n/comic.ttf/file

[Atamg1994]

Examples Issues:

• Corpse Party - Blood Drive #53
• Delay (Text with effects)? #35

HI I ran into a similar problem and the craftsmen for this game made a plugin for melon loader which makes it possible to replace the font with a dynamic one

those you do not need to compile resources in advance, just took a ttf file fired it up and everything works

the mod was on the IL2CPP version, but for one person I was able to remake it under mono

it would be nice if built directly into the software) https://github.com/bbepis/XUnity.AutoTranslator/issues/290 https://github.com/bbepis/XUnity.AutoTranslator/issues/287

[DearFox]

Examples Issues:

• Corpse Party - Blood Drive #53
• Delay (Text with effects)? #35

HI I ran into a similar problem and the craftsmen for this game made a plugin for melon loader which makes it possible to replace the font with a dynamic one

those you do not need to compile resources in advance, just took a ttf file fired it up and everything works

the mod was on the IL2CPP version, but for one person I was able to remake it under mono

it would be nice if built directly into the software) #290 #287

FontOverride is a really nice mod and it helps to fix font issues for some games. But I ran into a problem, because in one game the FontOverride mod (mono) causes an error. And the fonts are not replaced \ just disappear and the text is invisible.

[12:40:47.698] [FontOverride] [ERROR] System.TypeInitializationException: The type initializer for 'FontOverride.FontStore' threw an exception. ---> System.NotImplementedException: The method or operation is not implemented.
  at System.IO.FileSystemWatcher..ctor (System.String path) [0x00006] in <0813754e7cfa415ebfedff172c2dd6e0>:0
  at (wrapper remoting-invoke-with-check) System.IO.FileSystemWatcher..ctor(string)
  at FontOverride.FontStore..cctor () [0x0006b] in <5a97424209c24f5a8fe9476602f85fbb>:0
   --- End of inner exception stack trace ---
  at FontOverride.FontOverride.OnSceneWasLoaded (System.Int32 buildIndex, System.String sceneName) [0x00001] in <5a97424209c24f5a8fe9476602f85fbb>:0
  at MelonLoader.MelonEvent`2+<>c__DisplayClass1_0[T1,T2].<Invoke>b__0 (MelonLoader.LemonAction`2[T1,T2] x) [0x00000] in <6425afb6ac6a429aaf770255d2f7de57>:0
  at MelonLoader.MelonEventBase`1[T].Invoke (System.Action`1[T] delegateInvoker) [0x00018] in <6425afb6ac6a429aaf770255d2f7de57>:0

Sorry if this is in the wrong place, I couldn't find the FontOverride mod repository to report a bug. Game I'm having trouble with: the forest of love (this is an NSFW game, so be careful)

[rasberryrabbit]

Including characters in unicode 0020-052F,2000-9FFF,AC00-D7AF,D7B0-D7FF,F900-FAFF,FF00-FFEF for more CJK character support.

Example : NotosansCJKkr_bold

[Atamg1994]

FontOverride is a really nice mod and it helps to fix font issues for some games. But I ran into a problem, because in one game the FontOverride mod (mono) causes an error. And the fonts are not replaced \ just disappear and the text is invisible.

Sorry if this is in the wrong place, I couldn't find the FontOverride mod repository to report a bug. Game I'm having trouble with: the forest of love (this is an NSFW game, so be careful)

lets try instal this version https://mega.nz/file/XEBGGDTa#d3AAcAnhe9iHGzkCvZkCQVH07XdCvyW00rJVqMgwVWE and replace FontOverride_fix.zip and use melon loader 0.5.7 version The plugin does not work on large versions at all.

[DearFox]

FontOverride is a really nice mod and it helps to fix font issues for some games. But I ran into a problem, because in one game the FontOverride mod (mono) causes an error. And the fonts are not replaced \ just disappear and the text is invisible.

Sorry if this is in the wrong place, I couldn't find the FontOverride mod repository to report a bug. Game I'm having trouble with: the forest of love (this is an NSFW game, so be careful)

lets try instal this version https://mega.nz/file/XEBGGDTa#d3AAcAnhe9iHGzkCvZkCQVH07XdCvyW00rJVqMgwVWE and replace FontOverride_fix.zip and use melon loader 0.5.7 version The plugin does not work on large versions at all.

I am using MelonLoader v0.5.7 Open-Beta (maybe I should update it?)

In general, I still get errors with this mod. And as a result, XUnity Auto Translator does nothing at all Here is the log with errors, I copied part, let me know if you need full logs, but it seems the error is repeated

log.txt

[Atamg1994]

FontOverride is a really nice mod and it helps to fix font issues for some games. But I ran into a problem, because in one game the FontOverride mod (mono) causes an error. And the fonts are not replaced \ just disappear and the text is invisible.

Sorry if this is in the wrong place, I couldn't find the FontOverride mod repository to report a bug. Game I'm having trouble with: the forest of love (this is an NSFW game, so be careful)

lets try instal this version https://mega.nz/file/XEBGGDTa#d3AAcAnhe9iHGzkCvZkCQVH07XdCvyW00rJVqMgwVWE and replace FontOverride_fix.zip and use melon loader 0.5.7 version The plugin does not work on large versions at all.

I am using MelonLoader v0.5.7 Open-Beta (maybe I should update it?)

In general, I still get errors with this mod. And as a result, XUnity Auto Translator does nothing at all Here is the log with errors, I copied part, let me know if you need full logs, but it seems the error is repeated

log.txt

it looks like the problem is that the game has an old library and there are not enough methods System.MissingMethodException: bool TMPro.TMP_FontAsset.HasCharacters I'll try to fix it one of these days

[Atamg1994]

FontOverride is a really nice mod and it helps to fix font issues for some games. But I ran into a problem, because in one game the FontOverride mod (mono) causes an error. And the fonts are not replaced \ just disappear and the text is invisible.

Sorry if this is in the wrong place, I couldn't find the FontOverride mod repository to report a bug. Game I'm having trouble with: the forest of love (this is an NSFW game, so be careful)

lets try instal this version https://mega.nz/file/XEBGGDTa#d3AAcAnhe9iHGzkCvZkCQVH07XdCvyW00rJVqMgwVWE and replace FontOverride_fix.zip and use melon loader 0.5.7 version The plugin does not work on large versions at all.

I am using MelonLoader v0.5.7 Open-Beta (maybe I should update it?)

In general, I still get errors with this mod. And as a result, XUnity Auto Translator does nothing at all Here is the log with errors, I copied part, let me know if you need full logs, but it seems the error is repeated

log.txt

instructions steps 1 instal MelonLoader v0.5.7 Open-Beta (need this version) 2 instal XUnity.AutoTranslator mono MelonLoader last version 3 download this https://mega.nz/file/XEBGGDTa#d3AAcAnhe9iHGzkCvZkCQVH07XdCvyW00rJVqMgwVWE unpack game folder 4 download fix FontOverride_fix.zip

4.1 Config.ini - move in AutoTranslator folder (its great config) and change Language=ru set you need lang

4.2 FontOverride.dll - its fix move to "mods" folder

4.3 resizer.txt -its fix text size move to "AutoTranslator\Translation\You_language\Text"

see this before resizer see this after resizer

good game)

[DearFox]

FontOverride is a really nice mod and it helps to fix font issues for some games. But I ran into a problem, because in one game the FontOverride mod (mono) causes an error. And the fonts are not replaced \ just disappear and the text is invisible.

Sorry if this is in the wrong place, I couldn't find the FontOverride mod repository to report a bug. Game I'm having trouble with: the forest of love (this is an NSFW game, so be careful)

lets try instal this version https://mega.nz/file/XEBGGDTa#d3AAcAnhe9iHGzkCvZkCQVH07XdCvyW00rJVqMgwVWE and replace FontOverride_fix.zip and use melon loader 0.5.7 version The plugin does not work on large versions at all.

I am using MelonLoader v0.5.7 Open-Beta (maybe I should update it?) In general, I still get errors with this mod. And as a result, XUnity Auto Translator does nothing at all Here is the log with errors, I copied part, let me know if you need full logs, but it seems the error is repeated log.txt

instructions steps 1 instal MelonLoader v0.5.7 Open-Beta (need this version) 2 instal XUnity.AutoTranslator mono MelonLoader last version 3 download this https://mega.nz/file/XEBGGDTa#d3AAcAnhe9iHGzkCvZkCQVH07XdCvyW00rJVqMgwVWE unpack game folder 4 download fix FontOverride_fix.zip

4.1 Config.ini - move in AutoTranslator folder (its great config) and change Language=ru set you need lang

4.2 FontOverride.dll - its fix move to "mods" folder

4.3 resizer.txt -its fix text size move to "AutoTranslator\Translation\You_language\Text"

see this before resizer see this after resizer

good game)

Thanks for your help, this really works.

Although the first launch of the game was problematic, the game froze whenever new text was supposed to load (as I understand it) As a result, starting the game for the first time with this fix, the game occupied more than 16 gigabytes of RAM and then crashed, but apparently the second launch is more stable, and I no longer notice any delays, and the game itself consumes no more than 2 gigabytes of RAM

Thanks again for your help 🐈

[Atamg1994]

Thanks for your help, this really works.

Although the first launch of the game was problematic, the game froze whenever new text was supposed to load (as I understand it) As a result, starting the game for the first time with this fix, the game occupied more than 16 gigabytes of RAM and then crashed, but apparently the second launch is more stable, and I no longer notice any delays, and the game itself consumes no more than 2 gigabytes of RAM

you probably have Chinese or some other complex language with a lot of characters

you can set the map file in advance before starting the game

set in advance all the possible characters of your language and the game is easier to work with fonts at startup than during the game, the map file will then be filled with new characters if they are in the game

[omaryomario]

So, I got a prototype working after spending way too much time looking for fonts.

I have created two fonts that can be placed in the root game directory and referenced from the config file through the TextMeshPro config option.

• arialuni_sdf (I think in general this supports most characters)
• notosanscjk-regular_sdf (may not support russian)

Place the unzipped font files in the game root and set config OverrideFontTextMeshPro=notosanscjk-regular_sdf

There is a significant chance that this is not going to work in all games and it probably depends on both the version of the Unity Engine that is used and the version of TextMeshPro.

These files are generated using Unity 2018.1 and the TMP that comes with it and I can confirm it works with that ResizeMe game from #69

fonts.zip on mediafire XUnity.AutoTranslator-ReiPatcher-4.7.1.zip XUnity.AutoTranslator-BepIn-5x-4.7.1.zip

These asset bundles can be generated without programming knowledge, but the older the version of unity to target, the more difficult it may be.

I would like to know, if you can tell me, where it works and where it doesn't.

You can see which version of unity is used by right clicking on the game exe file and going to Details. From there, there is a File version and Production version which represents the unity engine version.

May i ask please how did you export it? how you removed the .asset from the exported sdf font?

EDIT: i made the asset bundle and it works super great on the game BUT, my language is arabic and arabic is from "Right-to-left". IS there a way of changing that in the config file?