Backport unicode support from Unvanquished

[ETLegacyTeam]

Author: @JanSimek (Radegast) Date: 2018-12-26 Redmine Issue: 171, https://dev.etlegacy.com/issues/171

---

"Unvanquished":http://www.unvanquished.net has UTF-8 support since last year[1]. Their Daemon engine is based on Dushan's OpenWolf engine (ET:Xreal fork) with bits added from ioquake3. We should be able to port unicode console[2] without losing any compatibility with old mods, but game unicode text would definitely break compatibility. We will have to find a way to get around this.

They disabled unicode text in all cvars except player's name. We should disable even that.

Right now we are able to translate only several Western European languages correctly. Czech translation, for example, is hard to read without accented characters, so it is useless in the current state. With unicode support we could translate the game even to Russian or Greek.

fn1. "Bob's unicode patch":https://github.com/Unvanquished/Unvanquished/commit/d19bfbf890188e9d746b67517458a62aad701e18

fn2. "Unicode console":https://github.com/Unvanquished/Unvanquished/commit/8bd932492733b926dc224d9d70f85f2259cb7f40

---

Related issue(s): #172

---

Related commit(s): ba240a7bf133e29884fcdd2ea74851a266350fcf - generic: added unicode utility functions, refs 17.. c200934111173f03a5932e1c5c0408872621f877 - renderer: use unsigned long for unicode codepoint,.. 5b9734cf0dbb77147ea1fc9c5b872b9927be88b6 - misc: removed unused defines, refs 171 4d2b245b3882e8706564435a3c4df90dc17278d3 - renderer: limit pre-rendered font to 256 chars to .. dc37861a274e597f3fb74aa6e2b2f5dc72bc7a82 - ui: prepare for unicode display in menus, refs 17.. b7cd5a376cb368d8975385c1be308d4a3d6d4186 - cgame: prepare for unicode input, refs 171 f5b197d1b3c2b53f88a38080ae9f5aa08b2e04a4 - client: enable unicode up until the end of Cyrilli.. 2265a783ec7eaaa8a58584777b3b557ae12b6c2c - Revert "client: enable unicode up until the end of.. 2a763c67a87c75e94383b37e3947434572fbe822 - cgame: fixed text positioning, refs 171 697ae51b0accd05246e5bb7b9fbfaa6db9fad237 - generic: split ascii and unicode glyphs for mod co.. 6f05c07788343bea0cb7d37d8f84dafcec5f977b - misc: replaced unavailable character with a cross,.. a08329ebca97456eb84b3b5cba333d4e6b3677ed - renderer: always reload fonts, refs 171 fb63b6ddff1151d4d40c8a82ae872e204bc2a383 - renderer: don't ALWAYS reload fonts, refs 171 4c0278851c0686e088b51c8891803942f2cae4ea - client: separate buffers for console text and colo.. 95a658521ba6eea56b955a0bff0ee4b17a98be51 - common: make it possible to request a glyph with a.. f4ceed653709bb6b5a0e86393e5dd66ffa5dfa7b - enable windows console utf8 translation 53bfbc8f468115169a985eb632f4cc16403f70d8 - client: partially fixed the console utf8 input and.. f7ede027b8997c9768412bc3212710f380c58103 - general: Extended ascii support, fixes 827

[ETLegacyTeam]

From: @JanSimek (Radegast) Date: 2015-01-04T14:13:06Z

---

ET:L client will support all European alphabets including the Greek one plus Russian Cyrillic alphabet.

Hebrew, Armenian and Arabic languages could be added later, but eastern Asian languages will likely not be supported (even if we increased the current limits we would still have to find fitting fonts).

[ETLegacyTeam]

From: @IR4T4 (IR4T4) Date: 2015-01-08T19:35:31Z

---

Current linux build doesn't show any texts (cgame) - see screeny.

[ETLegacyTeam]

From: @saukko28 (Saukko) Date: 2015-01-08T20:31:42Z

---

Same with me on Windows. All texts are gone and also main menu is pretty messed up.

-*S

[ETLegacyTeam]

From: @JanSimek (Radegast) Date: 2015-01-09T15:53:50Z

---

Thanks to Ensiform we have a fix for that. Unicode glyphs need to be added at the end of the fontInfo_t struct:

typedef struct
{
    glyphInfo_t glyphs[GLYPH_ASCII_PER_FONT]; // ascii glyphs <255
    float glyphScale;
    char datName[MAX_QPATH];
    glyphInfo_t glyphsUTF8[GLYPHS_UNICODE_PER_FONT]; // unicode glyphs >255
} fontInfo_t;
</code>

[ETLegacyTeam]

From: @JanSimek (Radegast) Date: 2015-01-10T21:31:30Z

---

{FIXME} unicode glyphs (codepoint > 255) don't load past menu. Texture handle is 0.

Edit: fixed in commit:a08329eb