Strings from titles.txt file with extended latin characters are not rendered correctly in the game

[ELjanoCZE]

Compared to the rest of the strings that Goldsrc renders (from files like gameui_english or valve_english), strings from titles.txt with extended latin characters are not rendered correctly in the game. This is probably an early engine limitation or bug. It would be nice to fix this, as it unnecessarily limits the translation of Goldsrc games into other languages.

Czech pangram which I used for testing: Příliš žluťoučký kůň úpěl ďábelské ódy

[kisak-valve]

Hello @ELjanoCZE, what character encoding did you use when saving that file? My understanding is that UTF 8 has the best chance of working.

[ELjanoCZE]

It is saved with UTF-8 encoding.

titles.txt

[SamVanheer]

This is probably happening because that text is drawn one byte at a time: https://github.com/ValveSoftware/halflife/blob/c7240b965743a53a29491dd49320c88eecf6257b/cl_dll/message.cpp#L301-L310

(not the only place characters are drawn)

UTF-8 characters are multi-byte so this needs updating to read whole code points. This may require a new engine function to draw UTF-8 characters since the current API only accepts one byte.

[libre-x]

If this would get fixed, #2490 would get fixed as well as it's the same issue. Maybe wouldn't be that hard of a fix?

[tmp64]

@SamVanheer pfnDrawCharacter can draw wchar_t values correctly. I used that to fix CHudMessage.

https://github.com/tmp64/BugfixedHL-Rebased/blob/7c94c49b62f1f2afd4e069488f8f73d7cfe10d39/src/engine/APIProxy.h#L596-L606

https://github.com/tmp64/BugfixedHL-Rebased/blob/7c94c49b62f1f2afd4e069488f8f73d7cfe10d39/src/game/client/hud/message.cpp#L251-L254

[SamVanheer]

Since it's possible to do breaking changes maybe the character type should be changed to UTF-32 instead? That way it should support all inputs. Mods should be able to update easily by updating to the new SDK and with a bit of luck the old version should just work if ASCII characters have the same integer value as its UTF-32 character equivalent.

[libre-x]

@kisak-valve can we get a VALVe developer to comment here please? I feel we’re onto something.