search

scp-fs2open / fs2open.github.com

Origin Repository for SCP FreeSpace 2 Open
https://www.hard-light.net/
Other
402 stars 161 forks source link

Error in accentuated character (encoding mismatch ?) #2226

Open dseddah opened 4 years ago

dseddah commented 4 years ago

Hi again,

I tried to launch the game with a different language (French) and all accentuated characters are wrongly displayed as it's often the case with encoding mismatch.

step to reproduce 1° pick a different language (French) in knossos 2° start a training campaign, press F2 and see all those characters being wrongly displayed

configuration fso: all versions I could try (to 3.8.0 to nightly) os: mac os 10.14.6

Djamé ps: let me know if bug reports are welcome even for a niche platform (mac os, French)

The-E commented 4 years ago

Bug reports, regardless of how niche a system may be, are always welcome.

BMagnu commented 3 years ago

Having played around with non-ASCII before, I tried reproducing this with the UTF-8 Option switched on (by setting $Unicode mode: true in a -mod.tbm). Turns out that this issue is much more serious, as the game won't even start in UTF-8 mode (when knossos is set to French or German), as there is some sort of parsing error in the default strings.tbl This probably correlates to the fact that the Debug Build complains about those tables being in latin-1 instead of UTF-8 (which should probably be remedied anyways). Edit: Apparently not, converting that table to utf8 did not help

BMagnu commented 3 years ago

To follow this up and provide up to date information:

The actual crash in unicode mode is a table issue, not strictly an engine one, and discussed specifically in #2994.

This still leaves the default non-unicode vf fonts being unable to render accentuated characters. With the game having support for german and french both needing non-ascii, and after discussing this with @z64555 on discord in regards to the new, localized control code which can bind keycodes that are not strictly ascii, it would probably be good to provide a basic default fallback font that can render unicode text even when not in unicode mode. It doesn't even need to look fancy or massively well integrated, just that the character itself is legible.

Goober5000 commented 3 years ago

The best way to do that, I think, would be to use the non-accented character glyph. Transform é to e, ü to u, etc. It would be necessary to 1) detect that the original glyph can't be displayed, and 2) map the undisplayable glyph to a displayable one.

BMagnu commented 3 years ago

For general text, I'd agree. Maybe it'll need special handling for keybinds, so that people who bind ü don't get confused, thinking they bound u