Closed L3337 closed 1 year ago
Managed to get this sorted out much faster than anticipated, because it is a massive change that touches pretty much everything. I have code in the engine-unicode-support branch that works on every Linux and Windows locale that I have tested it on, but it needs some intense end-to-end regression testing on multiple locales, which will probably take all week.
Still untested is MacOS, which I am hoping has proper support for utf8 as Linux does. Since I cannot create 10 VMs of different MacOS locales to properly test like I can for Windows and Linux. Figuring out proper worldwide MacOS support will likely come after the next release.
I'm going to say that this bug is fixed for now. Seems to be in good shape on every platform I have tested on.
At the present, there are some fairly deep issues that cause crashes on locales with large alphabets that use unicode, such as Korean and Japanese, probably others.
The solution is going to be to devise a cross-platform strategy to use utf-8 for all files and file paths in the Python code (the easy part), and properly handle these paths under the hood in the C code in a way that works on Windows and UNIX (the hard part).
Potential strategies to solve this are being evaluated right now, for now the only way to ensure that Stargate works is to use en_US locale.