Use UTF-8 for paths on Windows instead of locale - Githubissues

ryzom / ryzomcore

Ryzom Core is the open-source project related to the Ryzom game. This community repository is synchronized with the Ryzom Forge repository, based on the Core branch.

https://wiki.ryzom.dev

GNU Affero General Public License v3.0

333 stars 90 forks source link

Use UTF-8 for paths on Windows instead of locale #261

Closed ryzom-pipeline closed 8 years ago

ryzom-pipeline commented 8 years ago

Original report by Jan Boon (Bitbucket: [Jan Boon](https://bitbucket.org/Jan Boon), ).

Use the WCHAR version of all OS functions that deal with paths, and convert to UTF-8 std::string.

ryzom-pipeline commented 8 years ago

Original comment by Cédric Ochs (Bitbucket: [Cédric OCHS](https://bitbucket.org/Cédric OCHS), ).

Yep, and it'll be a huge task since fopen is used almost everywhere :p

ryzom-pipeline commented 8 years ago

Original comment by Jan Boon (Bitbucket: [Jan Boon](https://bitbucket.org/Jan Boon), ).

Make a NLMISC::fopen that converts from utf8 to wchar on Windows and uses the wchar fopen variant there

ryzom-pipeline commented 8 years ago

Original comment by Cédric Ochs (Bitbucket: [Cédric OCHS](https://bitbucket.org/Cédric OCHS), ).

Yes, I thought about that this night :)

ryzom-pipeline commented 8 years ago

Original comment by Cédric Ochs (Bitbucket: [Cédric OCHS](https://bitbucket.org/Cédric OCHS), ).

I need to work on it for next major version, because some users can have a username with unicode characters and since Ryzom will be installed in their Local profile, NeL could have some problems loading these files.

I already did some checks and even if using only "8 bits locale" everywhere works for French (é character for example), it'll fail for more complex characters like Cyrillic or Asian ones.

We have only 2 choices :

use same 8 bits locale everywhere under the same platform (UTF-8 under Unices and LATIN1/other under Windows) and Ryzom won't work when language is using more than 8 bits (what we're currently doing)
use UTF-8 everywhere and it'll work everywhere :p

ryzom-pipeline commented 8 years ago

Original comment by Jan Boon (Bitbucket: [Jan Boon](https://bitbucket.org/Jan Boon), ).

Yeah, it's best to consistently use UTF-8 everywhere, since resources like text files are also being loaded as UTF-8 and if they contain file names they'll technically be UTF-8 format anyway.

ryzom-pipeline commented 8 years ago

Original comment by Cédric Ochs (Bitbucket: [Cédric OCHS](https://bitbucket.org/Cédric OCHS), ).

Yep I agree :)

ryzom-pipeline commented 8 years ago

Original comment by Cédric Ochs (Bitbucket: [Cédric OCHS](https://bitbucket.org/Cédric OCHS), ).

Last changes for Unicode support under Windows, fixes #261