Closed ryzom-pipeline closed 7 years ago
Original comment by Meelis Mägi (Bitbucket: [Meelis Mägi](https://bitbucket.org/Meelis Mägi), ).
CI18N::readTextFile() which uses CI18N:: readTextBuffer() already supports utf-8 input (with bom).
It needs to be decided if text without BOM should be read as binary or utf-8. If it's going to be utf-8, then forceUtf8 probably becomes redundant.
There is also CI18N::writeTextFile() which needs to be decided if UTF-8 needs bom or not (or if we need to write utf-16 in the first place)
Original comment by Cédric Ochs (Bitbucket: [Cédric OCHS](https://bitbucket.org/Cédric OCHS), ).
Good idea :) I suppose this decision has been taken because Windows natively supports UCS-2 (almost like UTF-16) so perhaps it's faster to load them :)
Original comment by Cédric Ochs (Bitbucket: [Cédric OCHS](https://bitbucket.org/Cédric OCHS), ).
I'll try to load UTF-8 words txt files, but I suspect it'll work without any change in code or perhaps only in translation_tool :)
Original comment by Meelis Mägi (Bitbucket: [Meelis Mägi](https://bitbucket.org/Meelis Mägi), ).
make sure you use french or russian translations for test ;-)
Original comment by Cédric Ochs (Bitbucket: [Cédric OCHS](https://bitbucket.org/Cédric OCHS), ).
Sure :p But it'll be easier to compare changes in Mercurial :)
Original comment by Cédric Ochs (Bitbucket: [Cédric OCHS](https://bitbucket.org/Cédric OCHS), ).
As said Nimetu, it works fine apparently :)
We'll just need to (slightly) update translation_tools.
And yes, I think we could consider 8 bits text files to be UTF-8 by default if not having any BOM.
Original comment by Cédric Ochs (Bitbucket: [Cédric OCHS](https://bitbucket.org/Cédric OCHS), ).
Fixed: Translations files are all using UTF-8 by default now (and LF instead of CRLF), fixes #128
Original report by Jan Boon (Bitbucket: [Jan Boon](https://bitbucket.org/Jan Boon), ).
Currently these files are in utf-16, but mercurial only supports utf-8 for merging. We should support utf-8 as file storage for these files. Other translation files are in utf-8.