M-griffin / Oblivion2-XRM

Oblivion/2 XRM BBS Software
Other
89 stars 13 forks source link

[XRM-Server] Internal UTF-8 Encoding #213

Closed M-griffin closed 5 years ago

M-griffin commented 5 years ago

Encoding all internal data as UTF-8 then on output either push as is, or translate output to CP437 as needed. Most of the internal string and parsing methods already handle multi-byte sequences. This is more for proper internal localization.

NOTES: Internal ESC sequence and other characters < 128 do not need to be converted as they are shared single byte sequences. If certain control char glyphs are desired for ANSI character display then they should be encoded.

Cross Platform OSX WINDOWS LINUX

M-griffin commented 5 years ago

After some more testing, it appears the UTF-8 Internal translations are not 100% working properly in all instances.. Menu's, Configuration Editors, and other systems appears to work, but several snags get hit when testing the templates in the FSE who's ANSI template translations seem to keep bombing and not working properly on utf-8 glyph counting. So seeing multiple points of issues.

Glyph counting also appears to be an issues when testing the matrix screen so most likely the utf8-cpp library has issues looking individual glyphs which is a problem for most of the dynamic work needed when displaying.

This could be a show stopper even if we switch to ICONV. So might need something like UCS but then need to test that it's available and works across all platform the same. But itself C++11 internal methods are useless. And general support is just crud.

Boost was removed because it's Locale library didn't always build with UCS support by default making it difficult on some platform and next to impossible on others to get things working.

This could be the show stopper for c++ at this point.

M-griffin commented 5 years ago

Made some progress, looking into things a bit more. Last night was a ball of frustration.

M-griffin commented 5 years ago

After a late night debugging session the UTF-8 out and encoding issues appear to now be resolved.

M-griffin commented 5 years ago

Completed per pull request #216