Add Support For Japanese Text (ImGui)

[atom0s]

Feature Description

The current setup for Ashita does not properly support ImGui's ability to render UTF8 text. The two built-in fonts both do not contain the needed glyphs and the default font setup does not include the needed glyph ranges for proper UTF8 support.

The following would be ideal to implement/fix in a future update:

• [x] ImGui: Add a default UTF-8 compatible font to be included with Ashita.
• [x] ImGui: Add the proper glyph ranges to the fonts able to render UTF-8.
• [ ] ImGui: Ensure rendering from Ashita's Japanese string resources is valid/working.

This same issue carries over to the built-in font rendering of Ashita. The current font system dose not include any of the extended glyphs needed to render Japanese/UTF-8 text. This may not be ideal to implement in the current font system though due to how it works. (Perhaps wait until a newer system is added, such as GDI+ or D2D?)

Related issue: #46

[onimitch]

RE the third point I think the main thing will be converting from ShiftJIS (which the game uses) to UTF-8? Another point is also if needing to get JA input out of IMGUI (i.e from a text field to exec a text command), I don’t think the game will recognise it as UTF8?

[ThornyFFXI]

Automatically converting specific IMGUI fields, such as the issue you're clearly referencing from cross/thot bars, is outside of the scope of Ashita. Please be patient and allow atom0s time to determine the best way to handle the resources, and at such time as IMGUI properly supports japanese text I will handle my addons' integration with it.

Edit: It is a tricky issue, because the resources are created in shift-JIS from the game's files. They could be automatically converted to UTF-8 for compatibility with IMGUI, but at that point they'd no longer be compatible with abilities as you pointed out. A universal solution that allows the same resources to be used for both IMGUI elements and game commands seamlessly is probably not likely, unless IMGUI can be made compatible with shift-jis. But, it is not all that difficult to solve on an individual addon basis; the string can be stored as shift-JIS, converted to utf-8 for display purposes, and alterations to the utf-8 string converted back to modify the stored string. Or, the final command processor could convert it from UTF-8 to shift-JIS before passing it to the game and everything else could be kept as UTF-8.

[onimitch]

Don’t worry i’m not trying to rush the issue, i’m just trying to help scope out the feature (or help limit the scope as the case may be). I’ve seen that atom0s has been taking a break from ff11 projects for a while so i’m not suddenly expecting this to be done in the next week or month.

[atom0s]

Marking this as completed via https://github.com/AshitaXI/Ashita-src/commit/f4338a9a3d414cef4b00134c8f5288dcbfcd0009

The goal list above is 2/3 completed with the last one being considered out of scope.

ImGui: Ensure rendering from Ashita's Japanese string resources is valid/working.

At this time, we do not plan to add direct support of converting ShiftJIS to UTF8 within Ashita. There are a couple of factors for this choice, but ultimately we are going to consider it out of scope. It is pretty easy to convert from ShiftJIS to UTF8 though which can be found around the web in different coding languages.

In the future, we may include a lib for addons, but within the core and plugins, developers will be left to deal with the translation themselves for now.