Devan-Kerman / ARRP

A fabric api for creating resources and assets at runtime
Mozilla Public License 2.0
101 stars 25 forks source link

Translations for non english characters produce question marks in game #81

Open enchantinggg4 opened 1 year ago

enchantinggg4 commented 1 year ago

For japanese, cyrillic and I'm pretty sure for almost all non latin characters

val lang = JLang() .item(Identifier("krp", "magma_helmet"), "私は自分自身をたわごと")
ResourcePack.addLang(Identifier("krp:ru_ru"), lang)

game will show ????? instead of letters in game

Devan-Kerman commented 1 year ago

can you verify with a .dump on your resource pack that the json contains ???s

Devan-Kerman commented 1 year ago

it might be that the java file you wrote those characters in isn't utf8 encoded

enchantinggg4 commented 1 year ago

Yes, it generates file json files in cp1251 encoding(windows cyrillic) Howewer all my code files are in utf-8.

enchantinggg4 commented 1 year ago

RuntimeResourcePackImpl:

private static byte[] serialize(Object object) {
        UnsafeByteArrayOutputStream ubaos = new UnsafeByteArrayOutputStream();
        OutputStreamWriter writer = new OutputStreamWriter(ubaos);
        GSON.toJson(object, writer);
        try {
            writer.close();
        } catch(IOException e) {
            throw new RuntimeException(e);
        }
        return ubaos.getBytes();
    }

If no charset is provided, it OutputStreamWriter will use system's default charset I think its better to set UTF-8 here or add an option to override charset somewhere in resource pack class.

I can experiment locally and if it is the case, ill make a pr

enchantinggg4 commented 1 year ago

I got it I'm on 1.18.2, so i am using ARP 0.5.7, which doesn't have the fix for it https://github.com/Devan-Kerman/ARRP/commit/832b788261e658dd886b929b0117f8bab6ace1c4#diff-0525a3f0d683be2e4e572c0caa32e5cd3d90c026fbb7a2cfee77fd4c29e0db55R525 It came as a crash fix for 1.19

So 0.5.7 always writes system's default charset Is there a chance to backport this change to 1.18 version?