Localization with AI/ML translation

[JohN100x1]

Implement localization for all languages using machine translation. Possible options:

• https://github.com/facebookresearch/fairseq/blob/main/examples/translation/README.md
• https://github.com/OpenNMT/OpenNMT-py
• https://github.com/marian-nmt/marian
• https://github.com/tensorflow/tensor2tensor#translation
• https://github.com/simpleapples/chatgpt-translator
• https://github.com/yetone/openai-translator
• https://github.com/DeepLcom/deepl-python

[kjk001]

the question is if there is one that could be told the format of the json file and then just adds the code in the correct place

[JohN100x1]

I think I can make a simple script to read the json, perform an ensemble translation and pick the best one based on some evaluation metrics. The difficulty is that I have no ground truth for what the translated text should be.

[kjk001]

Maybe ask a discord team that specializes in translating games like the Amateur Modding Avenue what machine translators work best in their experience?

[JohN100x1]

sample translation from chatgpt which took about 2 mins

[
  {
    "Key": "714b99be4cc74d8ea02500d4c7cffa4e",
    "SimpleName": "$IsekaiProtagonistSpellbook.Name",
    "ProcessTemplates": false,
    "enGB": "Isekai Protagonist",
    "ruRU": "Исекай Протагонист",
    "deDE": "Isekai-Hauptfigur",
    "frFR": "Protagoniste d'Isekai",
    "zhCN": "异世界主角",
    "esES": "Protagonista de Isekai"
  },
  {
    "Key": "457405fbd37c412f96c2be6b5109c612",
    "SimpleName": "PlotArmor.Description",
    "ProcessTemplates": true,
    "enGB": "You gain a luck bonus to {g|Encyclopedia:Armor_Class}AC{/g} and all {g|Encyclopedia:Saving_Throw}saving throws{/g} equal to your character level.",
    "ruRU": "Вы получаете бонус удачи к {g|Encyclopedia:Armor_Class}КД{/g} и всем {g|Encyclopedia:Saving_Throw}спасброскам{/g}, равный вашему уровню персонажа.",
    "deDE": "Du erhältst einen Glücksbonus auf {g|Encyclopedia:Armor_Class}RK{/g} und alle {g|Encyclopedia:Saving_Throw}Rettungswürfe{/g}, der deinem Charakterlevel entspricht.",
    "frFR": "Vous obtenez un bonus de chance à {g|Encyclopedia:Armor_Class}CA{/g} et à tous les {g|Encyclopedia:Saving_Throw}jets de sauvegarde{/g} égal à votre niveau de personnage.",
    "zhCN": "你获得了一个幸运加值，对{g|Encyclopedia:Armor_Class}护甲等级{/g}和所有{g|Encyclopedia:Saving_Throw}豁免检定{/g}，加值等于你的角色等级。",
    "esES": "Obtienes un bonificador de suerte a {g|Encyclopedia:Armor_Class}CA{/g} y a todas las {g|Encyclopedia:Saving_Throw}tiradas de salvación{/g} iguales a tu nivel de personaje."
  },
  {
    "Key": "f161809e34be41d69fead36851be775a",
    "SimpleName": "UnlimitedPowerAbility.Name",
    "ProcessTemplates": false,
    "enGB": "Overpowered Ability — Unlimited Power",
    "ruRU": "Сверхмощная способность - Бесконечная мощь",
    "deDE": "Übermächtige Fähigkeit - Unbegrenzte Macht",
    "frFR": "Capacité surpuissante - Puissance illimitée",
    "zhCN": "超能力 - 无限力量",
    "esES": "Habilidad sobrepotenciada - Poder ilimitado"
  },
  {
    "Key": "ac27deaf79654a76a107f202cbc6d7a1",
    "SimpleName": "IsekaiDialogueHorgusReply.Text",
    "ProcessTemplates": false,
    "enGB": "\"My daughter?\" {n}Horgus takes a quick glance at Camellia.{/n} \"I don't think you know what sort of person she is. {n}Horgus takes a breath.{/n} Regardless, it seems my previous offer was somewhat insulting. Two thousand gold it is.\"",
    "ruRU": "\"Моя дочь?\" {n}Хоргус быстро взглянул на Камеллию.{/n} \"Я не думаю, что вы знаете, какой она человек. {n}Хоргус вздохнул.{/n} В любом случае, кажется, мое предыдущее предложение было немного оскорбительным. Две тысячи золотых монет - это цена.\"",
    "deDE": "\"Meine Tochter?\" {n}Horgus wirft Camellia einen schnellen Blick zu.{/n} \"Ich glaube nicht, dass Sie wissen, was für eine Person sie ist. {n}Horgus atmet tief durch.{/n} Wie auch immer, es scheint, dass mein vorheriges Angebot etwas beleidigend war. Es sind zweitausend Goldstücke.\"",
    "frFR": "\"Ma fille ?\" {n}Horgus jette un rapide coup d'œil à Camellia.{/n} \"Je ne pense pas que vous sachiez quel genre de personne elle est. {n}Horgus prend une inspiration.{/n} Quoi qu'il en soit, il semble que mon offre précédente était quelque peu insultante. Deux mille pièces d'or, c'est ça.\"",
    "zhCN": "\"我的女儿？\" {n}霍格斯快速地瞥了一眼卡梅莉娅。{/n} \"我不认为你知道她是什么样的人。{n}霍格斯深吸了一口气。{/n} 无论如何，我的前一份报价似乎有些冒犯。两千金币就是了。\"",
    "esES": "\"¿Mi hija?\" {n}Horgus echa un rápido vistazo a Camellia.{/n} \"No creo que sepas qué tipo de persona es. {n}Horgus respira hondo.{/n} De todos modos, parece que mi oferta anterior fue algo insultante. Dos mil monedas de oro es.\""
  },
  {
    "Key": "93e8122c42ad42eea553345845569377",
    "SimpleName": "FriendlyAura.Description",
    "ProcessTemplates": false,
    "enGB": "You emit an aura of friendship that cause enemies to subconsciously hold back.\nEnemies within 40 feet take a –4 penalty on attack and damage rolls.",
    "ruRU": "Вы испускаете ауру дружелюбия, которая заставляет врагов бессознательно сдерживаться.\nВраги в пределах 40 футов получают штраф –4 к броскам атаки и урона.",
    "deDE": "Du strahlst eine Aura der Freundschaft aus, die Feinde unterbewusst zurückhält.\nFeinde innerhalb von 40 Fuß erleiden einen Malus von –4 auf Angriffs- und Schadenswürfe.",
    "frFR": "Vous émettez une aura d'amitié qui pousse les ennemis à se retenir inconsciemment.\nLes ennemis dans un rayon de 40 pieds subissent un malus de –4 aux jets d'attaque et de dégâts.",
    "zhCN": "你散发出一股友谊光环，使敌人下意识地退缩。\n40英尺内的敌人在攻击和伤害骰上受到-4的惩罚。",
    "esES": "Emites un aura de amistad que hace que los enemigos se contengan subconscientemente.\nLos enemigos dentro de 40 pies sufren una penalización de –4 en los ataques y tiradas de daño."
  }
]

[JohN100x1]

this was for 5 entries so for the full localisation it's about 750 entries which is 2 mins * 750/5 = 5 hours 😵‍💫

[kjk001]

So do we have native speakers that can say anything about the quality of those translations?

Or volunters to actually enter that data because I am certain John will go mad if he is the one that has to spend those 5 hours doing it alone

[Mythalar]

I'm French so I will go for this one :p The translation is actually quite accurate.

I could do a review of the translated text to check errors (or the correct use of pathfinder vocabulary)

[Necht]

Maybe i can help with the translation into russian. But only in summer.

[JohN100x1]

I was able to get the localization script to work (took several days due to rate limiting from chatgpt). The translations will still need to be proofread.

Here's the localization pack: https://github.com/JohN100x1/IsekaiMod/blob/JOHN_5-1-0/IsekaiMod/Localization/LocalizationPack.json

[Mythalar]

Tomorrow is a holiday in France so I'll look into this.

Should be quite easy, even with so many prompts, since most have official pathfinder translations.

[JohN100x1]

My automatic localization pipeline finally worked. I feel so proud. 😊

[Mythalar]

I made all translation corrections I could see in the pack.

There must be some typos and I have to check if the conditions are well translated (since a number of them are very similar) but I think I've made quite a good job for the French part.

(and my eyes are bleeding...)

[Mythalar]

@JohN100x1 I don't know if you had my pull request.

I've made the modifications in one go and had some issue at the end (session expired I guess), if need be I saved everything.

Edit : had not send the actual request, it was a loooong reading indeed x)

[JohN100x1]

Cool I will take a look

[Necht]

This is a good translation, you did a great job! But there are mistakes in the game terminology.