[Feature Request]: Generate a translation preparation file

[wadhwamatic]

Provide a clear and concise description of what you want to happen.

Adding translation text is a tedious process right now that requires reviewing the UI and finding text which may or may not be translatable. The process is better now that we have the shared layers concept but could still be further improved.

I would like to be able to run a script that would automatically find all translatable text in the UI including text that has been included in the configuration process. The script should create a json file that includes an entry for every translatable text field. I would then work on that locally and include the translations in the json file used in PRISM.

Is there anything else you can add about the proposal? You might want to link to related issues here, if you haven't already.

No response

[ericboucher]

@wadhwamatic not sure if you remember but you can turn TRANSLATION_DEBUG to true in the code to help with that. I think we can reasonably create a script as you suggested but it would only cover "fixed" text and would not be able to account for info defined in layers.json and other files

[wadhwamatic]

Thanks for the reminder @ericboucher. I'm wondering why a script couldn't also get the layer config details too if we provide the country as in input?

[ericboucher]

@wadhwamatic we probably could but it would likely be a lot more complex if we want to be holistic and precise. But if we just assume that we want all the texts in layers.json translated for example, then that's a bit easier.

[wadhwamatic]

@ericboucher - I think it's safe to assume we want all text in layers.json for active layers in prism.json to be translated. It should include layers that came from either the local layers.json or the shared layers.json.

[gislawill]

Following up here after I dug into this a bit in #1273. I feel a similar frustration during development. Specifically, I have a hard time feeling confident that I'm maintaining translation coverage without a fair bit of manual qa. It feels like there's got to be a better way (though I definitely acknowledge it's not a simple problem).

I think there are two related problems we're trying to solve:

1. Make it easy for Amit to find what needs translations
2. Give visibility during development and PR reviews to translation coverage for a feature

I believe the solution to both involves some method of identifying all translation keys in use (which is hard) and identifying all translation keys available (a bit easier). The difference between these two are our missing translation keys.

The multiple sources of copy (config files, in code), multiple sources for translations (country and language files), and varying feature set by countries makes static analysis very very tricky (Thanks @ericboucher for helping me understand that in #1273). I think it's fair to say that performing static analysis would be near impossible and we need to actually run the code for a country to know which keys we need and which are missing (please correct me if you disagree @wadhwamatic or @ericboucher).

That would leave us with two (not mutually exclusive) options:

• Building off the pattern from TRANSLATION_DEBUG to perform runtime analysis. In addition to logging the missing values, we could do any of the following (or other ideas)
  • make it easier to view summaries in dev + qa environments
  • track missing translations in tools like Sentry
  • record these missing values in the test environment and report them in a PR from our CI/CD tools
• A bigger, more fundamental, shift: we could stop using the English copy as our translation key. This means our translation keys would shift from "Some copy in english" to "some_copy" and we'd maintain an english translation file. This english translation file would theoretically be easier to maintain and would serve as the source of truth to compare the others against (to find missing coverage)
  • English would remain our source of truth but at least the sources of truth would consolidated to one json file.
  • this wouldn't solve all issues (varying feature set by countries for example)

[wadhwamatic]

Thanks for digging (deeply!) into this @gislawill. I am not sure I understand the second option you've suggested. I'll try to catch you for a quick call later this week.

[ericboucher]

@wadhwamatic what @gislawill is suggesting is to move towards the "standard" react way to do this and have translations keys that look like this:

  "client.profile.identity.maiden-name": "Nom de naissance",
  "client.profile.identity.legal-personality": "Personnalité juridique",
  "client.profile.identity.legal-personality.physical": "Particulier",
  "client.profile.identity.legal-personality.moral": "Morale",
  "client.profile.identity.politically-exposed": "Politiquement exposée",
  "client.profile.identity.birth-date": "Date de naissance",
  "client.profile.identity.first-name": "Prénom",
  "client.profile.identity.marital-status": "Situation maritale",
  "client.profile.identity.marital.single": "Célibataire",
  "client.profile.identity.marital.married": "Marié(e)",
  "client.profile.identity.marital.no": "Aucun",
  "client.profile.identity.marital.community": "Communauté",
  "client.profile.identity.place-of-birth": "Lieu de naissance",
  "client.profile.identity.department-of-birth": "Département de naissance",
  "client.profile.identity.zipcode-of-birth": "Code postal de naissance",
  "client.profile.identity.country-of-birth": "Pays de naissance",
  "client.profile.identity.gender": "Genre",
  "client.profile.identity.marital-regime": "Régime matrimonial",
  "client.profile.identity.nationality": "Nationalité",
  "client.profile.contact.address": "Adresse",
  "client.profile.contact.phone": "Téléphone",
  "client.profile.contact.postal-code": "Code postal",
  "client.profile.contact.city": "Ville",

So we would also maintain the English keys and therefore always have a good knowledge of the keys that are needed. It makes the code a bit less readable overall and doesn't solve for our problem with legends, layer titles and such but does make things more strict and a bit easier to track. And maintaining keys between our layers.json and translation files could very rapidly become a nightmare. But we can discuss with Amit. For example, layer descriptions could be defined in the translation files under layers.description.{layerId}:

Building on what your great recommandations @gislawill maybe we could maybe try an in-between for now and:

• Add all the English keys to a file with key === value but at least we can leverage it as a source of truth and have tests or something else complain as soon as a key is missing
• Clean up translation and really make sure that all the common app language and shared layers are translated only once in /shared for each language and validate that all the english keys are present.
• Then for each country, only keep the country specific translation with the same mindset, anen.json file with all the english keys.
• To start, having a simple "memory" during the tests that logs all the missing translations and generate a quick report at the end / publish as a PR comment could be a big help. Provided that we've done the above mentioned steps, building on TRANSLATION_DEBUG could be interesting

[ericboucher]

Another shift we could consider though less urgent is to leverage translations with keys instead of bit by bit translations like we are currently doing.

Eg. something like that:

  "support.info": "Il semble que votre compte {{email}} n'ait pas encore été activé"

And then the code looks something like this:

          <Trans values={{ email: user?.user.email }}>
            <Typography>{t("support.info")}</Typography>
          </Trans>

[wadhwamatic]

@wadhwamatic not sure if you remember but you can turn TRANSLATION_DEBUG to true in the code to help with that. I think we can reasonably create a script as you suggested but it would only cover "fixed" text and would not be able to account for info defined in layers.json and other files

@ericboucher - FYI - this doesn't work very well as the translation check doesn't seem to happen until the UI element is rendered on screen. So, if you don't click on the navigation and activate sub-groups, or click on charts and go to compare periods, etc. the translation debug doesn't catch missing translations.

[ericboucher]

I updated the translation debugger (in the vite PR) to also note when the translation is missing in english for now and also gets logged out when we run tests so that hopefully we can surface a bit more items.

Then for layers/legends data we should write a script that adds all the necesary keys and checks that their translations are available for all the languages listed in that specific country. I can give this a quick go this week.