Untangling the language files

[snipe]

Our language files are a bit of a mess. They work fine, but the directory/file structure was created long before this app got so big, and there are a ton of repeated translations that should be merged into the general translation file. This doesn't affect us much as programmers, but it's unfair to translators who have to translate the same string over and over just because language stuff got out of hand.

I've looked into a few packages that handle Laravel translations, and while many of them are close, I haven't found one that truly de-dupes and reorganizes the way I need.

We don't want everything merged into one file, obviously - no need to load consumable translations when navigating through the assets section (and vice versa), but I'd love to de-dupe the translation files to make the load we put on volunteer translators a little easier. I'm not sure if I have to write a package, or what the best course of action would be here. I'm sure I'm not alone in having an app where the translation structure made sense at the time, but grew much larger than was originally expected.

Looking for ideas, feedback, plans of attack, etc. Please don't just create a giant PR for this without talking to me though - there are thousands of files that would be affected.

[CronKz]

I could help with the german translation. It really bothers myself to see the same string technically, but translated in various forms.

[snipe]

@CronKz I'd be happy to set you as a proofreader of the German CrowdIn stuff. That's sort of a separate issue though - I'm talking more about the file structure of the language files themselves, duplicated entries, etc. Cleaning it up so that it makes it easier for translators to help without wasting their time translating the same string in different sections.

If you're interested in being a proofreader (where you can accept/reject translations, add your own, etc), let me know what your CrowdIn username is.

[rbk]

Are you looking to eliminate duplicate "word for word" translations or translations that include duplicate words like the ones below?

'Confirm_Accessory_Checkin' => 'Confirm Accessory Checkin.',
'Confirm_accessory_delivery' => 'Confirm accessory delivery.',

Codewise I like how the translations are structured. Maybe there is a way to update the duplicate translations in the code with a script?

What is the process of getting the translations back from Crowdin?

[snipe]

@richardbenjamin I'd definitely want to do it via script - no one is that much of a masochist :) Plus anywhere we change the location of the translation, we have to update it everywhere it's references in the functional/blade code. I don't trust myself or any other human enough to get that right by hand.

Definitely word-for-word translations (as long as contextually they make sense of course - order meaning to order something vs order meaning an order number, etc) - but also to standardize some of the file structure. Terms that apply to all or most things should all be moved into the root level general.php. The admin dir should probably go away, or be used only for admin settings. And the subfolders are probably not necessary anymore.

Do we really need a general.php, a message.php, a form.php and a table.php? A lot of the words/phrases are reused across those. (You can make an argument for messages.php staying, as semantically it makes sense, but the other stuff seems unnecessary.)

Re: crowdin, I think I can reorganize the source files and because it already has the existing translations in their DB, it will just re-apply them. I think.

[snipe]

I'm also interested in consolidating some of the repeated phrases that we can handle via translations with variables. For example: "The :item ID :id is invalid". We can pass variables to that string and have it be one string to rule all invalid IDs ("The asset ID 3 is invalid", "The consumable ID 49 is invalid", etc) but we'd need to 1) delete the old individual strings, 2) update the code to use the newly created string and 3) make sure we were passing variables to that new string every time. It's a lot of work, but overall I think it would be worth it.

My worry there, other than human fallibility, is that some languages may not translate well that way (I'm thinking of Japanese, where they use different words to quantify small round things than they do to count flat things, etc.)

[snipe]

I feel like something like https://github.com/highsolutions/laravel-translation-manager is close-but-not-quite for what we need.

[CronKz]

@snipe I've logged into CrowdIn with my Github account. And yes I can do the proof reading for the german translation. Ok, so to untangle the files we have to think of a whole new structre for the general translation files and consolidate duplicated strings to one general file. But where do we start? How about listing all the strings in one file and then manually deduplicating them?

[rbk]

@snipe What exactly is missing from the translations manager you found (https://github.com/highsolutions/laravel-translation-manager)? I like the idea of storing the translations in the database and generating the language files. If we can get everything into a data model, we can dedupe the translations without a lot of manual work. I could write a script to insert the translations into a database table.

In updating the code base, I ran a script and found there a 2368 uses of the translation function: grep -ro "trans(" . | wc -l | xargs echo "Total matches :" We can update these locations via a script if we know the new structure. We would need to make a key value map for the new locations of the translations and then run a script to update the code base (scary!).

[snipe]

@richardbenjamin namely that it doesn't do any of what we need lol. It could be a replacement for crowdin, but it's really meant for managing+adding translations, not restructuring existing ones.

I like the idea of storing the translations in the database and generating the language files

I mean, as a one-time thing, maybe - but not in perpetuity. How then do I provide those translations to users? How do we handle conflicting ideas of what a translation should be? Etc.

Also, there may be some remaining Lang() uses - Laravel updated that kinda recently, and the old facade still works, so there may be some lingering around. I've been updating them to trans() as I go, but this is a big app and I'm sure I missed some somewhere.

[CronKz]

@snipe What I've seen so far is that we could make the sentences used more modular to shrink the ammount of strings. For example the sentences in every "message.php". "Department was not created, please try again." "Company was not created, please try again."

This could be split up into 3 Parts. [Company] = "Super Duper Inc." [was_not_created] = " was not created." [please_try_again] = " Please try again."

trans([Company], [was_not_created], [please_try_again]) trans([Department], [was_not_created], [please_try_again])

So these fields could be used in one "general-messages.php" file. This case would then work for every other case, at least for german. I don't know if it would work for all other languages.

There are plenty of more cases this would apply too.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

[stale[bot]]

Okay, it looks like this issue or feature request might still be important. We'll re-open it for now. Thank you for letting us know!

[stefanp2021]

I'm looking for the place to translate the Actions correctly: Where can I translate this? Thanks!

[marcusmoore]

@stefanp2021 I think those will be in here (de-DE) or here (de-if).

[stefanp2021]

@stefanp2021 I think those will be in here (de-DE) or here (de-if).

Yes but where? I can't seem to find the right string. Thanks.

[marcusmoore]

@stefanp2021 I'm sorry. I'm thinking about my process of adding new english strings to the application. You'll want to ignore what I said and check out the docs on contributing translations.

(My process is different because I'm contributing source code and need to set the original translation strings in english)