search

joomla-extensions / weblinks

This repo is meant to hold the decoupled com_weblinks component and related code.
GNU General Public License v2.0
45 stars 88 forks source link

Language packs and translations #2

Closed eddieajau closed 10 years ago

eddieajau commented 10 years ago

Need to talk to the TT's about how decoupled extensions are going to work in terms of translations.

infograf768 commented 10 years ago

@daviddeutsch Existing tools do NOT fill our needs, and especially OT. https://github.com/joomla-extensions/weblinks/issues/2#issuecomment-39862370 (BTW, I just checked the translation in French for one of Bakual's extensions on OT : it is real bad. Not only typos but values which directly come from Google Translation, some of which are totally wrong, full misinterpretation...)

To be brutally frank: I think joomla has lost it's bargaining position to ask in that manner. People are not coming to the table. That's the problem. Maintaining the status quo is not how you fix that.

I will not be brutal, just frank and no bargain: goodwill people come to the table, enjoy it and help. Otherwise I would have been alone to take care of languages since Mambo times. The project IS concrete and would help. You are free evidently to feel differently.

@Hils We have not found a tool that fully filled our needs. The best to date I am aware of is https://webtranslateit.com/

Hils commented 10 years ago

@infograf768

Existing tools do NOT fill our needs, and especially OT.

I am desperately trying to continue to be polite and to discuss this matter agnostically. Perhaps you would do the same and refrain from adding your 'opinions' please?

toretto commented 10 years ago

"(BTW, I just checked the translation in French for one of Bakual's extensions on OT : it is real bad. Not only typos but values which directly come from Google Translation, some of which are totally wrong, full misinterpretation...)"

So you saw a problem, and chose to ignore it? Good job, JM.

infograf768 commented 10 years ago

I am not an OT Translator. In case you do not know, I do already quite a lot for Joomla. I told Bakual about it.

Hils commented 10 years ago

Let's stop this - it doesn't matter who the heck anyone is - WE SHOULD HAVE A COMMON GOAL - internationalisation for Joomla and peripheries

daviddeutsch commented 10 years ago

Existing tools do NOT fill our needs

And I'm still curious to hear why.

I will not be brutal, just frank and no bargain: goodwill people come to the table, enjoy it and help. Otherwise I would have been alone to take care of languages since Mambo times.

Times are changing, people go where things are interesting and moving forwards. How much has the translation setup changed 'since Mambo times'? How come you are so sure it's still a good, competitive setup and that com_localise would help? Again: Where is the cost-benefit analysis?

@Hils

I am desperately trying to continue to be polite and to discuss this matter agnostically.

Agreed, let's use "Transifex" from now on instead of OT. Which brings me to: @infograf768 - How is Webtranslateit more suitable than Tansifex? Can we please get a discussion going on the actual merits of existing solutions? I have no idea why you keep speaking in the abstract when we could be having a productive discussion.

toretto commented 10 years ago

"I am not an OT Translator. In case you do not know, I do already quite a lot for Joomla. I told Bakual about it." In that case, it would only be suitable if you stop mentioning OT in every 2nd comment you made. This isn't a debate about OT but you keep bringing OT up and wonder why we come here to comment...

infograf768 commented 10 years ago

I will stop mentioning anything as well as posting here. I made my case clearly and we are ready to test anything that fits PLT, lets get correctly translated strings and not make us lose languages for any "core supported" extensions.

daviddeutsch commented 10 years ago

I made my case clearly and we are ready to test anything that fits PLT

I'm sorry, but you have not done the former and the way you argue in this thread does not support the latter.

lets get correctly translated strings and not make us lose languages for any "core supported" extensions.

I don't know what that means.

wilsonge commented 10 years ago

@daviddeutsch I think you're being very provocative here. I don't know how much of the above you've actually bothered to read but JM has listed at least 3/4 things that are not right with transifex (for example the issues with pluralisations here: https://github.com/joomla-extensions/weblinks/issues/2#issuecomment-39862370)

horus68 commented 10 years ago

on @daviddeutsch https://github.com/joomla-extensions/weblinks/issues/2#issuecomment-40367744 There are several teams that really work with different systems: See: As a component developer, what are my translation options? - http://forum.joomla.org/viewtopic.php?f=11&t=786377

Our pt-PT community used Webtranslate-it https://webtranslateit.com (and also several other platforms) since long time. Since 2 years we opted only by Transifex https://www.transifex.com and moved all our projects into there. Even i'm not the one to make an updated compare on systems, why did we move?

Note: OT is not Transifex... and there are many projects using the translators teams from OT and others using their own translation teams. You can set up a project using just one translator for each language, is up to the project administrator

It works for us, we have now more then 100 extensions translated with zip to be installed for pt-PT see here: http://translations.joomlapt.com/languagepacks There are issues? sure. But you can also be a bad translator using just notepad++

I believe that coders would prefer WTI but most translators, me included, are not coders, so we are the ones limited to use the things that are published. See my tutorials on Transifex with joomla: Keep Calm and Translate Joomla - https://sites.google.com/site/transjoomla/

Hils commented 10 years ago

@George Paulo just dealt with plurals in his post

daviddeutsch commented 10 years ago

@wilsonge I have read everything in this thread since it started and the reason why I decided to participate was because it was frustrating to see it drifting into the abstract and speculative.

The arguments voiced against Transifex were cleared up, with the exception that you mention. For that, I have found this. So it's either still being worked on, or could at the very least be resolved with community input to Transifex. Also: If, for instance, Webtranslateit solves this issue, that would be interesting to hear.

I suppose, that is my problem with this discussion: Instead of stating issues and solving them, issues are being stated and used as argument stoppers. Why not, you know, work this out properly?

Finally: As I've said in my first comment in this thread: Why not put a little more trust in what developers around joomla (3PDs) have done for years and years. They have been working on these problems with the tools and/or the tool providers and have thus already done the work that is claimed to still be ahead of joomla.

wilsonge commented 10 years ago

I'm not saying there aren't compelling reasons to move to transifex (not even using OT) - I was the first one to bring up transifex in this discussion.

Also with the commented out lines - @toretto talked about having to use the API - that isn't ideal for all our TT's - some of them may well not be dev's (I know he also mentioned adding them manually - but that's not exactly the most ideal thing when they're sitting there as comments either!)

However I think we need to get in contact and fix the remaining issues for sure before we can start using it for these core extensions. We can't force the TT's to move to something that doesn't support everything we need. But I agree it's just something that needs smoothing out a bit! Maybe if we can smooth out these issues in advance then JM would more willing to consider it?

AmyStephen commented 10 years ago

@wilsonge - What is the list of issues? It would be helpful to extract clear points into a list.

@infograf768 How does the Joomla project ensure quality translations?

Bakual commented 10 years ago

Paulo just dealt with plurals in his post

@Hils @horus68 The post states that Transifex can manage them for PO and XML files. Can it do it for Joomlas INI files? I don't know. Does the portugese language use special plurals like the russian does? Or just the regular "zero / one / more" stuff like we have in german and english?

@ all: Personally I think that the translation process sure needs some love. But I don't think that this is something which can be changed in 3 months. It is very likely out of scope for the decoupling of com_weblinks as it just produces to many issues within the community and workflows which need to be settled first. A deadline in July doesn't help for that process. It's still something we need to find a solution for, and which needs brainstorming for. Let's take that as a separate topic for our roadmap which doesn't have to be solved for 3.4.

The decoupling of com_weblinks is meant as a project to detect those issues. PLT didn't expect that it works flawlessly. Let's do what is possible to do for 3.4. And think about ways to improve and solve the issues which are not that easy to solve. Those would be medium- (3.5, 3.6., 3.x) or even longtime (4.0) plans.

However I'm not sure if this platform is the best place to solve that puzzle. The picture is becoming much bigger than just com_weblinks.

eddieajau commented 10 years ago

@Bakual I have no doubt we can come up with a working solution in a short time and I did expect we could come up with a good plan to completely decouple an extension in the time allocated.

How about this:

  1. We limit this trial only to Weblinks until it is proven it can work (which could be 3.6, 3.7, whatever).
  2. We continue to brainstorm how language packaging can work in completely independently
  3. We move forward to trial the best idea or ideas
  4. We make a call about the readiness of those ideas come 28 May

Sound fair?

The reality is the code part is more or less finished. The only things left to do are to sort out wiring up the build process, sorting out documentation and this.

So let's get back to business. We can store all language files in the repo. It may be easier to collect all the language files in a central folder per language, it may not - I'm not sure (need input from all here). The major problem I think we need to solve is how to publish individual extension language packs since I think we all agree now that shipping all languages with the extension is not practical (although, it is only disk space). If we solve that problem we also solve it for the 3PD's and that is a good thing. We need a home for where translation packs for extensions can be listed per extension, and I think we should eventually extend that to any 3PD extension.

wilsonge commented 10 years ago

@daviddeutsch OK http://support.transifex.com/customer/portal/articles/1070366-support-for-plurals this is the issue for example - transifex only supports those features for .po not .ini files

AmyStephen commented 10 years ago

@wilsonge Do we have an example? I believe @mbabker uses Transifex for JIssues. Let's try a plural and see how it works. I'm thinking that it is possible it works just fine -- but, maybe it doesn't. The only way to know for sure is to try it. Really good to have specific issues otherwise it's not clear where one would get started.

mbabker commented 10 years ago

JIssues is also using .po files.

AmyStephen commented 10 years ago

@horus68 @hils @toretto - Do you have an example of Joomla ini file that uses a plural to share? Agree with @daviddeutsch that concrete examples of issues are best. If this already works, sharing an example of it working should help resolve concerns.

AmyStephen commented 10 years ago

I just did a quick search on some of @horus68 's files and there are definitely translated plurals. It appears to me that Joomla's %s approach is the same as .PO, so, maybe when Transifex says it works for .PO -- it also works for Joomla's special .ini files?

Does that help resolve concerns? Or, am I missing the point? (I am not a language expert, that's for certain.)

brianteeman commented 10 years ago

You're missing the point. The plurals issue has been well described in this thread already

AmyStephen commented 10 years ago

That's entirely likely, @brianteeman -- would you kindly link me to the comment that describes the issue in some detail? Thanks! =)

wilsonge commented 10 years ago

@AmyStephen https://github.com/joomla-extensions/weblinks/issues/2#issuecomment-39862370 Read the bit JM addressed to me here. The issue is in non-latin dialetcs for this particular issue (rather than something like spanish/portuguese/french which is my straightforward translations).

AmyStephen commented 10 years ago

@wilsonge - regarding lines 37-49 of: https://github.com/joomla/joomla-cms/blob/master/administrator/language/en-GB/en-GB.plg_captcha_recaptcha.ini#L37"

@toretto addressed that point a few comments above @infograf768 and @nicksavov

Again, in looking at @horus68's translations, he has indeed translated those commented out lines.

So, if I understand the point is that using the API or manually adding those lines is considered to be a significant problem? (Not mocking - asking.)

The second issue then on the plurals is a question as to whether or not languages that require another line for plurals can manually add them. Is that correct?

If so, are those the two issues we have identified for Transifex? (Or, are there others?) Just trying to get a good handle on the problems. When some are pointing at that post to say plurals work and others are saying not for Joomla's ini's -- it's clear -- not everyone is understanding things the same. At least we can iron that out.

Thank you.

wilsonge commented 10 years ago

Yes they are definitely two issues. A lot of the translators as I understand are not devs in any way. So adding the lines via an API is just bad. Manually adding is OK I guess as long as the translators can easily view the commented out lines (I genuinely don't know how easy that is - I know comments on lines are easy - but don't know about multiple commented lines).

And yes the second issue is definitely an issue - from what the API says it's not possible for ini files (and before people mention it I think many arguments have been had over this before so let's no bring up the .po vs. .ini argument :P)

Another issue JM raised with me was that translation teams are allowed to add their own copyrights to the top of their lang packs. But apparently transifex can't deal with this - it will only copy the copyright from the existing file.

By the way JM also said there's always manual testing of these packs before they are released so I think we should also have a transifex integration built into com_localise as well so that TT's can easily download these packs and test them without having issues of needing to use a CLI script to download the translated files.

I'm not a translator - that's JM's role :P But maybe he can raise if they are anymore issues. But those 4 are definitely a good place to start

mbabker commented 10 years ago

If folks are willing to work together, almost every issue being highlighted can be addressed. We can work with Transifex to request support on things lacking that others could benefit from and use our own code to make things simple. I already maintain a simple Transifex API wrapper (which is used in the issue tracker) that could seemlessly hook into com_localise if we would actually follow through on something like that. If we'd stop nitpicking, finding reasons to argue, or shooting down any idea that isn't the current status quo, all of our time would be better used.

On Tuesday, April 15, 2014, George Wilson notifications@github.com wrote:

Yes there definitely two issues. A lot of the translators as I understand are not devs in any way. So adding the lines via an API is just bad. Manually adding is OK I guess as long as the translators can easily view the commented out lines (I genuinely don't know how easy that is - I know comments on lines are easy - but don't know about multiple commented lines).

And yes the second issue is definitely an issue - from what the API says it's not possible for ini files (and before people mention it I think many arguments have been had over this before so let's no bring up the .po vs. .ini argument :P)

Another issue JM raised with me was that translation teams are allowed to add their own copyrights to the top of their lang packs. But apparently transifex can't deal with this - it will only copy the copyright from the existing file.

By the way JM also said there's always manual testing of these packs before they are released so I think we should also have a transifex integration built into com_localise as well so that TT's can easily download these packs and test them without having issues of needing to use a CLI script to download the translated files.

I'm not a translator - that's JM's role :P But maybe he can raise if they are anymore issues. But those 4 are definitely a good place to start

— Reply to this email directly or view it on GitHubhttps://github.com/joomla-extensions/weblinks/issues/2#issuecomment-40490515 .

horus68 commented 10 years ago

A- On Plurals rules and Joomla:

B- On plurals and Transifex: Supported but not yet for INI files Transifex do not have pluralization support for INI files. It works for other files so you need to ask them and be involved in order transifex recognize a pluralized entry on INI files.

BTW: it works as a single field to translate but with more then one box for it, so you have to translate both boxes before you are able to save the line!

How to solve the issue:

C- On comments and Transifex: Supported but not yet for INI files Issue detailed here by JM: https://github.com/joomla-extensions/weblinks/issues/2#issuecomment-39862370 regarding comments on files like this (line 20 to 24) https://github.com/joomla/joomla-cms/blob/staging/administrator/language/en-GB/en-GB.plg_twofactorauth_totp.ini

Transifex can recognize comments on language files and display them on the box where you will translate the line. But this is not implemented for INI files

horus68 commented 10 years ago

D- On copyrights and Transifex: There is no issue here Actually some teams had been including extra lines (or removing the ones from en-GB) language files to include their copyrights (usually they are not for the person that actually translate each file but for the community that creates the language package)

E- On automatic translations and Transifex: There is no issue here

F- On quality translations and Transifex: There is no issue here If you want that only the TT should translate the decoupled extensions, then you can also do that using Transifex or any platform!

AmyStephen commented 10 years ago

I'd be happy to help with any development work needed -- if help is needed. Just let me know who to talk to get started. I'll need direction and feedback. @horus68 - thanks for going thru the list and summarizing!

wilsonge commented 10 years ago

RE: D Copyrights for INI files should not be added by hand. Those are joomla code files and the headers must be the same as the en-GB files! Copyrights are in the XML files.

As you said the copyrights should not necessarily be the same - TT coordinators often add their own copyrights in addition to OSM and this is perfectly OK according to JM (obviously the rest must be the same). It's possible maybe to integrate this into the com_localise component?? But it's definitely an issue

So we need to solve point A through D (A-C with transifex) and work out whether D should be via transifex or the com_localise in development and as you say if we aren't going to be with OT (which I think is completely off the table according to JM) then as you say E and F aren't a problem

So we need to get in touch with transifex and also try and work through getting com_localise up and running.

wilsonge commented 10 years ago

P.S. @AmyStephen if you wanna help with dev work we're working on bringing back com_localise - it was dropped during the middle of 1.5 to 1.6 conversion work. So if you (or indeed anyone else) wants to help https://github.com/joomla-projects/com_localise

AmyStephen commented 10 years ago

Thanks @wilsonge -- I'll fork it and take a look. I'll target you with questions (so don't hide!)

Hils commented 10 years ago

@nonumber has solved this for his translations - this is one of his french translations: https://opentranslators.transifex.com/projects/p/nonumber-advancedmodulemanager/viewstrings/#fr_FR/com_advancedmodules/1818995

wilsonge commented 10 years ago

@Hils do you know how this works back into the folder? Is this just a frontend visable copyright rather than the copyright of the files?

On 15 April 2014 19:27, Hils notifications@github.com wrote:

@nonumber https://github.com/nonumber has solved this for his translations - this is one of his french translations: https://opentranslators.transifex.com/projects/p/nonumber-advancedmodulemanager/viewstrings/#fr_FR/com_advancedmodules/1818995

— Reply to this email directly or view it on GitHubhttps://github.com/joomla-extensions/weblinks/issues/2#issuecomment-40516726 .

ghost commented 10 years ago

I have a custom script on my local machine that converts that language string to a comment line, like: https://gist.github.com/nonumber/10764726 So there is no one-on-one solution AFAIK. It needs some post-processing.

eddieajau commented 10 years ago

@all, the relative merits of individual editors and tools is off-topic. This is not the problem we are trying to solve.

The issue is to devise a new way to manage, build and publish language packs for individual extensions. Solving this problem will also greatly benefit the 3PD community.

In order to eliminate some variables for now, let' just assume that all language packs for Weblinks are stored in this repo and we can build them somehow. When someone installs Weblinks, how will they be able to find the language pack for it? Do we put that information in the XML manifest for the package so that the Joomla Extension Installer is aware of? Can we do something generic to simply suggest other extensions to install (cf https://getcomposer.org/doc/04-schema.md#suggest) and that could include any type of extension (that would actually be really cool)?

Thinking caps on please ...

Hils commented 10 years ago

The issue is to devise a new way to manage, build and publish language packs for individual extensions.

If I understand correctly, you want us to completely rethink translating for Joomla? If so, should Joomla translations be dependent upon a decision that must have been made nearly ten years ago, to use what are known as the restrictive "Joomla .ini" files rather than the universally acceptable .pot files for example? Could someone please explain the logic in that decision as it applies today?

eddieajau commented 10 years ago

If I understand correctly, you want us to completely rethink translating for Joomla?

Only in so much as 3PD's already have to find their own solutions,

Could someone please explain the logic in that decision as it applies today?

Yes. It's quite simple. You can't load more than one .pot file per page.

wilsonge commented 10 years ago

https://github.com/joomla-projects/translate-joomla/blob/master/README.md#why-we-use-ini-files-instead-of-po-mo There's a further list of reasons JM drafted up here. I don't think we could consider .pot files before J4 and even then there are definite issues.

Although out of interest how does WP work if you can't load more than 1 file a page? They have extensions like us?

Bakual commented 10 years ago

If I understand correctly, you want us to completely rethink translating for Joomla?

I understand it more in distributing the translation, not translating itself.

With our current system, you either ship all available languages with your extension, or you make language packs. First variant makes the extension bigger than needed and ties extension releases and language updates (or new languages) together. Second variant is more flexible, but the user needs to find, install and update the language pack manually in a separate step after installing the extension.

We likely want a third - yet to invent - variant.

wilsonge commented 10 years ago

+1 to what @Bakual says. But I think creating the packs in a different way requires something different. Because current methods are geared towards producing everything in one massive pack

eddieajau commented 10 years ago

Although out of interest how does WP work if you can't load more than 1 file a page? They have extensions like us?

The only practical way for us to use .pot's is to assemble one huge file for each language. But you have to do that for each site and you'd have to rebuild the file each time there was a change in language string (similar to how LESS works for CSS). And if you did that, the format of the base language files is irrelevant because you just convert it to .pot anyway.

I think that would be a good experiment for someone to do, but I would not advocate changing the .ini standard for 3.x (actually, I don't see a case for changing it at all, but I wouldn't stop someone from trying it).

In the mean time, what if we simply used a post-install message to show a list of the language packs that are available? Click on the links to automatically install them?

AmyStephen commented 10 years ago

Maybe just install the languages (and always update)?

The files will just be ignored unless Joomla assigns the language to the user object.

On Wed, Apr 16, 2014 at 4:50 PM, Andrew Eddie notifications@github.comwrote:

Although out of interest how does WP work if you can't load more than 1 file a page? They have extensions like us?

The only practical way for us to use .pot's is to assemble one huge file for each language. But you have to do that for each site and you'd have to rebuild the file each time there was a change in language string (similar to how LESS works for CSS). And if you did that, the format of the base language files is irrelevant because you just convert it to .pot anyway.

I think that would be a good experiment for someone to do, but I would not advocate changing the .ini standard for 3.x (actually, I don't see a case for changing it at all).

In the mean time, what if we simply used a post-install message to show a list of the language packs that are available? Click on the links to automatically install them?

— Reply to this email directly or view it on GitHubhttps://github.com/joomla-extensions/weblinks/issues/2#issuecomment-40656931 .

wilsonge commented 10 years ago

If you auto-install languages then you still have the issue Bakual describes about having to update them separately to updating the lang pack separately. Also how you planning on automatically installing the language packs?

eddieajau commented 10 years ago

I think we are confusing terminology here. A language pack would be a separate language extension that you would install. To include all language as Amy suggests, you just include them in the language folders just as any 3PD would do. To be honest, the latter is the simplest solution available to get things done. But the disadvantage is you will be chewing up disk space. Not a huge issue for one extension, but when you have 5, 10, 15 times 60+ translations, it would add up I suspect so it's not even a good short term solution because 3.5 is going to decouple even more extensions.

On the other hand, managing 60x optional language packs is no easy feat either without a lot of automation. Aside from that, each Github release is going to have 60+ files attached to it with a full complement of translations.

Soooo, if we look at the problem from another side - should we centralise on the language, not the extension and have a repo for each language, and that repo supports languages for many extensions (maybe even the core translations)? Thinking out loud, something like: http://github.com/joomla-extensions/fr-FR

eddieajau commented 10 years ago

But then you are installing languages for extensions you don't need and we are back to the same problem we have of including the base en-GB files in core ....

AmyStephen commented 10 years ago

Looks like the CMS is sitting at around 55 MB. Less than 2 MB is language files. I can't see this getting to be a big problem disk space wise. These are small files.

Admin: 12 KB en-GB.com_weblinks.ini 4 KB en-GB.com_weblinks.sys.ini

Site: 4 KB en-GB.com_weblinks.ini 4 KB en-GB.mod_weblinks.ini 4 KB en-GB.mod_weblinks.sys.ini

Personally, I would keep it simple and ship all the "extension things" together. Install them together. Update them together. Remove them together. Easiest for version management, too.

The goal is flexibility -- so, there will also be savings from not installing extensions people do not use.

Easier for users to have the language in place when they add a core language.

On Wed, Apr 16, 2014 at 5:25 PM, Andrew Eddie notifications@github.comwrote:

But then you are installing languages for extensions you don't need and we are back to the same problem we have of including the base en-GB files in core ....

— Reply to this email directly or view it on GitHubhttps://github.com/joomla-extensions/weblinks/issues/2?utm_campaign=website&utm_source=sendgrid.com&utm_medium=email#issuecomment-40660190 .

eddieajau commented 10 years ago

These are small files.

Remember to multiply that by, potentially, 60+. You'd be adding 2-3 meg of just-in-case language files per extension. But when you consider how much space image galleries chew up, probably not a huge issue.