Move translation process to TMS

[Andrulko]

Hey,

It's Andriy from Crowdin, a localization management platform. Quite recently I discovered WinMerge (thanks to Ninite 😊) and it helps me basically every day now! There is always something to compare/validate in my work.

After noticing some inconsistent Ukrainian translations / untranslated segments, I was curious to see how the existing l10n process is organized and found this guide: https://github.com/WinMerge/winmerge/blob/master/Translations/README.md

I believe it works but it may be too complex for the regular user and it's causing many syntax errors I believe so you need to solve a lot of issues before publishing translations live.

Did you think of using TMS for this purpose? Let me illustrate how it may work:

a) Create a project in Crowdin (it's free for open source and license request can be submitted here)

b) Connect your GitHub repository with Crowdin, so any changes made to English.pot would be synced automatically and Crowdin will create localized files. To make it simpler, I have already prepared the configuration file in my forked repo: https://github.com/Andrulko/winmerge/blob/master/crowdin.yml

Of course, all existing translations will be imported.

c) Translations are delivered to the repository as a PR, you only need to merge it. Here is a sample in my test repo: https://github.com/Andrulko/winmerge/pull/2

Temporary, I've made this demo project public, so you can see how people will see it: https://crowdin.com/project/wmdemo

There is also Crowdin Enterprise available (free for OSS too), it has different look and you can check Discourse organization: https://discourse.crowdin.com/

I've noticed that WinMerge docs are translated into Japanese only, so you can also launch a documentation translation effort (Markdown files are well-supported too)

Please let me know how it sounds, I truly wish WinMerge will get a simpler localization process soon :)

[Gitoffthelawn]

Hi Andriy! How is Crowdin funded, or is it a volunteer project?

[sdottaka]

Thank you for your suggestion. I'll try it later. I would like to give each translator a choice between using a TMS like Crowdin and the traditional method.

[Andrulko]

Hi Andriy! How is Crowdin funded, or is it a volunteer project?

It's a privately held company (LLC). You can see the existing users here: https://crowdin.com/page/customer-testimonials

Crowdin is free for Open Source since it was launched back in 2009. Translations can be made by invited translators/LSPs/volunteers.

All public projects are also available here: https://crowdin.com/projects#showcases

Thank you for your suggestion. I'll try it later. I would like to give each translator a choice between using a TMS like Crowdin and the traditional method.

Sure! It would be also possible to use GitHub sync for users who translate via TMS + translations made in the GitHub repository directly can be automatically imported to your translation project. It's a pretty new feature btw :)

[Gitoffthelawn]

Hi Andriy! How is Crowdin funded, or is it a volunteer project?

It's a privately held company (LLC). You can see the existing users here: https://crowdin.com/page/customer-testimonials

Crowdin is free for Open Source since it was launched back in 2009. Translations can be made by invited translators/LSPs/volunteers.

All public projects are also available here: https://crowdin.com/projects#showcases

Thanks Andriy. I've heard good things about Crowdin, and have contributed some translations using it over the years. I've never had the time to look too closely at it, but I'm personally taking a closer look now.

You mentioned that Crowdin has been free for Open Source since it was launched back in 2009. That's an impressive 13 year track record. As an active member of the OSS community, I thank you.

Is Crowdin committed to continuing their current free offerings to open source projects for the foreseeable future?

[Gitoffthelawn]

BTW, maybe using Crowdin would help with #992?

[Andrulko]

Is Crowdin committed to continuing their current free offerings to open source projects for the foreseeable future?

Forever, without limitations (the only requirement - your project will be public). Crowdin Enterprise is also free for OSS, organization can be created here: https://accounts.crowdin.com/workspace/create

If the current plan is to use TMS for 1 project, then I propose to start with crowdin.com and in a while, you can migrate to Enterprise once your Crowdin ninja skills will level up :)

Yes, there would be only 1 pull request with all translations inside. Here is an example: https://github.com/Andrulko/winmerge/pull/2

Besides the native GitHub Connector, there is also a possibility to use GitHub Action: https://github.com/crowdin/github-action

[Gitoffthelawn]

@Andrulko Thank you for all your answers and info.

I took some time and looked at the details of Crowdin. It looks quite good. I may use it for some of my projects in the future (WinMerge is not my project), and I have added it to my list of recommended services.

@sdottaka Do you see any big downsides to Crowdin, besides the initial time-commitment to familiarize yourself with it and get it set up? It sounds like @Andrulko may be a great resource to minimise that time commitment, which is a pleasant opportunity.

[sdottaka]

I will try after version 2.16.24 is released.

[Andrulko]

@sdottaka that's great to hear! ❤️ If you need any further assistance, please let me know.

You can try the configuration file I shared earlier for the GitHub sync 👌 Alternatively, Crowdin CLI or Crowdin GitHub Action can be used to automate files sync

[sdottaka]

First, I created the following project and used crowdin-cli.

https://crowdin.com/project/winmerge

I noticed two things.

• WinMerge uses an older version of InnoSetup to create installers for Windows XP support. It does not support UTF-8, so the encoding of the .isl file and the Readme file displayed within the installer are legacy encodings such as CP-1252. When I downloaded translated files from crowdin.com with the crowdin download command, those files were automatically converted to UTF-8.

• We have embedded the comments in the .po files and use them, but when I download them from crowdin.com with the crowdin download command, they seem to disappear as shown in the attached image below.

[Andrulko]

Hi @sdottaka, thanks a lot for sharing comments :)

WinMerge uses an older version of InnoSetup to create installers for Windows XP support. It does not support UTF-8, so the encoding of the .isl file and the Readme file displayed within the installer are legacy encodings such as CP-1252. When I downloaded translated files from crowdin.com with the crowdin download command, those files were automatically converted to UTF-8.

UTF-8 is applied by default, yes. We can customize import/export on our end (in the near future, end-users will get this possibility), so encoding will match. Just to confirm, you would like CP-1252 to be applied to all languages in all *.isl files?

We have embedded the comments in the .po files and use them, but when I download them from crowdin.com with the crowdin download command, they seem to disappear as shown in the attached image below.

Hmm, do you refer to headers at the top of PO file? Can you please let me know what headers are the most critical for you? I'm not sure if we can keep headers like "X-Poedit-Country" because these are generated by Poedit and Crowdin allows multiple people to be involved (from different locations) in translation process, so this attribute may be not very accurate and helpful.

[Andrulko]

Just noticed that Cyrillic chars were corrupted during import as well 😬 https://crowdin.com/translate/WinMerge/16/en-uk

We should properly convert them to UTF-8 on import (to make them readable within Crowdin) and on output, CP-1252 encoding should be applied

[sdottaka]

UTF-8 is applied by default, yes. We can customize import/export on our end (in the near future, end-users will get this possibility), so encoding will match. Just to confirm, you would like CP-1252 to be applied to all languages in all *.isl files?

The .isl file uses a different encoding code page for each language. For example, Ukrainian.isl uses CP-1251 and Japanese.isl uses CP-932 (ShiftJIS).

I think we'll have to give up on WindowsXP support after a while, so when we stop supporting WindowsXP this won't be an issue.

Hmm, do you refer to headers at the top of PO file? Can you please let me know what headers are the most critical for you?

We generate pages on https://winmerge.org/translations/?lang=en using translator information in comments like:

# Translators:
# * Warrior <warr11r at rambler.ru>
# * Vitaliy Stopchans'kyy <stopcha at gmail.com>
#

[Andrulko]

> I think we'll have to give up on WindowsXP support after a while, so when we stop supporting WindowsXP this won't be an issue.

That's great to know :) The switch to UTF-8 would be the simplest solution. Probably, only a portable version could be provided to XP users

We generate pages on https://winmerge.org/translations/?lang=en using translator information in comments like:

Oh, I see. The last translator username is included in the PO header but it may not be very good for crowdsourcing. There is a "Top Members" report in place that allows collecting contributors list, you can see how many words were translated as well: https://support.crowdin.com/project-reports/?q=Reports#top-members

Of course, there is an API available for the same purpose: https://developer.crowdin.com/api/v2/#operation/api.projects.reports.post

A spreadsheet with contributors can be extracted, then you can publish results on the page. You can also display the number of translated words, it will be some kind of competition 🙂

[JadranR]

Hi, It's high time some people forget about outdated operating systems and support for them. We follow the times and advanced solutions.

With respect,

Jadran Rudec

V V sre., 9. nov. 2022 ob 15:07 je oseba Andriy Poznakhovskyy < @.***> napisala:

> I think we'll have to give up on WindowsXP support after a while, so when we stop supporting WindowsXP this won't be an issue.

That's great to know :) The switch to UTF-8 would be the simplest solution. Probably, only a portable version could be provided to XP users

We generate pages on https://winmerge.org/translations/?lang=en using translator information in comments like:

Oh, I see. The last translator username is included in the PO header but it may not be very good for crowdsourcing. There is a "Top Members" report in place that allows collecting contributors list, you can see how many words were translated as well: https://support.crowdin.com/project-reports/?q=Reports#top-members

Of course, there is an API available for the same purpose: https://developer.crowdin.com/api/v2/#operation/api.projects.reports.post

A spreadsheet with contributors can be extracted, then you can publish results on the page. You can also display the number of translated words, it will be some kind of competition 🙂

— Reply to this email directly, view it on GitHub https://github.com/WinMerge/winmerge/issues/1532#issuecomment-1308818282, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMUTEQVTSAHTGUCGQVXBT2LWHOVZVANCNFSM6AAAAAARJGSAY4 . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[Gitoffthelawn]

Hi, It's high time some people forget about outdated operating systems and support for them. We follow the times and advanced solutions. With respect, Jadran Rudec

With respect, that's not respectful or realistic.

There are currently trillions of computing devices in use on planet Earth, plus billions in use in our planet's atmosphere, plus millions in use outside of our planet's atmosphere.

Many of these devices cannot, or will not, ever be updated.

Deciding whether or not to support specific operating systems is a complex decision and is not simply made by a random person on the internet unilaterally declaring "it's high time some people forget about outdated operating systems".

[JadranR]

Hi,

I am grateful for any opinion aimed at the progress and development of WinMerge. That is why I am surprised by those who expect WinMerge to perform procedures that are basically not supported by outdated operating systems. It doesn't matter how many computers there are on planet earth with outdated operating systems. More important is how many WinMerge users are using an OS older than Windows 7. Do a user survey and find out the real situation. Otherwise, WinMerge will fail due to inability to maintain and debug. Competition in this area will weed out free apps that don't make a profit.

Best Regards, Jadran

[Gitoffthelawn]

...JadranR...

What Jardran wrote in their second comment obviously contradicts what they wrote in their first comment, and of course misrepresents what I wrote and what others wrote. Oh well, that's how it goes. Let's move on and discuss the topic on hand of potentially adding Crowdin TMS support.