Multilingual support with Crowdin

[InessaPawson]

For the up-to-date information on this issue refer to the following:  https://github.com/Shekharrajak/scipy-sphinx-theme-v2/issues

Testing with the current NumPy doc: https://github.com/numpy/numpy/pull/14648

The latest Sphinx theme: Demo link

[Shekharrajak]

Refer: https://github.com/numpy/numpy.org/pull/61, https://github.com/numpy/numpy.org/pull/62

[rgommers]

Just did some testing, this looks nice! I think for testing we can do Mandarin (@zjpoh offered to help) or Hindi if you want to do that @Shekharrajak. I think Dutch was good to try the interface, but we don't really need/want a Dutch translation - Dutch people are perfectly happy with English usually.

[Shekharrajak]

Yeah! I will try to do that.

[rgommers]

Related gh-329 for Korean translation.

[rgommers]

I think we're good to go on this, the content is stable and everything seems to work. I'll re-enable the language selector as well.

@zjpoh would you still be interested to work on the Chinese translation?

I've added a wiki page at https://github.com/numpy/numpy/wiki/Translations-of-the-NumPy-website on how to get started.

[rgommers]

Re-enabling the dropdown is a matter of undoing the small patch in gh-171

[rubentl134]

I would like to participate on the Spanish translation. Is there any way I can start? I feel kind of lost here. Sorry

[rgommers]

That's great, thanks @rubentl134! Spanish is already enabled on Crowdin, so to get started see the instructions at https://github.com/numpy/numpy/wiki/Translations-of-the-NumPy-website. Basically, sign up for Crowdin via the link there, select Spanish, and start:)

If you have any issues, please let us know.

[comradekingu]

Can you use Weblate instead, so I can contribute without getting spied on, leak info, and also contribute to their closed translation memory? You need a license to be able to use the translations btw. :)

[rgommers]

Can you use Weblate instead, so I can contribute without getting spied on, leak info, and also contribute to their closed translation memory?

We have spent significant time setting up Crowdin, after good experiences Spyder had with it. And the Crowdin team has been quite helpful, so I see no reason to switch.

You need a license to be able to use the translations btw. :)

I don't know what you mean by that.

[rgommers]

@comradekingu we noticed you're going around and posting the same request on many projects in this scientific Python ecosystem:

https://github.com/sympy/sympy.github.com/issues/130#issuecomment-683452822 https://github.com/jupyterlab/jupyterlab/issues/8829#issuecomment-683452632 https://github.com/carpentries/glosario/issues/59

Furthermore, you're one of the top contributors to Weblate, and clearly only interested in promoting your own tool with no intention to actually contribute to NumPy or any of the other projects linked above. That's just rude behaviour, please stop doing that. If you want to promote Weblate, please do so on your own project/website/blog/channels.

[comradekingu]

Weblate isn't my project any more so than my contributions to every other platform dictates. Either I have experience and know what I am talking about, or I am still damned if I don't? Why wouldn't I contribute to NumPy if I have contributed to so many other projects? Most projects are happy to move away from Crowdin, and I pretty much get around to warn every libre software project that joins Crowdin. It isn't a lot of project nowadays, but that is all in the doing of the reputation Crowdin deserves.

It so happens if you want to avoid a project where people working for the project actually promote it without being honest about who they are, "Crowdin" is something you would want to stay away from. I can document this claim if you want. If they can do it, someone like me can set the record straight.

I didn't make up my other claims either, and you can read either https://github.com/jitsi/jitsi-meet/issues/5056#issuecomment-612239691 or find the same text in the respective files on Crowdin.

Why wouldn't I advise against that? I have no say in the matter, and you do with the information I provide as you wish. If you want a rundown of what alternatives are libre software, made in Python and check other possible boxes, I'll happily do so. More than picking any platform does anything, I discourage use of Crowdin.

Full disclosure I have gotten a Pootle T-shirt and one from Weblate, if you work the numbers that averages out to not the best way to put clothes on my back. Heck, I've only ever donated money to said projects, I am at a loss…

Hopefully sharing my intention to do better by avoiding Crowdin, Spyder only has coverage in a handful of langauges after 9 months on said platform. Beside that being a low bar to clear anyway, I can call upon people to help out. That is the sort of rudeness I get up to I suppose. The audacity!

There is no license file in the numpy.org repo. A lot of jurisdictions have laws protecting against using commissioned work without compensation, so you will want to state a license clearly. https://github.com/numpy/numpy does it right. Yes, I also have an interest in libre software projects avoiding licensing issues. What a guy!?

[rgommers]

Please stop posting here. You are being rude. At this point I have zero interest in interacting with you more.

[giacomodeodato]

I would like to contribute, are you planning to translate in Italian as well?

[rgommers]

Hi @giacomodeodato, thanks for asking. We aimed to start with the languages for which we expect there's most demand (Chinese, Spanish, Portuguese, Japanese, Korean), and expand based on that experience. I'm not sure, but I would personally expect that most Italians who use NumPy would be okay using English, so it'd be lower prio. It's always hard to say no to an offer of help, but would you mind if we didn't do Italian at least till we get a better sense of how maintainable adding more languages is?

[giacomodeodato]

Of course, I totally understand

[rgommers]

@goanpeca suggested that we should organize a sprint to push translations forward. That does seem like a good idea. The rest of this year is already very full, but maybe we can pick a date in January - I'll put it on the community meeting calendar to discuss.

[MahatKC]

I'd be happy to help with Portuguese translation.

[rgommers]

Thanks @MahatKC! The Portuguese translation is actually pretty far along, thanks to @melissawm. With another push we should be able to get it complete and published relatively soon. Please jump in! Instructions are at https://github.com/numpy/numpy/wiki/Translations-of-the-NumPy-website, let us know if you need any help.

[melissawm]

@MahatKC let me know if you need any help - I'm in the process of populating the glossary so we can get consistent translations. Thanks!

[melissawm]

@MahatKC are you available to review the Portuguese translation?

[vinisalazar]

Hi, I'd also like to help with the review. I considered starting by reviewing the case-studies directory. Would that work @melissawm ?

[melissawm]

Sure, pick whatever document you prefer!

[vinisalazar]

I've reviewed and made suggestions for all four documents in the case-studies directory. I hope it helps! 😊

[rgommers]

Thanks @vinisalazar!

[rgommers]

This is actually a little confusing:

Two alternative translations, and my comment on it doesn't show up in the interface (still new to Crowdin I guess). So commenting here - @AtsushiSakai do you have the approval button as well? It doesn't make sense for me to approve, since I cannot judge which translation should be preferred (or if a hybrid would be best).

[AtsushiSakai]

@rgommers I cannot see "approve" button. I can see "upvote" and "downvote" button. (Maybe approve is only for manager?)

I upvoted some translations if I think his translation is better than mine.

[kngwyu]

Hi, thank you for catching my Japanese translations. I guess approving needs some authority as @AtsushiSakai says. Note that my drafts are still work-in-progress. I'll re-check my translations this weekend, but one or two more Japanese speakers to check the translation would be helpful.

[rgommers]

@rgommers I cannot see "approve" button. I can see "upvote" and "downvote" button. (Maybe approve is only for manager?)

I guess so - so I made you a manager @AtsushiSakai :)

Note that my drafts are still work-in-progress. I'll re-check my translations this weekend, but one or two more Japanese speakers to check the translation would be helpful.

Thanks @kngwyu, good to know. I'm working on a PR today to integrate everything into the site and enable the language dropdown, but I'll wait for the green light from one of you before deploying.

[rgommers]

I integrated the Japanese and Brazilian Portuguese translations in gh-413. It included quite a few fixes, and turned up some more untranslated content - individual sentences and things like header and footer navigation link names. For current master, here is how to render the translations (not deployed to numpy.org yet because there's more to do on the front page):

1. Ensure you have Hugo Extended installed
2. In config.yaml.in, comment out the disableLanguages line
3. Run python gen_config.py (this updates config.yaml with the language-specific content)
4. Run hugo (builds the site)
5. Run hugo server (serves the site on localhost:1313)

I added content/en/config.yaml to Crowdin, but it doesn't look all that easy to translate there because it contains a mix of translatable and non-translatable content. @AtsushiSakai, @melissawm, @kngwyu could you have a look at that file and give your opinion on it - best to translate within Crowdin, or through a PR?

[kngwyu]

I guess PR is better if only one or two people translate the document. Crowdin could be an easier entry point to fix mistakes for readers, though.

[AtsushiSakai]

Crowdin is best for me too for translation. If I can get notifications correctly, I think it is perfect.

[melissawm]

Any way is ok with me, so I'll go with whatever solution you all find easier.

[rgommers]

Thanks all. That makes sense. Crowdin does give notifications and is easier for small updates. How about you update content/ja|pt/config.yaml via a PR now @AtsushiSakai and @melissawm, and then future updates go via Crowdin?

[AtsushiSakai]

@rgommers OK. I will try it in #415.

[melissawm]

Here you go: #418

[map0logo]

Hi, I would be glad to help with the Spanish translation.

[rgommers]

Hi @map0logo, that's great, thanks! You can add yourself at https://github.com/numpy/numpy/wiki/Translations-of-the-NumPy-website and follow the instructions there to start adding translations on Crowdin.

[Tiffilore]

I can offer translations to German.

[JannickTie]

@Tiffilore I guess it's the same thing as with Italian. @rgommers answered it here

Hi @giacomodeodato, thanks for asking. We aimed to start with the languages for which we expect there's most demand (Chinese, Spanish, Portuguese, Japanese, Korean), and expand based on that experience. I'm not sure, but I would personally expect that most Italians who use NumPy would be okay using English, so it'd be lower prio. It's always hard to say no to an offer of help, but would you mind if we didn't do Italian at least till we get a better sense of how maintainable adding more languages is?

But if the time has come to create a german translation, i would love to help you translating.

[rgommers]

Thanks for the offers of help @Tiffilore and @JannickTie!

Indeed, that's still the answer, please give us a little bit of time. If you look at the Netlify preview on the most recent PRs to this repo, you can see we have Japanese and Portuguese >95% complete - an initial release is not far off.

[melissawm]

I think there is something funny going on with Crowdin. I went to translate the additional text added recently and realized that several of the previously translated strings have suggestions that are "off by one": they are mostly section titles that are at the wrong level. To fix this I am approving my own translations but it would be better to check if this is happening for other languages as well. An example:

2021 NumPy survey has the translation previously associated with NumPy version 1.20.0

[comradekingu]

It tries to make use of already existing strings by matching by %? Looks like it chopped out part of an older string (green), and suggests the stuff in white as "close enough".

Either that or Ralf actually made that translation/mistake?

@AtsushiSakai So you have fuzzy strings in the overview, and the actual string far apart from it. The translation is spaced apart from that still, and the way to catch it is by scanning for errors in the fuzziness, or to notice the light yellow on light gray…?

The whole overview scrolls because there is something shoehorned in above it. It does as poor a job at presenting lineshifts as possible, because it wraps artificially at half the width. If you are using the V2 API, that is now deprecated and insecure. The fails go on and on.

Good thing Ralf didn't upvote his suggestion, because then your keen eye would have been wasted on what is a first-taker voting system.

[rgommers]

I did not make any changes in Crowdin directly. I merged a few PRs. There are two similar-looking news items, an old translated one and a new one from gh-452. Somehow their similarity threw off Crowdin when pulling in changes from master.

[AtsushiSakai]

previously translated strings have suggestions that are "off by one": they are mostly section titles that are at the wrong level.

I realized similar things happened in Japanese translations.

So you have fuzzy strings in the overview, and the actual string far apart from it. The translation is spaced apart from that still, and the way to catch it is by scanning for errors in the fuzziness, or to notice the light yellow on light gray…?

I scanned and fixed these manually.

[rgommers]

It may be fixed if we add the new English strings to the Chinese and Portuguese translations of news.md. That way the mapping from one language to the other is much more consistent. Shall we try that as an experiment, or are manual fixes already underway?

[melissawm]

I planned on doing a thorough review anyway, I'll also try to get a second reviewer so a manual fix seems ok.

[ghost]

Does anyone do Hindi translation too ?

[Bohlintb]

I would like to participate on the Persian(farsi) translation. Is there any way I can start?

[InessaPawson]

@Bohlintb Thank you so much for offering your help! Unfortunately, we don’t have plans for Farsi translations for the nearest future. If anything changes, I’ll reach out.