Localization of the notes

[astrojuanlu]

As of Sphinx 1.1, there is the posibility to generate translatable strings (.pot files) from a Sphinx project, like it is done in the Open Data Handbook:

https://github.com/okfn/opendatahandbook#building-the-english-source-for-translation-i18n

which can be later translated, for example, through a crowdsourcing service like Transifex:

https://www.transifex.com/projects/p/opendatahandbook/

(also self-hosted if needed, otherwise the platform is free for open source projects)

Would it be possible to have this for the notes? For the case of Spanish language, we started our own translation:

https://github.com/Pybonacci/scipy-lecture-notes-ES

but it's very difficult to catch up with the changes upstream, and everything gets messed up. I would like to know if there would be the possibility, otherwise translating is probably not worth the effort.

[esc]

I am certainly open to supporting this. Having said that, I think It will be quite a lot of effort, however, to set everything up and I am weary of being able to accomplish this on thinly spread spare time -- of everyone involved -- alone. If on the other hand, we were to raise funds to support, say one or two man months of full-time work, I think this could be achieved. This would also mean, we could overhaul the existing chapters, bring them into shape, use consistent conventions throughout the chapters etc.. Also, maybe even make the necessary alterations so that we can go from sphinx/rst to IPython notebook (I have a half-baked script which goes quite far already). There are many potential ways to fund such a project, e.g. crowd-funding, university involvement, commercial sponsorship, or even offering the material as part of a massive open on-line course in multiple languages.

Also, you are not the only one wishing for some translations, I think I saw some translations to Japanese somewhere too, so it may be worth prepping the code-base to be translatable.

Anyway, these are just thoughts, but the general message from my side is, I would be open to it. :)

[astrojuanlu]

Also, maybe even make the necessary alterations so that we can go from sphinx/rst to IPython notebook

You mean having a way to also deliver the lectures in IPython notebook format or working on that format directly instead of Sphinx? (I guess the former, otherwise we'd lose the advantage of Sphinx-i18n)

Anyway, I will patiently wait for that funding to arrive :)

[esc]

Yeah, keep the sphinx, have it localized and generate IPython notebooks based on that.

Regarding the funding, I am tied up until the end of the year with various projects, so the earliest start date for me to even start looking for funds would be January. I'll make a note in my diary. ;) Of course, If anyone else would like to champion this project, please feel free!

[uchida]

I am working on Japanese translation with gettext transifex. Its outcome is the following:

• http://turbare.net/transl/scipy-lecture-notes/ (hosting on my website)
• https://www.transifex.com/projects/p/scipy-lecture-notes-2013-1/ (using free plan for opensource projects on transifex)

The scipy-lecture-notes project is possible to translate with gettext and transifex. Transifex has a translation memory (TM) system to provide automatic suggestions based on similarities between source strings. It helps catching up with the changes upstream.

One annoying thing is that version control especially file composition changes (this is a problem due to transifex system). Transifex suports "release", but its Web API is poor in the present time. To evade this, I made a project corresponding to release version such as scipy-lecture-note-2013-1 for release 2013.1 tag. To catch up upstream change, use shared TM across projects.

Consult transifex support page about TM for more information.

I will give a concrete explanation to translation process with gettext and transifex later.

[GaelVaroquaux]

A few points:

• Funding: that funding is not there. Someone has to find it.
• Amount of effort: writing an maintaining a translation is work, a lot of work. Some technology may help, but I wonder how automatic maintaining the translation will be, and how much it will require work from the side of the core version. Quality translation being very much a manual work, it begs the question: should a translation be maintained by forking the main project at regular intervals, rather than as a continuous process?

And finally, I'd like to stress that from my point of view, having translations is great, but it should not come as a drag to the English version. Right now, there are plenty of things that we would like to change in the lecture notes and we cannot find time to do them.

[astrojuanlu]

And finally, I'd like to stress that from my point of view, having translations is great, but it should not come as a drag to the English version. Right now, there are plenty of things that we would like to change in the lecture notes and we cannot find time to do them.

I haven't tried Transifex in any real project yet (@uchida can tell better) but in my mind, for the people right now working the situation would be exactly the same:

• they do their changes in the repo,
• strings get updated on Transifex, (maybe this can be done with a git or web hook)
• then some other people take care of translating them.

http://support.transifex.com/customer/portal/articles/996211-pushing-new-translations

Of course translating is a lot of work, but if we can have some help from upstream just introducing what seems to me little changes we would be able to do it better. I don't even want the scipy-lectures team to support or make official the translations or something like that, because they might be incomplete: that's the responsibility of the corresponding translation team.

[uchida]

On the side of the core version, I think it's not so hard to keep lecture notes easy to translate, consult development documentation of sphinx http://sphinx-doc.org/latest/intl.html#using-transifex-service-for-team-translation. Of course I think it's hard to maintain translation and manage translation team.

[esc]

Sorry, if I did not make that clear. @GaelVaroquaux is of course right, the funding isn't available but needs someone to find it. My note about January was primarily intended to perhaps begin looking for some funding. Sorry, if I wasn't clear on that.

[GaelVaroquaux]

Hi,

On the side of the core version, I think it's not so hard to keep lecture notes easy to translate, consult development documentation of sphinx http://sphinx-doc.org/latest/intl.html#using-transifex-service-for-team-translation.

I had a look at the page, and it seemed interesting, althought they are many things that I don't understand (for instance, I don't know what .pot and .po files are), so I couldn't really get a full grasp of the workflow. I am sorry, I am really bandwidth limited, and I don't have the time to learn this, although it does seem very useful.

From a pragmatic perspective, the only thing that worries me is: is it going to make building the project more difficult, and if so how much.

If you want to give it a try in a fork with a minimalist setup and send us a pull request, it would be easier for us to judge based on some code and if it looks simple enough, we could merge it.

[astrojuanlu]

Oh well, I took it as granted that you knew all the fuss about .pot, .po and whatnot, sorry. All right, then either @uchida or me can give it a try. Probably it will add a one-liner to the build process but let's check that for real.

[GaelVaroquaux]

I took it as granted that you knew all the fuss about .pot, .po and whatnot.

Actually, I don't :(. I heard about them, but never had time to investigate. It's great to have you on board to bring in new knowledge.

[astrojuanlu]

You never wondered how to translate all this stuff to French? :)

Anyway, I hope we reach to the conclusion that the build process does not get too complicated in the end. Thanks for the hard work!

[GaelVaroquaux]

You never wondered how to translate all this stuff to French? :)

I tell my students to learn English.

[uchida]

To build (es and ja) in shell (for example html):

$ tx pull -a # to get translation files on transifex
$ for lang in "es ja"; do sphinx-build -b html -d build/doctree -D language="$lang" . build/html/$lang; done

the problem is tx pull is slow (because of checking updates of files), for my environment it takes several minutes.

My plan of workflow is following:

• some settings on conf.py and Makefile
• upstream release: $ git tag <release>
• create a projects corresponding to the release version
  • create transifex project on Web UI
  • register translation files $ make gettext && sphinx-intl update-txconfig-resources -p build/locale --transifex-project-name scipy-lecture-notes-<release>
• Import older translation on to new release version can be done via shared Translation Memory (TM)
• translate on transifex Web UI or download and edit po files and upload them.
• build them

Creating a project corresponding to the release version is tweak to evade transifex's poor file management, there may be a better way.

[GaelVaroquaux]

I am closing this issue, as there has not been activity in 3 years, and I am not sure that we have the bandwidth to undertake such an endeavor.