Commit access for TranslateWiki in LibreMesh repos

[patogit]

(Continuing the conversation from PR #254)

@nicopace and @Nikerabbit (and anyone else who's interested), let's see if we can agree on how to arrange commit access for translation files from TranslateWiki. I imagine that whatever we work out here can apply to the LiMeApp too ( @gmarcos87 ), and maybe other repos if they're ready to translate before January.

How about if commits go to a protected branch for the first two months, and then we revisit the idea of TranslateWiki having direct push commit access?

In other repos, @TranslateWiki makes about one commit per month (usually on a Monday or Thursday). So after two commits maybe there will be more confidence to allow direct push... or maybe the LibreMesh development flow really depends on translations going through protected branches just like all the other code.

@Nikerabbit, would that work for you? Do you have other ideas?

[Nikerabbit]

Chef looks okay now, I can enable it whenever you want.

For the other two, I still see a bad qqq.json and a missing pot file. As far as I know you have fixed these, is that right? So I wonder why I cannot see those changes. Can you check whether the following configuration is correct (repo and branch):

libremesh:
  group: libremesh-*
  repos:
    libremesh/lime-packages:
      type: github
      branch: feature/tw-translations
      url: https://github.com/translation-bridge/lime-packages.git
      url|export: git@github.com:translation-bridge/lime-packages.git
    libremesh/lime-app:
      type: github
      branch: issue-119
      url: https://github.com/translation-bridge/lime-app.git
      url|export: git@github.com:translation-bridge/lime-app.git
    libremesh/chef:
      type: github
      url: https://github.com/translation-bridge/chef.git
      url|export: git@github.com:translation-bridge/chef.git

[patogit]

figuring this out now. I think I overlooked the coordination between branches. I'll update branches.

[patogit]

@Nikerabbit, I think I've fixed lime-app (thanks @gmarcos87 for the git help!!): https://github.com/translation-bridge/lime-app/tree/issue-119/i18n Does generic.json fulfill the role of a .pot file in this case?

[patogit]

This is what I did, guided by @gmarcos87 (noting it here for future reference):

cd GitHub-translation-bridge/lime-app
git fetch upstream
git checkout issue-119
git rebase upstream/develop
git push origin

[patogit]

Now I've done the same with lime-packages: https://github.com/translation-bridge/lime-packages/tree/feature/tw-translations/packages/lime-webui/po

[patogit]

@Nikerabbit, in the config you posted, chef doesn't require specifying the master branch?

[patogit]

We also plan to use the page translation tool to translate how-to guides, software readme files, and website pages. As I understand, we'll have to paste all of that into pages we create on TranslateWiki, and then copy the translations to the appropriate places.

[Nikerabbit]

1) The generic.json is not actually needed here. en.json is sufficient. lime-app looks now okay too.

2) master is default value and can be left out

3) With lime-packages I get the following error: Contents of /resources/projects/libremesh/lime-packages/packages/lime-webui/po/es/lime-webui.po are not valid utf-8

Can you (or I) convert it to utf-8? It's currently in some ISO-8859 encoding.

4) We should discuss separately what's the best way of translating those different types of content.

[patogit]

3. With lime-packages I get the following error: Contents of /resources/projects/libremesh/lime-packages/packages/lime-webui/po/es/lime-webui.po are not valid utf-8

Can you (or I) convert it to utf-8? It's currently in some ISO-8859 encoding.

I'm asking developers to make sure that switching to UTF-8 won't crash the routers.

4. We should discuss separately what's the best way of translating those different types of content.

Ok, continuing the thread from https://github.com/libremesh/lime-packages/issues/257#issuecomment-365522564 . Would you prefer to discuss on GitHub, on TranslateWiki, via email, on Riot/Matrix, or some other channel?

[Nikerabbit]

Email, Telegram, GitHub and IRC are all okay to me, pick your poison :)

[patogit]

opened https://github.com/libremesh/lime-packages/pull/357 to change encoding to UTF-8, since developer responses via text chat were not decisive.

[patogit]

what email address?

[patogit]

I'll send an email to you and the lime-users list, if that's ok with you. Otherwise, we can have the conversation on GitHub.

[Nikerabbit]

I'm niklas.laxstrom at gmail.com

[patogit]

@Nikerabbit I think everything is ready to go. I just realized that since TW will import from the translation-bridge repo, and I've already made the UTF-8 change there and tested it on a router, it's ready. (We don't have to wait for the official repo to pull the change.)

So, please enable all three!

[patogit]

opened #357 to change encoding to UTF-8, since developer responses via text chat were not decisive.

pull request closed, so now everything is in order in the official repo too. Thanks @G10h4ck

[Nikerabbit]

This is now live: https://translatewiki.net/w/i.php?title=Special:Translate&group=libremesh

I've marked it as experimental which in practice is only about letting translators to know that there might be hiccups in integration and until https://translatewiki.net/wiki/Setup_of_a_new_project#Wiki_configuration is completed. Help is welcome there.

[patogit]

Thank you @Nikerabbit and the LibreMesh folks who've helped get this set up!

I have:

1. Added LibreMesh to https://translatewiki.net/wiki/Group_descriptions but it doesn't show up as available to translate into Spanish.
2. Put some starting info on https://translatewiki.net/wiki/Translating:LibreMesh.
3. Created the LibreMesh category, support category, user template. Put the logo on all the pages. Added myself as a translator with the babel box.
4. Sent an invitation to translators to sign up and begin! (lime-users email list, https://lists.libremesh.org/pipermail/lime-users/2018-May/001147.html)
5. Noticed that all strings are translated into Greek and Mandarin Chinese. I don't know how that happened since I don't seen those in any repo, but great!
6. Added AGPLv3 license to chef and submitted a pull request to change lime-app from GPLv3 to AGPLv3 so that all the licenses are the same. Text will have a different license.

Next:

• Figure out text translation. I sent an email to you and the list on May 14, and if you @Nikerabbit reply to the list, your message will go to the list.
• Add more message documentation.
• Anything else you recommend doing?

[Nikerabbit]

1) I've marked the page for translation. You can find it here https://translatewiki.net/w/i.php?title=Special:Translate&group=page-Group+descriptions&action=page&filter=%21translated&language=es 2) Thanks! I made some tweaks 3) Yay! 4) Yay! 5) These are new translations already coming in from our community :) 6) Nice

Next steps for me is to make sure also export back to github repo works, some tweaks to group configuration (remove "experimental", add logo to there too) and continue discussion about remaining translatable content.

I will be attending https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2018 for rest of this week, so please be patient :)

If you have feedback about the translation process in translatewiki.net and setting up a new project for translation there, I would be glad to hear it. I know this has taken a while because I've been busy, but perhaps there are ways simplify the process or add more documentation.

[patogit]

@nicopace our promise to translatewiki.net was to act promptly on pull requests related to translation (https://github.com/libremesh/lime-packages/issues/257#issuecomment-346884784). This has happened for chef and lime-app, but there are two and soon will be three pull requests for lime-packages that need attention. Who can be a reliable person or two people to merge pull requests to lime-packages twice a week? I see that recent merges have been done by @G10h4ck, @aparcar , @nicoechaniz , @p4u , @altergui. https://github.com/libremesh/lime-packages/pull/358 https://github.com/libremesh/lime-packages/pull/359

[nicopace]

I did... But I'm worried about the peer review... How will we ensure that the translations are actuate?

On May 21, 2018 9:49:05 AM CDT, patogit notifications@github.com wrote:

@nicopace our promise to translatewiki.net was to act promptly on pull requests related to translation (https://github.com/libremesh/lime-packages/issues/257#issuecomment-346884784). This has happened for chef and lime-app, but there are two and soon will be three pull requests for lime-packages that need attention. Who can be a reliable person or two people to merge pull requests to lime-packages twice a week? I see that recent merges have been done by @G10h4ck, @aparcar , @nicoechaniz , @p4u , @altergui. https://github.com/libremesh/lime-packages/pull/358 https://github.com/libremesh/lime-packages/pull/359

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/libremesh/lime-packages/issues/257#issuecomment-390676871

-- Enviado desde mi dispositivo Android con K-9 Mail. Por favor, disculpa mi brevedad.

[Nikerabbit]

I can answer to that briefly:

1. We vet (check) translators who sign-up to keep out spammers and vandals
2. We provide review tools.

The best way to support high quality translations is to provide message documentation (which you have), a glossary/terminology and good quality English source strings.

Our large translation community and translation memory helps with terminological consistency between all projects, not just inside your project.

Finally, we encourage that translations are shipped to users in timely manner, so mistakes can be spotted early and fixed very easily and also in a timely mannery.

[nicopace]

Thanks Nick!

Will be merging the changes as they go.

Some other things:

• how are we gonna deal with new languages: indigenous communities have languages that may not have been classified
• can translations be based in spanish or french instead of English sometimes? The second language is not always english
• how are we gonna do with upstream translations (ie openwrt luci)

Regards

On May 22, 2018 5:19:19 AM CDT, "Niklas Laxström" notifications@github.com wrote:

I can answer to that briefly:

We vet (check) translators who sign-up to keep out spammers and

vandals

We provide [review

tools](https://www.mediawiki.org/wiki/Help:Extension:Translate/Quality_assurance).

The best way to support high quality translations is to provide message documentation (which you have), a glossary/terminology and good quality English source strings.

Our large translation community and translation memory helps with terminological consistency between all projects, not just inside your project.

Finally, we encourage that translations are shipped to users in timely manner, so mistakes can be spotted early and fixed very easily and also in a timely mannery.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/libremesh/lime-packages/issues/257#issuecomment-390939830

-- Enviado desde mi dispositivo Android con K-9 Mail. Por favor, disculpa mi brevedad.

[Nikerabbit]

We have a pretty good coverage of languages already (500+) and can add more if the language has a language code (many thousands already do). How likely is that you will have translators for a language that does not have a standard code yet?

We have setting in user preferences that can be used to display translations from language(s) other than English in the translation interface. We are exploring the options of making it easier to use by replacing the English text with the translated text when available.

I don't know about the third question.

[nicopace]

On 2018-05-22 09:39 AM, Niklas Laxström wrote:

We have a pretty good coverage of languages already (500+) and can add more if the language has a language code (many thousands already do).

That is great!

How likely is that you will have translators for a language that does not have a standard code yet?

Don't know... just asking. Also, we work in First nation territories, so it may happen... that is why I ask :) Thanks for the fast response btw!

We have setting in user preferences that can be used to display translations from language(s) other than English in the translation interface. We are exploring the options of making it easier to use by replacing the English text with the translated text when available.

Excellent, thanks!!!

I don't know about the third question.

this one was more general, and to Pato to see if he tought about it

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/libremesh/lime-packages/issues/257#issuecomment-391015524, or mute the thread https://github.com/notifications/unsubscribe-auth/ABFX1YIOh10OYzcPSDfibDGKIFMbOae8ks5t1CM6gaJpZM4QfB39.

[patogit]

Hi @Nikerabbit,

I hope the Wikimedia Hackathon went well.

Seems like the import/export process of translation files between translatewiki.net and GitHub is working in both directions.

Any progress thinking about the translation of our other content that isn't in translation files?

If you have feedback about the translation process in translatewiki.net and setting up a new project for translation there, I would be glad to hear it. I know this has taken a while because I've been busy, but perhaps there are ways simplify the process or add more documentation.

I'll gather my thoughts about the process into clear feedback sometime on June-ish.

[patogit]

Continuing the conversation from this email regarding translation of other files: Most of the files are in plain text at the moment, so I'm considering two options:

1. Page translation, as explained here.
2. use txt2po, put the po files in a GitHub repo connected to translatewiki.net, and use po2txt to generate final files.

Both options require manual work. I might try both to see what seems easier.

[patogit]

Something to understand about our current workflow: The instructional booklets (and some other documentation) are created in Spanish, not English. So either:

a) we need a way to mark Spanish as the source language (and maybe when English translation is complete, switch the source language to English in order to facilitate translation into other languages; or b) we need to translate to English before using the translatewiki.net platform, and then paste the Spanish original in (only if we use Page Translation, since PO file translation we can place the Spanish PO file in the GitHub repo); or c) something else.

[patogit]

@Nikerabbit you once mentioned that we could upload screenshots for message documentation to translatewiki.net, and this would enable directly showing the images in the translator interface, instead of showing a link to the image hosted on GtiHub as we do right now. To do this, I just upload files using the https://translatewiki.net/wiki/Special:Upload page? Is there any way to upload files in bulk, in other words upload 25 files at once?

[Nikerabbit]

About mass upload: I would need to check about installing one of the bulk upload extensions:

• https://www.mediawiki.org/wiki/Extension:SimpleBatchUpload
• https://www.mediawiki.org/wiki/Extension:UploadWizard

There is also a command line script I can use for one time bulk uploads.

[patogit]

@Nikerabbit are the pages of translatewiki.net indexed be search engines? I imagine they are not. I just want to confirm, because it doesn't make sense for users to end up at translatewiki.net when they search for LibreMesh documentation -- we want them to go to the LibreMesh website.

[patogit]

I added translate tags to our project page, but it won't let me enable translations.

For ease of dealing with the files instead of wiki pages, I'm favoring the idea of creating a GitHub repo for our documents and connecting it to translatewiki.net. We can use txt2po, and structure the repo like this:

/
/Instructional-book-1
/Instructional-book-1/en.po
/Instructional-book-1/es.po
/Essay-1
/Essay-1/en.po
/Essay-1/es.po

Or does the repo need a different structure?

[Nikerabbit]

We don't translate project pages, at least not currently.

The repo structure looks okay, but I'd like to have look what do the po(t) files look like to make sure the they are appropriate for the translatewiki.net.

[G10h4ck]

I think it is better to have one repository for each document

[patogit]

@G10h4ck , for each repo, @Nikerabbit has to add a few lines of config and test the connection, it's much easier if we do all the documents in one repo. This way we can add documents whenever we want, and they automatically get picked up by translatewiki.net

[patogit]

The repo structure looks okay, but I'd like to have look what do the po(t) files look like to make sure the they are appropriate for the translatewiki.net.

@Nikerabbit , here's the beginning of a file run through txt2po. I'm just pasting it here because this is easier for me. The actual file I know needs to be in UTF-8. How does this look?

#. extracted from PLANIFICACION.txt
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2018-06-11 07:20-0500\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"X-Generator: Translate Toolkit 2.3.0\n"

#: PLANIFICACION.txt:3
msgid ""
"1)\n"
"PLANIFICACION DE UNA RED LIBRE Y COMUNITARIA"
msgstr ""

#: PLANIFICACION.txt:7
msgid ""
"2a)\n"
"Para iniciar una red comunitaria\n"
"lo primordial es la comunidad!"
msgstr ""

#: PLANIFICACION.txt:10
msgid ""
"2b)\n"
"Como punto de partida es necesario conformar un grupo de personas "
"interesadas en trabajar por la autonomía de sus comunicaciones, que sienta "
"la necesidad de emprender este proyecto tan tecnológico pero, sobre todo, "
"tan social."
msgstr ""

#: PLANIFICACION.txt:18
msgid ""
"3)\n"
"Planificación\n"
"Analizar nuestra comunidad\n"
"Organizar la acción\n"
"Pensar el financiamiento\n"
"Ordenar la administración\n"
"Planificar el despliegue"
msgstr ""

#: PLANIFICACION.txt:21
msgid ""
"4a)\n"
"Analizar nuestra comunidad"
msgstr ""

#: PLANIFICACION.txt:24
msgid ""
"4b)\n"
"Los análisis socioculturales interrelacionan los factores del contexto "
"social que, junto con una comprensión de las necesidades, las normas y los "
"valores que definen a nuestra comunidad, nos permiten una formulación "
"precisa de soluciones."
msgstr ""

#: PLANIFICACION.txt:29
msgid ""
"4c)\n"
"El análisis de nuestra realidad como comunidad nos ayudará a tomar "
"decisiones colectivas acertadas y a generar una identidad que nos represente."
"\n"
"Las decisiones sobre las características de nuestra red libre y comunitaria "
"se tomarán en base al tipo de relaciones que se dan en nuestra comunidad.\n"
"Por eso es tan importante hacer este análisis para prever las limitaciones y "
"necesidades que pueden surgir en cada proyecto de red respecto de las "
"personas que la usarán y la utilidad que le darán."
msgstr ""

[patogit]

@Nikerabbit how does this look? I hope we can move forward with this phase of document translation next week.

[Nikerabbit]

I suppose that's okay length-wise. Did you decide how to translate them to English and how much of text there is going to be overall?

[patogit]

I can translate them to English before uploading, that seems the easiest.

how much of text there is going to be overall?

Multiple booklets (maybe 6 booklets with about 1700 words each, though I'm not certain), and some web pages. If it works well, we could use it for more texts in the future.

[patogit]

@nicopace or @nicoechaniz can you give me whatever powers I need to administer the repo at https://github.com/libremesh/lime-docs ? I figure that's a logical place for the documents.

[nicopace]

@patogit done

[patogit]

@Nikerabbit can you check the structure of this repo and tell us if it looks feasible? https://github.com/patogit/lime-docs . Only one of the directories has a POT file, https://github.com/patogit/lime-docs/tree/master-new/Booklet-02-Planning , since I haven't translated the other to English yet.

[patogit]

Thinking about the process of translatewiki.net creating new translation files, and the structure of the lime-packages repo which uses PO files, I wonder if translatewiki.net would function better with a structure like this:

/Booklet-01/booklet-01.pot
/Booklet-01/es/booklet-01.po

What do you think @Nikerabbit ?

[Nikerabbit]

The directory structure doesn't really matter for us as long as it is consistent for each translation.

[patogit]

@Nikerabbit , this is ready for translatewiki.net now https://github.com/translation-bridge/lime-docs . Note that I added .pdf.txt to .gitignore because I'm using that extension to store the URL of the PDF file for each language, so the .pdf.txt files do not need translating.

[Nikerabbit]

I am traveling this week. Feel free to ping me again if I haven't replied by early October.

[patogit]

@Nikerabbit I hope your travels went well. How can we move forward with this?

[patogit]

@Nikerabbit ping pong....

[patogit]

following the format from this comment https://github.com/libremesh/lime-packages/issues/257#issuecomment-388729798

libremesh/lime-docs:
      type: github
      url: https://github.com/translation-bridge/lime-docs.git
      url|export: git@github.com:translation-bridge/lime-docs.git

Any chance you can configure the repo and activate this space in translatewiki.net this week?

[patogit]

Update on this thread: The booklet documentation format changed (from PO to JSON), but the old format got activated on translatewiki.net. So we're working on converting the translations already done into the new format. Part of the process I've completed by making a macro in jEdit (https://github.com/libremesh/lime-docs/issues/29).

Now we want to figure out how to make these converted JSON files and the automated SLA-to-JSON-to-SLA pipeline that @nicopace created compatible with each other.