Proposal : integrate a translation macro into the core

[sycom]

There had been many discussions about TiddlyWiki internationalization capabilities, here as issues or on the discussions groups. Such a macro should allow both developers to offer translations for their work (plugins, macros, themes...) and users to enable multi-languages wikis.

I made a working demonstrator called i18n macro (source code) that I'm using for my developments (you know, eat your own dog-food) and that could solve a lot of cases listed below. It's far from perfect but can be a good start for the discussion about implementation for the core.

@Jermolene already said (in #2722 discussion) he'd rather have it as real macro than a javascript macro. There are a lot of other challenges to make it useful. I just have a few in my head right now that I put here. Please put yours :

• how tiddlers content will still be searchable if their translations are within $:/some_domain/i18n/lg-LG/some_title tiddlers ?
• How to deal with titles for whole tiddlers translations. It's a major issue when languages are using non-Latin alphabet?
• maybe the macro doesn't have to solve all those, but can be used by plugins that will do more deep mods?

For people who need to down write ideas, I made a draft of reflexion around a, more radical, full i18n system in TiddlyWiki. That's on a CodiMD. Feel free to go there and make your edits: https://demo.codimd.org/ak7D5_2tQRmPt60zvbJkag

References

• issue #786 : Plugins that expose strings should be translatable
• Issue #983 : Theme descriptions in ControlPanel/Appearence should be translatable
• issue #1372 : instructions for plugin language support
• issue #1513 : Language Switcher API
• issue #2597 : changing tiddlers in the story when changing the language
• issue 12 in GSD5 : Language support
• discussions around the i18n macro
  • with a potential Ukrainian user who have needs around titles, timelines that imply also templates modifications
  • with some users about the best way to do plugins translations.
• discussions around i18n concept and other attempts
  • about a potential solution that have emerged around 5.1.12 version

[sycom]

I worked on a first draft of this macro. It is now a real TW (no-js) macro as prefered by @Jermolene . I think it is ready for a PR, but I it's a good idea to ask for comments on the GGroup before. Its first purpose is plugins translations and other system / shadow tiddlers. It may be used for other translations but there are issues (search capabilities, title translations...)

https://sycom.frama.io/TiddlyWiki-Plugins/dev/i18n.html

You may also visit the collection wiki to see it in action and explore limitations.

[sycom]

Thanks to @pmario and @AnthonyMuscio (see google groups thread) I was able to refactor it a bit. It's more readable and gained about 400 octets. More gains are possible removing some metadata (and maybe whites paces when switching from review to "production")

https://sycom.frama.io/TiddlyWiki-Plugins/dev/i18n.html

Maybe some would discuss parameters order, or other details so I still wait a bit before merge request.

[pmario]

It's easier to read now. So I can dig deeper and try to understand it.

[sycom]

@pmario : I dropped a behaviour section in the i18n Macro documentation tiddler. Might be useful.

[linonetwo]

As I'm propagandizing TiddlyWiki in China, I think the i18n framework is becoming an urgent need, it will help more users get to use TW instead of other commercial note apps.

[Jermolene]

Hi @linonetwo

As I'm propagandizing TiddlyWiki in China, I think the i18n framework is becoming an urgent need, it will help more users get to use TW instead of other commercial note apps.

Absolutely! Are there other areas that need improvement? Could I invite you to make a ticket "Helping TiddlyWiki Adoption in China" where you can list the concerns and we can make plans.

[joshuafontany]

Localization is also on my Bulletin List of items to keep track of. Coming back around to TW work here & I remember this being a really good start on Localization concerns. The RPG Guild-Wiki App thing I am building needs to support multi-lingual audiences. :)

[linonetwo]

Could I invite you to make a ticket "Helping TiddlyWiki Adoption in China" where you can list the concerns and we can make plans.

I will create an issue with a checklist.

We have a QQ group and a Zhihu article that are full of feedback, and I'm already solving some of the needs I see with plugins, I think there are not many changes needed in the core, thanks to the powerful plugin mechanism!

But still, there are things like i18n that are better to be provided by the core. CPL was another thing I thought we need the core to provide, but we managed to make it work now.

[AnthonyMuscio]

As a mono-language user, perhaps a little french, I would love to know the code patterns to write non javascript macros and plugins that use english language tiddlers such that my international friends in the community who are bilingual could spend a few minutes translating and adding the new language tiddlers and send them to me for inclusion, without needing for anyone to touch the main code.

[linonetwo]

@AnthonyMuscio This is call i18n framework, that restrain the pattern to write l10n data.

My i18n framework in TidGi is i18next , it uses l18n data like https://github.com/tiddly-gittly/TidGi-Desktop/blob/master/localization/locales/en/translation.json

It is pretty standard, and thus can benefit from automatic translation from https://marketplace.visualstudio.com/items?itemName=Lokalise.i18n-ally :

By hitting this button, the text will auto translate by calling bing API or so, then fill in the JSON.

And any function call matching the t('xxx') pattern will have a preview

I think by choosing the i18n's syntax and data structure, we can benefit from it too.

[AnthonyMuscio]

Sounds good @linonetwo but I write tiddlers and macros so I will need more simplified instructions (as will others) or tools, to simply know how to take any "english text", captions, messages etc... I would include in my macros and plugins and ensure they use the appropriate reference or name so other language speakers can just add to it, or use automation like you demonstrate.