Update to cover all Wiktionary transliterations?

[twardoch]

How feasible would it be to update this to cover all Wiktionary transliterations, and possibly expose additional functionality?

1. Update to cover all transliteration modules: https://en.wiktionary.org/wiki/Category:Transliteration_modules

2. Change the API so we can specify the input ISO 639 language code and/or ISO 15924 script code, possibly only the ISO script code without specifying the language.

3. Change the API so we can optionally specify the output language and/or script code, so we can transliterate to other scripts/languages, not just Latin, using modules like https://en.wiktionary.org/wiki/Module:Newa-Deva-translit

4. It would be best that language and script lookups would be done using native Lua modules (see dependencies below)

5. Provide instructions or some script that automatically updates the Lua components from Wiktionary (I don’t know how this is done).

Possible dependencies

Integrate languages:

• https://en.wiktionary.org/wiki/Module:languages
• https://en.wiktionary.org/wiki/Module:languages/alldata
• https://en.wiktionary.org/wiki/Module:languages/byTranslitModule
• https://en.wiktionary.org/wiki/Module:wikimedia_languages
• https://en.wiktionary.org/wiki/Module:wikimedia_languages/data
• https://en.wiktionary.org/wiki/Module:languages/JSON
• https://en.wiktionary.org/wiki/Module:language-like
• https://en.wiktionary.org/wiki/Module:etymology_languages

Integrate families:

• https://en.wiktionary.org/wiki/Module:families
• https://en.wiktionary.org/wiki/Module:families/data

Integrate scripts:

• https://en.wiktionary.org/wiki/Module:scripts
• https://en.wiktionary.org/wiki/Module:scripts/data
• https://en.wiktionary.org/wiki/Module:scripts/by_name
• https://en.wiktionary.org/wiki/Module:scripts/code_to_canonical_name
• https://en.wiktionary.org/wiki/Module:scripts/print
• https://en.wiktionary.org/wiki/Module:writing_systems

Integrate translit-redirect/data

• https://en.wiktionary.org/wiki/Module:translit-redirect
• https://en.wiktionary.org/wiki/Module:translit-redirect/data

Dependencies:

• https://en.wiktionary.org/wiki/Module:JSON
• https://en.wiktionary.org/wiki/Module:debug
• https://en.wiktionary.org/wiki/Module:table

[twardoch]

Ps. Of course stub/replacement modules can be provided, as it’s already done.

[twardoch]

Ps. @tatuylonen has https://github.com/tatuylonen/wiktextract/ and https://github.com/tatuylonen/wikitextprocessor/ — I’ve opened an issue https://github.com/tatuylonen/wiktextract/issues/68 where I inquired about updating the Wiktionary transliteration modules.

I see that tatuylonen’s modules also use some "safe" workarounds to run Lua code. Perhaps your Wiktra could somehow integrate with wikitextprocessor/wiktextract so that the same techniques are used?

[twardoch]

I see that it’s possible to do https://en.wiktionary.org/w/index.php?title=Module:scripts&action=raw

[twardoch]

And And https://en.wiktionary.org/w/api.php?action=query&list=categorymembers&cmtitle=Category%3ATransliteration_modules&cmlimit=500&format=json

[twardoch]

I’ve created a Python tool that downloads Wiktionary modules: https://github.com/twardoch/wiktra-update

[twardoch]

I made this into a pull request https://github.com/kbatsuren/wiktra/pull/4 from my own fork https://github.com/twardoch/wiktra/

[kbatsuren]

Thank you so much for the detailed suggestions. I will have a look at it as soon as I come back from my vacation.

[twardoch]

OK, I think I can close this now :)