search

WeblateOrg / weblate

Web based localization tool with tight version control integration.
https://weblate.org/
GNU General Public License v3.0
4.64k stars 1.02k forks source link

Frontend plugin integration scenario #1066

Open rvanlaak opened 8 years ago

rvanlaak commented 8 years ago

The new API could allow a scenario I'd like to propose, which is based on https://www.transifex.com/lp/transifex-live/

Also see what a frontend library would look like: https://github.com/transifex/transifex-live-wordpress

This would be a really big step in continuous translation, from a translator's perspective. Please let me know what you think, maybe the API already has most use cases covered?

nijel commented 8 years ago

The API definitely doesn't have enough yet. For 2.6 only basic translation level operations will be supported, it will be extended for future releases.

nijel commented 8 years ago

I've added checkboxes to your list and updated with current status.

uhlhosting commented 3 years ago

WordPress support is a must!

nijel commented 3 years ago

@uhlhosting It's on our roadmap for next year. If you have any specific requirements on it, please share them here.

CC @ilocit @orangesunny

uhlhosting commented 3 years ago

If any beta out there would like to test. Will keep this thread updated if anything.

lecoqlibre commented 3 years ago

Hi, as I need right now a WordPress integration of Weblate I might code it because I can't find such an existing project... But maybe I'am wrong and some WordPress plugin already exists somewhere?

orangesunny commented 3 years ago

Hey @lecoqlibre, it will be much appreciated if you do it. There is no ongoing project on this at the moment. @nijel will help you with any Weblate questions that will pop up during the development :)

lecoqlibre commented 3 years ago

@orangesunny Thank you for the information. So I must write a WP integration by myself...

I need this integration for the www.diyseeds.org website I'm currently developing (since 3 years). We provide (free) educational video about seed production. I used other WP plugins to translate the website before but with the number of translation and the website growing we can't continue this way anymore. Here Weblate comes (thank to all contributors) and we plan to fully migrate to Weblate for the next fall (October/November 2021) as I'm writing a V2 of the Diyseeds website.

I'm wondering how to integrate WordPress with Weblate to suit to our needs and such a way that it can be useful to other people too. For the moment the scenario is not completely clear. I have in mind that translators only come to our Weblate instance to translate the strings of our website (so translators do not have access to the WP admin area). In WP we would only have source language posts (in English for example) that act like HTML templates containing the source strings. WP would extract those strings and put them into Weblate using the Weblate API. To render translated posts WP would use PO files on the fly (cloned locally). We would have a button to commit and push Weblate modifications directly from WP...

ilocit commented 3 years ago

@lecoqlibre @orangesunny @nijel great that this thread is being picked up. I did some further investigation into this late last year and was even about to start / fund the development of a WP PlugIn myself. But then, as per points above, I noticed that Weblate is missing some aspects (API) to support a close integration.

Furthermore, I have some ideas and input with regards to comment handling and "context for translators" via "images" of the website, similar to the "html2canvas" mentioned above, just more granular and more detailed.

@lecoqlibre ping me if you want to exchange ideas. Maybe I can support you.

lecoqlibre commented 3 years ago

Another solution to render translated posts would be to (re)generate them from Weblate API /api/components/<project>/<component>/translations into WP transients. This would avoid to have a local copy of the VCS repository into WordPress unless this would be a good idea?

I must precise that my first short term goal is to get Weblate translations rendered into WP website with localized URLs. What is a "close integration of Weblate" about? It's like editing translations from within WordPress?

@nijel Do you have particular requirements for an integration of Weblate into a WordPress plugin? Do you have any idea of how you would do it or some recommendations?

@ilocit Thank you, it would be great to exchange idea. Did you already make a model of such a WP plugin?

WordPress transient

nijel commented 3 years ago

I have no knowledge of WordPress, so it's hard to evaluate that side for me. Using transients might be problematic as they can disappear any time and you would end up either with untranslated page or request to Weblate. It's probably better to have translations stored persistently and update them in the background.

Did you look at other localization plugins for WordPress? Looking at existing practice might be a good starting point.

ilocit commented 3 years ago

@lecoqlibre I spend some time last year looking into making WordPress and Weblate work together. The #1 problem is the multilingual support in WP which you will need to really support an efficient localization process. After checking out some of the options, I did decide to try WPML (https://wpml.org/features/). It is the most common multilingual WP PlugIn, I think. So this would take care of all the "translation" related aspects in WordPress. And this can be a lot to consider, given how many different content types exist in WP (page text, blog text, categories, keywords, menus,...). When talking with them (WPML) I signed up as a translation partner and got their API documentation (not publicly available otherwise).

When thinking about implementing a solution with WPML and WordPress, I stopped short after because (a) I did not have the time, and (b) was unsure about the requirements towards the Weblate API (like which functionality is required, which exists already and which would need to be built. So that is where I stand at the moment.

The downside of WPML for the weblate community might be that you need to purchase a WPML license in order to make such solution work. But honestly, given the complexity (see above) of WP, I think this is the way to go.

There is also a small cost attached to sending content back and forth using the WPML API, so this might not work for the open-source concept of WordPress, not sure. Let me know.

I would, nonetheless, still be pretty much interested in such an integration and be willing to sponsor it, in case someone is interested in building it.

lecoqlibre commented 3 years ago

@nijel WP transient can be set with no expiration time so it won't disapear until we do something. It is just a value persisted in database used for temporary data. I could use these to save Weblate instance metadata into WordPress.

Yes I already used some other WP plugins: WP Multilang and WPML.

WP Multilang store translation into the same post content's using a custom syntax like [:en]Hello world[:fr]Bonjour le monde[:]. This plugin is simple and completly free as no licence is required. But it does not support URL localization and this project seems abandonned as it has not been updated since 2 years.

WPML stores translations into new posts, supports localized URL and provide a dedicated translation interface but it requires a licence.

This two plugins are good but do like a deep copy of the posts without tracking translation strings (as I recall for WPML). The result is that if you change the content of the source language post, translators will have to find the change by themselves and edit the post in their language. Furthermore, translators must have access to your WP admin area: when the number of languages grows it might represents dozens of concurrent accesses to the WP admin area and the translation process starts to be heavy. Finally, it exists great dedicated tools for translating that tracks strings like Weblate so why reinvent the wheel into a WP plugin?

@ilocit Thank you for sharing your experience and your approach of the problem. I'm not sure to understand what is the "cost attached to sending content back and forth using the WPML API" about? Regarding WPML I would prefer not to work with projects that require a licence and don't provide public access to API documentation. Developers must be paid but I'm tired of that licencing system that get money using force. I would like to try something else.

Regarding the Weblate integration into WordPress I started developing a prototype. The main idea is to generate translation strings from WP posts and put them into Weblate. Each WP posts would have some metadata to store the source lang, the Weblate instance, project and component associated with. Translated posts would then be generated from Weblate translation units: this would make URL localization and content searching easier.

ilocit commented 3 years ago

@lecoqlibre as I thought, and understand, you don't want to work with partners/plugins which have a cost/license attached to them. And that is fine. But the problem remains that WP is quite complex and has various content types. You would need to develop a plugin which (1) handles all those content types and (2) exposes a UI to the Translation Manager (the user who is responsible for managing the translation of the respective website) with which she can create translation projects/components and send them to Weblate and more importantly, manage the status of translations etc. Quite some task, I think. WPML for instance is able to create XLF files which could be synced via a connected repo and sent to weblate. (Or via other sync/exchange methods). Those XLF files would also capture the changes to source text and show them in Weblate (with DIFF). So the translators do not need a WP account nor do they need to search for changes. Probably we have very different use cases in mind and/or picture different translation processes / scenarios ;-)

ilocit commented 3 years ago

re

Regarding the Weblate integration into WordPress I started developing a prototype. The main idea is to generate translation strings from WP posts and put them into Weblate. Each WP posts would have some metadata to store the source lang, the Weblate instance, project and component associated with. Translated posts would then be generated from Weblate translation units: this would make URL localization and content searching easier.

@lecoqlibre Would you be available to show a demo of it? And maybe discuss the different approaches?

lecoqlibre commented 3 years ago

@ilocit Yes I would be able to show a demo in few days and available to discuss the different approaches. I guess it would be very useful to write our respective translation processes/scenarios. Have you some time and would you like to do it?

nijel commented 3 years ago

Crowdin or Transifex are integrated into WPML, see https://wpml.org/translation-service/transifex/, https://wpml.org/translation-service/crowdin/ or https://blog.crowdin.com/2021/07/22/wpml-crowdin-integration-for-wordpress-localization/.

lecoqlibre commented 3 years ago

When I used WPML I did not tried integrated third party services like Transifex or Crowdin. These integrations look good and it would be easier I guess to integrate Weblate into WordPress using WPML. But are the WPML policy and philosophy complying with the Weblate's ones?

Furthermore, on WPML limitations we can read that "WPML is licensed under GPL" but on WPML terms and conditions we can read that we "cannot distribute any of our plugins for free or sell them.": does not that violate GPL clause 2 (the freedom to distribute copies)?

Technical information about WPML can be found here. By the way when viewing this page I recall that WPML use custom database tables which are not recommended by the WordPress team. For now, my plugin prototype's tries to use only native WordPress tables.

nijel commented 3 years ago

I'd prefer to have libre integration for WordPress, but if WPML is the most widely spread solution, it's probably way to go.

Looking at other plugins, there is https://wordpress.org/plugins/weglot/ which seems to work instead of WPML.

ilocit commented 3 years ago

Looking at other plugins, there is https://wordpress.org/plugins/weglot/ which seems to work instead of WPML.

Yes, Weglot is an alternative but way more expensive than WPML! (https://weglot.com/pricing/)

ilocit commented 3 years ago

Crowdin or Transifex are integrated into WPML, see https://wpml.org/translation-service/transifex/, https://wpml.org/translation-service/crowdin/ or https://blog.crowdin.com/2021/07/22/wpml-crowdin-integration-for-wordpress-localization/

Yes, that is why I looked at WPML too. It is just a well established solution in the WP ecosystem. I guess building a new PlugIn which handles all this on WP is a nice idea. Competition is never a bad thing, but it is also a very tedious job and WPML (plus some others) are already really well established.

But as I said, if someone were to build a new Multilingual Plugin for WP specifically for and with Weblate integration, fantastic. I will support this. I just think it is now a straight forward task. But maybe I am wrong and just haven't spoken to the right developers thus far ;-)

ilocit commented 3 years ago

@lecoqlibre

Yes I would be able to show a demo in few days and available to discuss the different approaches. I guess it would be very useful to write our respective translation processes/scenarios. Have you some time and would you like to do it?

Great! I will try to put a short document together next week. OK? Thanks!

ilocit commented 3 years ago

re WPML, I guess, when looking at the files/code WPML installs in WP, there is a license.txt in the main folder which mentions GNU GENERAL PUBLIC LICENSE Version 3.

So maybe WPML core modules could be evaluated and reused in order not to start from scratch? Problem would be how to maintain their security updates and future releases to keep it working with future releases or WP itself. Difficult to judge, for me at least.

nijel commented 3 years ago

I did not mean to use Weglot, but to look at how they handle the localization and whether that approach is usable for Weblate as well. If their plugin is friendly licenced (I didn't check), some of it's code could be reused as well.

ilocit commented 3 years ago

Right. Makes sense, I will have a look a their license model.

ilocit commented 3 years ago

OK. General remark, since this topic starts with Transifex Live, I now got the impression that we are talking very different scenarios @lecoqlibre . Is your goal to build a "proxy-based" translation solution for WordPress utilizing Weblate, like Transifex Live or many others, like smartling, etc...?

Because that is very different from a process where you build what I would call "localization packages" within WP and send them (manually - already possible now, with e.g. WPML, semi-automatically or automatically) to Weblate for translation, translate using weblate and retrieve the translations back in WP for publishing. That is actually what I am looking for, since I do not believe in proxy-services as a viable solution for customers. "Proxy-solutions" are easy to setup and easy to use (for the most part), but they lock you in with one solution and one vendor.

I am leaning more towards professional managed solutions where you also have the freedom of choice and can opt-out and migrate to another vendor rather easily.

What is your plan/goal?

nijel commented 3 years ago

The proxy like solution could evolve from Weblate CDN. It does cover the HTML extraction and publishing the translations, but it lacks the proxy part (what would be a huge project).

When talking about WordPress integration, my expectation was to have a WordPress plugin that would push strings to translate to Weblate, receive translations from Weblate and render them localized. This will most likely need some extensions in the Weblate API (or at least introduction of webhooks to notify WordPress about new translations).

lecoqlibre commented 3 years ago

Yes my goal is to provide a WP plugin able to extract strings from any post, page, category, or whatever WP object and push them to whatever Weblate instance. Once translated from Weblate the plugin will be able to load the translations into WP to deliver the website in another language (translated).

To help us to understand each other, here are some screenshots of the prototype I'm making:

image Here you can add any Weblate instance you want to translate your WP website with.

image As an example, you see on this admin list of posts, a French post with an English version available in the "Translations" column. The URL of the French post is "/fr/bonjour-tout-le-monde" while the URL of the English post is "/en/hello-world".

image On the edit screen of every WP post, page or other, I'm adding a "Weblate" metabox to connect the WP object to a Weblate component.

Is this what you want too?

ilocit commented 3 years ago

Looking good! Glad we could clarify the issue about the proxy-solution, this is much in-line with my view on WP translation.

Also, thanks for sharing the screens.

What I don't understand though, why would you add the info/selection option for Weblate Instance to each page/post (element)? Shouldn't that be a site-wide configuration option in the plugin settings? Along with the Weblate project. For the Component (name), yes, it might be nice to set, and perhaps edit, this piece for each page/topic/element.

How did you plan to go about the "generation" of the Component name?

Plus, one would need to select another PlugIn to enable multilingual support in WP in general, I assume? Or do you plan to also include this in your Weblate PlugIn?

In case it was a 3rd Party PlugIn, I understand there will most likely be some interdependencies, no?

Anyhow, not sure that this github issue is the right form to discuss this in such detail. Bu as I said, I am happy to support in one or the other fashion.

lecoqlibre commented 3 years ago

@ilocit I added the Weblate instance selection option to each page/post because one could have components hosted on different instances let's say componentA is hosted on instanceA.com and componentB is hosted on instanceB.com. Bulk actions could also be added to attach several posts/pages to the same instance at once.

About the generation of the component name I would use the post type followed by the lowercase post title. For example if your post title is "Hello world", the component name would be "post - hello world". But I guess it would be nice to let the user choose this name if the default does not suit. I plan to also let change this default with some hooks (apply_filters). Do you have any suggestions about this naming or other proposals?

What do you call "multilingual support of WP"? That is planned if that means to handle Website title, subtitle and every other important WP options and settings plus the SEO.

@nijel If the plugin I'm making has a chance to become a Weblate's officially supported plugin I guess a good place to discuss all this would be on a dedicated repo with tickets system (I'm used to framagit.org, a french instance of Gitlab hosted by Framasoft)?

nijel commented 3 years ago

For official Weblate parts, I want to stick with GitHub. You can start the repo in your namespace and once it's ready, it can be transferred to @WeblateOrg.

ilocit commented 3 years ago

@lecoqlibre

What do you call "multilingual support of WP"? That is planned if that means to handle Website title, subtitle and every other important WP options and settings plus the SEO.

Well, when you look at the options in e.g. WPML, there is a lot you might need to consider and make configurable. Looks like this: image

Options are, for example:

Other Options / Settings

And so on ;-)

And you need to adjust this with each new release or WP, probably. So I guess it might be worthwhile thinking about "just" supporting and using another, existing PlugIn, no? What do you think? How far should all this be taken? There are other multilingual plugins, simpler ones like Bogo. But one might need all the above, depending on the complexity of the site...

Difficult choice.

Adding to this, since you will not know from within Weblate, how much of your site is translated, you will for sure need such kind of overview (ideally per Content Type) in WP: image

ilocit commented 3 years ago

An alternative, for techies ( ? ;-) ), could be to simply put all the above in a structured file. That is basically how WPML does it, I understand. So the config you can manage through the UI is saved in a wpml-config.xml and placed in the root folder of the plugin or theme. See https://wpml.org/documentation/support/language-configuration-files/

lecoqlibre commented 3 years ago

@nijel @ilocit OK for Github. To start a repo we will need a name for the Project. For now I just called my plugin "Weblate". I also thought about "Wordlate" (like Weblate but with the Word of WordPress) or "WPot" (WordPress + POT files). Keep in mind that this name should be shown into the WordPress plugins repository so WP users can easily install it (one click). Any ideas?

lecoqlibre commented 3 years ago

@ilocit Thanks, yes, pretty much of the list you mentioned have to be implemented. You are right about supporting another plugin to do this stuff. That is code re-usability and it's great! But as you said it is not simple as that because we need to find I think a plugin that handles these things without conflicting with our plugin, without licensing issues, without hidden costs or trackers. It also must be stable and frequently updated. It should be well written and easy to reuse. Depending of the complexity of this plugin, sometimes it could be better to start a new one from scratch.

So for me the agenda could be like:

  1. Write use cases, validate them with priority order and write specifications.
  2. List existing plugins that could be reused.
  3. Validate plugins that satisfies our needs and choose one.
  4. Evaluate cost of development of each use case.
  5. Get funding for each use case and start implementing them.
  6. Test each implemented use case.
  7. When everything is great, release plugin!

@nijel Regarding the funding, does the Weblate project could fund such a plugin's development?

nijel commented 3 years ago

AFAIR there were some customers interested in funding this, @orangesunny will know more.

lecoqlibre commented 3 years ago

@orangesunny Do you know some people interested in this plugin? Maybe a crowdfunding would be an option?

I created a new repo and started to write use cases.

lecoqlibre commented 3 years ago

For now I wrote 4 use cases + 1 for introduction:

@nijel @ilocit can you provide some feedback, especially for use cases 2, 3 and 4?

lecoqlibre commented 3 years ago

@nijel @ilocit do you validate the previous use cases? I got no response for 9 days so I'm wondering if you are still interested by this plugin or maybe there is something wrong?

ilocit commented 3 years ago

Sorry, @lecoqlibre , for not being more responsive and supportive recently. Problem was and is, that we are currently a little swamped with projects... I will review your use case ASAP.

lecoqlibre commented 3 years ago

@ilocit thank you for telling me. I can work some few days on other projects waiting for your feedback.

lecoqlibre commented 3 years ago

Regarding how to handle a change made on a source string in the WordPress plugin, should we use Gettext as a monolingual format to handle that change automatically? Because with bilingual po files we would have to apply the change to every translation string within the plugin or to tell the user of Weblate to use a module on the Weblate component (msgmerge). @nijel @ilocit what is your opinion?

lecoqlibre commented 3 years ago

@nijel Why Weblate does not allow the source string edit when using bilingual GetText files? It could apply the change on the source string of every translated .po files, right?

nijel commented 3 years ago

It could, but in most cases this is not the actual source of the string - it is extracted from the source code. So changing the source string in the PO file is not going to change the code where the string is defined. If the PO files are generated from some database and you want to edit the source as well, use some identifier as msgid, store the translation is msgstr and use monolingual setup.