i18n: Support generating pot files

grappler commented 6 years ago

In Gutenberg you can run npm run gettext-strings which runs cross-env BABEL_ENV=gettext webpack to create a POT file. We should look into what it takes to have a POT file created for a plugin.

ahmadawais commented 6 years ago

Definitely! Would you like to contribute that? I'll welcome that change. You'd add that to the scripts folder in cgb-scripts packages inside the packages folder of this repo. It should run what is needed to build a pot file.

grappler commented 6 years ago

After looking at it in more detail I would like to find a better way then copying https://github.com/WordPress/gutenberg/blob/master/i18n/babel-plugin.js to the plugin and using that.

grappler commented 6 years ago

This is where the plugin is registered https://github.com/WordPress/gutenberg/blob/d907d35/.babelrc#L37-L43

ahmadawais commented 6 years ago

@grappler Yes, I can see that. We can add this to the feature list of version 2.0.0 and you can work on it if you'd like to contribute.

We can create a new package for this in the packages dir and then improve babel-preset-cgb. What do you say?

ahmadawais commented 6 years ago

@grappler Any thoughts?

Viper007Bond commented 6 years ago

Isn't using POT files kinda deprecated now? https://make.wordpress.org/meta/handbook/documentation/translations/

ahmadawais commented 6 years ago

@grappler Any updates on this one?

grappler commented 6 years ago

Isn't using POT files kinda deprecated now?

I am not sure if the Make-POT on WordPress.org yet parses JS files. Would be something to look into.

If it is not supported then it makes sense to use NPM to create the POT files in the time being. I would love to work on this but I have a few other tasks to work on before I can work on this.

grappler commented 6 years ago

This has been partly started with https://github.com/ahmadawais/create-guten-block/commit/6c92f4af8f3fe156c4ad4770255edfa63f1facc2

Viper007Bond commented 6 years ago

You're right. Even Gutenberg itself generates a POT file: https://github.com/WordPress/gutenberg/tree/master/i18n

I'm so used to using wp_localize_script().

ahmadawais commented 6 years ago

@grappler I'd be happy to see you lead this feature of create-guten-block? I can offer help where I can. @Viper007Bond is right. We can mimic Gutenberg's .POT file generation and add it here.

grappler commented 6 years ago

It looks like Gutenberg is changing their .POT generation process. https://github.com/WordPress/gutenberg/pull/6007

@ahmadawais What I can't figure out is which package.json do I need to add "@wordpress/babel-plugin-makepot": "1.0.0",.

ahmadawais commented 6 years ago

@grappler Read this https://github.com/ahmadawais/create-guten-block/blob/master/CONTRIBUTING.md

You'd want to do all that in cgb-scripts folder and it's package.json. This is a monorepo which means it hosts multiple packages in one repo. The root is create-guten-block CLI which has nothing to do with the process, it only creates a folder and install cgb-scripts + package.json file to it and rest is all done by packages/cgb-scripts.

grappler commented 6 years ago

Here is an example how the make-pot can be setup for a single block https://github.com/swissspidy/gutenberg-i18n-block/blob/master/package.json

I am not sure how to add the babel task to the build.js.

ahmadawais commented 6 years ago

@grappler here's the babel config for create-guten-block https://github.com/ahmadawais/create-guten-block/tree/master/packages/babel-preset-cgb

and here is the webpack https://github.com/ahmadawais/create-guten-block/tree/master/packages/cgb-scripts/config

maximebj commented 6 years ago

Hope you'll add this feature soon!

FYI A friend of mine use pot-to-php and makepot to do this with his tool. https://github.com/loicblascos/gutenberg-custom-blocks

ahmadawais commented 6 years ago

I can add it pretty easily, I am just so short on time nowadays. Also, I am confident @grappler will soon pick it up 👍 and will add this feature for the upcoming release.

ahmadawais commented 6 years ago

Also pretty sure WPCLI cannot translate from JavaScript.

grappler commented 6 years ago

Sorry, I am not sure I am going to have time to work on this feature. 😞

ahmadawais commented 6 years ago

That's sad! @asharirfan 🤔

swissspidy commented 6 years ago

Quick update on https://github.com/wp-cli/i18n-command on https://github.com/swissspidy/gutenberg-i18n-block:

Work on the WP-CLI command has been completed and it's part of WP-CLI 2.0, which was released yesterday.
The Gutenberg I18N Block contains different branches with different examples: with or without Babel, with or without WP-CLI
The WP-CLI command extracts strings from JS and PHP

For plugins hosted on WordPress.org:

Until we're able to update WordPress.org and WordPress core, your best bet is to generate a PHP file containing the extracted strings, just like Gutenberg does it. This is to ensure that WordPress.org can find these strings and users can translate them.

Use wp.i18n.setLocaleData and gutenberg_get_jed_locale_data to pass the translations to your script.

For private plugins:

Just use WP-CLI to extract all the strings and generate a POT file for you. Load translation files as usual with load_plugin_textdomain().

Use wp.i18n.setLocaleData and gutenberg_get_jed_locale_data to pass the translations to your script.

ahmadawais commented 6 years ago

Thanks for the update @swissspidy 👏

ambienthack commented 3 years ago

UPDATE: removed references to Loco Translate

Came across the problem of block localization for blocks created using create-guten-block. Haven't localized blocks before, so had to go through trial an error. Below is my current workflow.

Preparation

1.1. create an empty /languages folder inside your block plugin

Generation of translation files

2.1. generate .pot by executing wp i18n make-pot . languages/[your-text-domain].pot in the root folder of your plugin 2.2. for every string in .pot, add #: dist/blocks.build.js:1 reference. before:

#: src/contacts/block.js:17
msgid "Contacts"
msgstr ""

#: src/cover/block.js:17
msgid "Cover"
msgstr ""

after:

#: src/contacts/block.js:17
#: dist/blocks.build.js:1
msgid "Contacts"
msgstr ""

#: src/cover/block.js:17
#: dist/blocks.build.js:1
msgid "Cover"
msgstr ""

(see How .json translations work for details): 2.3. create .po files 2.4. generate .json translation files with wp i18n make-json languages --no-purge

Loading translation files

3.1. locate generated init.php file inside your plugin 3.2. locate generated function function your_plugin_name_cgb_block_assets() in init.php 3.3. inside the function locate wp_register_script( '[you_plugin_js_handle]', ... ) call and note the handle name 3.4. at the end of your_plugin_name_cgb_block_assets() function add wp_set_script_translations( '[you_plugin_js_handle]', '[your-text-domain]', plugin_dir_path( dirname(__FILE__) ) . 'languages' ); (replace values inside square brackets with the handle name and the text domain name).

You are done! Translations in your block should work now.

How .json translations work

To register a block, several php function calls should be done (this php code is generated by create-guten-block). One of these calls is wp_register_script( '[you_plugin_js_handle]', ... ), which registers a script with js code of your block. Note that when you register translations for your block using wp_set_script_translations( '[you_plugin_js_handle]', ... ), translations are bound to the same js handle. Also note, that wp_register_script() registers js script file with the compiled code of your block, located in dist folder of your plugin. So, the translations are bound to the compiled js file of your block.

Here is an example of translation .json file name: your-text-domain-ru_RU-a4f783c851c57db1967c3d9a58cfd198.json. It consists of three parts: domain name, locale name and md5 signature of the associated .js file path. So, the third part should be md5('dist/blocks.build.js'), which is a4f783c851c57db1967c3d9a58cfd198. wp i18n make-json calculates this automatically and assigns appropriate file names to .json files. It takes file paths from comments inside of .po files (remember #: dist/blocks.build.js:1 ?).

Why doesn't .pot file have the correct js references automatically? Because i18n calls (__(), etc) are done inside the source js files, located in the src folder, so references (created automatically) point to the files inside the src folder. That is why step 2.2. is important.

swissspidy commented 3 years ago

No need for the manual steps in 2.2 nor using Loco Translate if you just use WP-CLI all the way, and generate the pot file from the build files instead the source folder. Just an FYI :-)

ambienthack commented 3 years ago

@swissspidy had no luck with it, i think because compiled files contain module source code inside strings:

(function(module, __webpack_exports__, __webpack_require__) {

"use strict";
eval("/* harmony import */ var __WEBPACK_IMPORTED_MODULE_0_classnames__ = __webpack_require__(/*! classnames */ 0);
...
from wp.blocks\n\nregisterBlockType('arbora/cover', {\n\ttitle: __('Cover', 'my-text-domain'), 
...
");
}),

am i missing something?

swissspidy commented 3 years ago

That shouldn't matter, the important piece is that the function calls like __('Cover', 'my-text-domain') aren't mangled. You might need to adjust the webpack/terser config if string extraction can't find the strings from the build files.

ambienthack commented 3 years ago

It looks like wp i18n cannot deal with code packed inside eval'ed strings, as it analyzes abstract code tree. 'eval' modules are produced by webpack in create-guten-block dev mode. If you make a production build with npm build, produced js files can be parsed with wp i18n. Webpack config option of interest is devtool: 'cheap-eval-source-map'.

So here is a refined version of section Generation of translation files:

2.1. make a production build of your plugin with npm run build (not dev build produced by npm start) 2.2. generate .pot by executing wp i18n --exclude=src make-pot . languages/[your-text-domain].pot in the root folder of your plugin 2.3. create .po files 2.4. generate .json translation files with wp i18n make-json languages --no-purge

Thanks, @swissspidy

ahmadawais / create-guten-block