Message context for Yii::t()

[Deele]

I feel that Yii::t() lacks ability to provide context of textual value and when translations start to count in couple hundreds, it starts to be quite hard to understand what was meant with translation.

If we look at popular translation systems like "Crowdin" (example), they allow to provide message context to support translators.

Currently, only way to provide some helpful information using Yii::t() is to write something in "key" area, that in turn defies purpose of "source language", for example, if you write a message "Propose (button in issue tasklist table header)" until I change that translation to "Propose", that message still be visible.

I am guessing, that there is performance issue with long keys, and length limitations does not allow long context descriptions.

For backwards compatability, I would like to suggest 5th optional attribute to Yii::t($category, $message, $params = [], $language = null, $context = null) (but this is not very convenient) that would allow to provide some sort of context description.

Maybe, we could convert Yii::t() to allow passing in first attribute array of attributes, like Yii::t(['category' => 'message.category', 'message' => 'message text', 'params' => [], 'language' => null, 'context' => null]) or even allow shorthand for category and message with numeric keys Yii::t(['message.category', 'message text', 'params' => [], 'language' => null, 'context' => null]).

With \yii\i18n\DbMessageSource, they would be stored in separate table source_message__context, where we would store source message ID and context description with 255 characters long string.

With \yii\i18n\PhpMessageSource, they would be stored in separate file, right next to current message files, in {category}_context.php format, where we would have associative array with source message category as a key, and array with context descriptions, as a value.

I have no experience with \yii\i18n\GettextMessageSource, but I see in their documentation, that they already have an option to provide context descriptions. But I would like to see some suggestions for this solution.

This feature would allow to display all of those context description together with message in interface, to help translator to translate messages and understand context of it.

[Deele]

@lynicidn The context is required because you can have message that is related to the same page, form or block, but still, it requires a bit explanation so that translator can understand better the gist of it. For example, when message string contains variables and translator needs to know what sort of thing is meant to go there.

@PowerGamer1 Message string is used as a key to identify message, making it longer just gives bigger overhead for translation lookup procedure.

@samdark To wrap it up from my point of view, this is what script procedure should look like in order of precedence (correct me if I'm wrong):

1. Multi-line comment right after Yii::t(), but before concatenation dot or semicolon

function foo() {
    $variable = 'Lorem ipsum ' .
        Yii::t('dymmy.category', 'Message yiisoft/i18n#1') /* Hint for "dymmy.category:Message yiisoft/i18n#1" */ .
        'dolor sit amet, ' .
        'consectetur ' .
        'adipiscing elit.';
}

Resulting hint:

Hint for "dymmy.category:Message yiisoft/i18n#1"

Multi-line hint (ignoring leading * (leading white-space, star, white-space) prefix like in PHPDocumentator):

function foo() {
    $variable = 'Lorem ipsum ' .
        Yii::t('dymmy.category', 'Message yiisoft/i18n#1') /*
        Hint for "dymmy.category:Message yiisoft/i18n#1".
        Second line of hint for "dymmy.category:Message yiisoft/i18n#1".
        */ .
        'dolor sit amet, ' .
        'consectetur ' .
        'adipiscing elit.';
    $variable = 'Lorem ipsum ' .
        Yii::t('dymmy.category', 'Message yiisoft/i18n#2') /*
         * Hint for "dymmy.category:Message yiisoft/i18n#2".
         * Second line of hint for "dymmy.category:Message yiisoft/i18n#2".
         */ .
        'dolor sit amet, ' .
        'consectetur ' .
        'adipiscing elit.';
}

Resulting hint for dymmy.category:Message yiisoft/i18n#1:

Hint for "dymmy.category:Message yiisoft/i18n#1"\nSecond line of hint for "dymmy.category:Message yiisoft/i18n#1".

Resulting hint for dymmy.category:Message yiisoft/i18n#2:

Hint for "dymmy.category:Message yiisoft/i18n#2"\nSecond line of hint for "dymmy.category:Message yiisoft/i18n#2".

Multi-line hint in HTML/PHP template:

<div class="row">
    <div class="col-md-6">
        <h2><?= Yii::t('dymmy.category', 'Message yiisoft/i18n#1') /* Hint for "dymmy.category:Message yiisoft/i18n#1" */ ?></h2>
        <p class="text-info"><?= Yii::t('dymmy.category', 'Message yiisoft/i18n#2') /*
    Hint for "dymmy.category:Message yiisoft/i18n#2"
    Second line of hint for "dymmy.category:Message yiisoft/i18n#1".
*/ ?></p>
    </div>
</div>

Resulting hint for dymmy.category:Message yiisoft/i18n#1:

Hint for "dymmy.category:Message yiisoft/i18n#1"

Resulting hint for dymmy.category:Message yiisoft/i18n#2:

Hint for "dymmy.category:Message yiisoft/i18n#2"\nSecond line of hint for "dymmy.category:Message yiisoft/i18n#2".

2. Single-line comment right after Yii::t() and line ending semicolon

function foo() {
    $variable = Yii::t('dymmy.category', 'Message yiisoft/i18n#1'); // Hint for "dymmy.category:Message yiisoft/i18n#1"
}

Resulting hint:

Hint for "dymmy.category:Message yiisoft/i18n#1"

3. Multi-line comment right after Yii::t() and line ending semicolon

function foo() {
    $variable = Yii::t('dymmy.category', 'Message yiisoft/i18n#1'); /* Hint for "dymmy.category:Message yiisoft/i18n#1" */
}

Resulting hint:

Hint for "dymmy.category:Message yiisoft/i18n#1"

Multi-line hint (ignoring leading * (leading white-space, star, white-space) prefix like in PHPDocumentator):

function foo() {
    $variable = Yii::t('dymmy.category', 'Message yiisoft/i18n#1'); /*
    Hint for "dymmy.category:Message yiisoft/i18n#1".
    Second line of hint for "dymmy.category:Message yiisoft/i18n#1".
    */
    $variable2 = Yii::t('dymmy.category', 'Message yiisoft/i18n#2'); /*
     * Hint for "dymmy.category:Message yiisoft/i18n#2".
     * Second line of hint for "dymmy.category:Message yiisoft/i18n#2".
     */
}

Resulting hint for dymmy.category:Message yiisoft/i18n#1:

Hint for "dymmy.category:Message yiisoft/i18n#1"\nSecond line of hint for "dymmy.category:Message yiisoft/i18n#1".

Resulting hint for dymmy.category:Message yiisoft/i18n#2:

Hint for "dymmy.category:Message yiisoft/i18n#2"\nSecond line of hint for "dymmy.category:Message yiisoft/i18n#2".

4. Single-line comment right after Yii::t() and concatenation dot

function foo() {
    $variable = 'Lorem ipsum ' .
        Yii::t('dymmy.category', 'Message yiisoft/i18n#1') . // Hint for "dymmy.category:Message yiisoft/i18n#1"
        'dolor sit amet, ' .
        'consectetur ' .
        'adipiscing elit.';
}

Resulting hint:

Hint for "dymmy.category:Message yiisoft/i18n#1"

5. Single-line comment in previous line, where Yii::t() was used

function foo() {
    // Some other comment
    // Another comment that is ignored
    // Hint for "dymmy.category:Message yiisoft/i18n#1"
    $variable = Yii::t('dymmy.category', 'Message yiisoft/i18n#1');
}

Resulting hint:

Hint for "dymmy.category:Message yiisoft/i18n#1"

Same hint for all messages that are used in same line

function foo() {
    // Some other comment
    // Another comment that is ignored
    // Hint for "dymmy.category:Message yiisoft/i18n#1" and "dymmy.category:Message yiisoft/i18n#2"
    $variable = 'Lorem ipsum ' . Yii::t('dymmy.category', 'Message yiisoft/i18n#1') . 'dolor sit amet, ' . Yii::t('dymmy.category', 'Message yiisoft/i18n#2') . 'consectetur adipiscing elit.';
}

Resulting hint for dymmy.category:Message yiisoft/i18n#1:

Hint for "dymmy.category:Message yiisoft/i18n#1" and "dymmy.category:Message yiisoft/i18n#2"

Resulting hint for dymmy.category:Message yiisoft/i18n#2:

Hint for "dymmy.category:Message yiisoft/i18n#1" and "dymmy.category:Message yiisoft/i18n#2"

6. Multi-line comment in previous line, where Yii::t() was used

function foo() {
    // Some other comment
    // Another comment that is ignored
    /* Hint for "dymmy.category:Message yiisoft/i18n#1" */
    $variable = Yii::t('dymmy.category', 'Message yiisoft/i18n#1');
}

Resulting hint:

Hint for "dymmy.category:Message yiisoft/i18n#1"

Multi-line hint (ignoring leading * (leading white-space, star, white-space) prefix like in PHPDocumentator):

function foo() {
    /* Ignored comment */
    /*
    Hint for "dymmy.category:Message yiisoft/i18n#1".
    Second line of hint for "dymmy.category:Message yiisoft/i18n#1".
    */
    $variable = Yii::t('dymmy.category', 'Message yiisoft/i18n#1');
    // Ignored comment
    /*
     * Hint for "dymmy.category:Message yiisoft/i18n#2".
     * Second line of hint for "dymmy.category:Message yiisoft/i18n#2".
     */
    $variable = Yii::t('dymmy.category', 'Message yiisoft/i18n#2');
}

Resulting hint for dymmy.category:Message yiisoft/i18n#1:

Hint for "dymmy.category:Message yiisoft/i18n#1"\nSecond line of hint for "dymmy.category:Message yiisoft/i18n#1".

Resulting hint for dymmy.category:Message yiisoft/i18n#2:

Hint for "dymmy.category:Message yiisoft/i18n#2"\nSecond line of hint for "dymmy.category:Message yiisoft/i18n#2".

[samdark]

How about nested Yii::t() calls?

[Renkas]

How or why would you nest Yii::t() calls? As far as I understand that's a thing you should never do.

[klimov-paul]

See #8286

[Renkas]

Well message parser handles it - couldn't it handle comments as well?

But in a bigger picture I think it's kind of a edge case that is very easy to write in some other way if you need to add context hints.

[bizley]

This long example is a great way of showing how overcomplicated comment hints can become. I would drop this idea in favour of category context mentioned earlier. Or we could always extend signature a bit so category can be optional array with first element being category name itself and second one being a context hint - this way is much easier to track.

[Renkas]

I would drop multi-line comment support and everything would be much easier. There's no need to go the route to support every possible comment markup and position. It would be nice but it's not necessary.

These hint's are optional and good practice would be to use them only where needed.

[Deele]

@samdark I disagree for nested calls.

@bizley No, I do not agree that categories are enough for that. They are just like folders and give simple hierarchy while not exactly explaining what variables do or how this text is used.

I think for starters, the 1. Multi-line comment right after Yii::t(), but before concatenation dot or semicolon (ignoring multiple white-space and line endings) would be enough and easiest of all to implement. Others can be added in future, when hints will be already implemented and evaluated by users of framework as a feature.

[samdark]

@Deele nested calls are reality. They're used often:

echo Yii::t('app', 'here is {something}', [
    'something' => Yii::t('app', 'another string'),
]);

[Deele]

@samdark Oh, sorry, I imagined another thing with "nested calls" - yeah, nested calls are reality and used. But the most important thing here is that they do not help to explain, to give hints about context, they just make work of translators even worse - any kind of chopping up messages and creating more of them, or increasing number of variables in messages increase work.

[bizley]

@Deele What about array parameter for category?

[Deele]

@bizley What exactly did you mean by "array parameter", show us example. The only alternative to comments I see - fifth argument for the [Yii::t()](http://www.yiiframework.com/doc-2.0/yii-baseyii.html#t()-detail) function, that will require providing optional third and fourth argument.

The problem is that translator never sees the code where that specific Yii::t() is used, so the intention is to allow describing message with both category and description, so that they could be interpreted and translated independently from application developers or code.

[bizley]

@Deele like I said in my comment. Something like:

Yii::t(['app', 'Hint for this'], 'Original text');

[Deele]

@bizley We should avoid changing syntax of existing Yii::t(). I guess, Yii::t() is most used function in any multi-lingual Yii based app, we should avoid adding any more code to it.

[bizley]

I think we have listed here all the caveats. Let's wait now for implementation proposal.

[FilipBenco]

Hi, I drafted implementation of this feature in Yii2 https://github.com/yiisoft/yii2/pull/17137 . Afterwards, I'll port it also to Yii3, but right know I need to use id in Yii2 framework. Thx.

[samdark]

@FilipBenco than you! Unfortunately, 2.0 doesn't accept enhancements since last year with little exceptions.

[FilipBenco]

That's a pity. @samdark and if I port this feature also to Yii3? I really would like to have this feature and to be integrated directly into Yii2. This way both version will have this. Is there any way, I can have this feature even in Yii2? (Besides having to maintain custom Console command)

[machour]

Hi @FilipBenco,

Although we understand your pain here, Yii 2 is officially in feature-freeze mode and we cannot bend that rule now, or else we'll open the door for more requests like that.

You can probably make a separate package for yii2 and share it with other developers. I'll adopt it in my Yii2 projects where I have faced this context problem.

I'm also really interested in seeing this land in Yii 3, don't hesitate to ping me on slack if you need help porting it.

[samdark]

https://github.com/yiisoft/yii2/pull/17137

[samdark]

In order to add this feature later we need to ensure that extra context could be passed to the message of MessageWriterInterface:

interface MessageWriterInterface
{
    public function write(string $category, string $locale, array $messages): void;
}

That won't affect the interface itself but will affect format of $messages. Likely it needs to be changed to

'key' => ['translation' => 'string', ...]

[samdark]

Interface part in this package and PHP / DB writers are done.

[samdark]

The rest should be implemented in the message parsing tool.