Feedback loop between localizers and Mother Tongue/Mozilla staff

[gueroJeff]

We need to define a feedback loop between localizers and Mother Tongue/Mozilla staff. This should seek to leverage bugzilla, as all parties have access to the platform.

The general idea would be:

Mother Tongue translates content -> Content is published to mozorg -> Localizers casually visit mozorg -> Localizers see an error with their language -> Localizers file a bug -> Mother Tongue acts on the bug.

This will require:

• [x] Bugzilla template
• [x] A keyword designation to flag Mother Tongue
• [x] A recommended bugmail filter for Mother Tongue to use

[gueroJeff]

The bugzilla template should:

• Assign the bug to Pei (as PM, she's responsible for making sure the feedback is assessed and resolved with Mother Tongue)
• Label the bug as a "Defect"
• Use the locale-specific component
• Use the new keyword: vendor-l10n-error
• Use a description template for reporting these issues uniformly

[flodolo]

Most of this can be done by sharing a URL. The only thing remaining would be selecting the locale component.

[gueroJeff]

Most of this can be done by sharing a URL. The only thing remaining would be selecting the locale component.

Agreed, I can't create the URL until the new keyword is created though. Mostly trying to outline requirements for when the time comes.

[flodolo]

Agreed, I can't create the URL until the new keyword is created though.

Should be use the whiteboard instead? There are no constraints there, and that would unblock us.

[gueroJeff]

We could use the whiteboard instead. Let's give bugzilla admins at least a day to respond to the bug I filed though. If they don't respond by tomorrow, we'll go with the whiteboard until they can create the new keyword.

On Thu, Mar 25, 2021 at 6:33 AM Francesco Lodolo @.***> wrote:

Agreed, I can't create the URL until the new keyword is created though.

Should be use the whiteboard instead? There are no constraints there, and that would unblock us.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/mozilla-l10n/pm-projects/issues/290#issuecomment-806655507, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOXEZEWR27TJ6QQPTWICDTTFMUQ7ANCNFSM4ZZGJTCA .

[gueroJeff]

The keyword has been created.

I took a stab at defining some bugmail filters for Mother Tongue. I don't use bugmail filters, so this is kind of new to me. I'd appreciate any feedback:

To receive notifications of new Mozilla Localization bugs that contain a keyword, you need to set up a series of bugmail filters. A prerequisite is that you are first watching the relevant components to your work (see Component Watching settings). 

First you need an exclude filter to exclude all change notifications made to bugs in for locale components you are watching:
Field:  __Any__
Product:    Mozilla Localizations
Component:  __Any__
Relationship:   Watching
Changer:    (empty) 
Action:     Exclude

Then you need an include filter to indicate that you want to receive notifications when a keyword is added to a Mozilla Localization bug for locale components you are watching (note that we only use the vendor-l10n-error keyword):
Field:  Keyword
Product:    Mozilla Localizations
Component:  __Any__
Relationship:   Watching
Changer:    (empty)
Action:     Include

And finally, to receive bugmail about relevant changes to those bugs, you'll need the following filters:

1) When there's a new comment on the bug:
Field:  Comment created
Product:    Mozilla Localizations
Component:  __Any__
Relationship:   Watching
Changer:    (empty)
Action:     Include

2) When a flag request has been changed on a bug:
Field:  Flag Requestee
Product:    Mozilla Localizations
Component:  __Any__
Relationship:   Watching
Changer:    (empty)
Action:     Include

3) When the bug has been assigned to a user:
Field:  Assignee
Product:    Mozilla Localizations
Component:  __Any__
Relationship:   Watching
Changer:    (empty)
Action:     Include

4) When the bug has been resolved:
Field:  Resolution
Product:    Mozilla Localizations
Component:  __Any__
Relationship:   Watching
Changer:    (empty)
Action:     Include

[gueroJeff]

I think we need two templates here:

The first is for reporting a single error.

The second is for reporting a batch of errors.

[flodolo]

For the "following" part: I'm honestly not sure this will work. The first step excludes all mails from components you're watching, so I expect the next rules to not be applied (there's nothing left).

I also don't know if you could set up an include rule for Mozilla Localizations without watching the product first. The examples below seem to hint at that (e.g. Then an include filter to indicate that you want to receive notifications when a bug is created:), so maybe they just need one rule, without watching the product at all

Field:  Keyword
Product:    Mozilla Localizations
Component:  __Any__
Relationship:   __Any__
Changer:    (empty)
Action:     Include

Maybe ask in #bmo on Slack, to see if someone can confirm it?

The single error template looks good, for the second one I really feel like the entry barrier is way too high (I would give up) :-(

[gueroJeff]

Thanks for taking a look at this :-)

For the "following" part: I'm honestly not sure this will work. The first step excludes all mails from components you're watching, so I expect the next rules to not be applied (there's nothing left).

This was part of the recommendations for complex filters in the bugmail filters section of preferences. Granted, I haven't done this before, so I'm only basing my assumptions on what they say there. I can ask in #bmo though.

I also don't know if you could set up an include rule for Mozilla Localizations without watching the product first. The examples below seem to hint at that (e.g. Then an include filter to indicate that you want to receive notifications when a bug is created:), so maybe they just need one rule, without watching the product at all

I thought the same thing, so I added a prerequisite at the beginning that you have to be watching the product for these filters to work. Again, I'll ask in #bmo to be sure.

Field:    Keyword
Product:  Mozilla Localizations
Component:    __Any__
Relationship:     __Any__
Changer:  (empty)
Action:   Include

Maybe ask in #bmo on Slack, to see if someone can confirm it?

The single error template looks good, for the second one I really feel like the entry barrier is way too high (I would give up) :-(

I imagine the second will rarely be used, but in order for the vendor to work through the feedback quickly on like 20+ translations, the feedback has to be well-structured and concise.

[peiying2]

Do we still want to use Bugzilla to funnel all the linguistic feedback? How do we make it discoverable, especially feedback comes from mostly outside of Mozilla? So far, error reports come from a few channels:

• File a bug but not always through Mozilla L10n but through mozilla.org website product.
• Communication through Matrix or even email to me.

Source of errors:

• If translation error -
  • If vendor's error, depending on content type, the fix can be a few ways, Smartling (fix the source so TM is correct), track down the final output, Survey (might be late), SuMo, or In-App description
• If the source, I file an issue directly at the product repo.

Who to be notified on the vendor side:

• Since we are not always sure who is responsible for the error, I don't think it's a good idea to auto cc someone.
• So far, I investigate, relay the error issue but most of the fixes have to be done by us. And I record the errors so I can look back the history within a calendar year.

[peiying2]

@flodolo given what I experienced this year, should we approach this differently? More fluidly in terms of how reports reach us?

[flodolo]

How do we make it discoverable, especially feedback comes from mostly outside of Mozilla?

Most of the feedback comes from localizers, right?

I think Bugzilla is still the right place for it, because that's where bugs in localized products are normally reported. But there's no point in asking vendors to interact there, since we're moving everything to Jira for that side of things.

I think it's OK to fix errors ourselves, for sake of speed and possibly cost, but we need a way to report them back. If we use the keyword vendor-l10n-error, it's easy to group them in search results, but it might be too much work. Maybe track a summary manually in a spreadsheet (locale, project, type of error) and then link to the bug for more details?

[peiying2]

How do we make it discoverable, especially feedback comes from mostly outside of Mozilla?

Most of the feedback comes from localizers, right?

Correct. For mozilla.org content, copywriter has direct access to Smartling to make changes. I need to pull report to see activities, otherwise not obvious.

I think Bugzilla is still the right place for it, because that's where bugs in localized products are normally reported. But there's no point in asking vendors to interact there, since we're moving everything to Jira for that side of things.

I think it's OK to fix errors ourselves, for sake of speed and possibly cost, but we need a way to report them back. If we use the keyword vendor-l10n-error, it's easy to group them in search results, but it might be too much work. Maybe track a summary manually in a spreadsheet (locale, project, type of error) and then link to the bug for more details?

The reports could come from bugzilla or repo issue. I have kept a tracker here. After each report, I always relay the message to the responsible vendor. Let me know if we have a better way to keep a record of these. Do we need to write a document on how to fix the error (spreadsheet, Smartling platform, marketing followup), document it, relay the message to vendors, etc.

[flodolo]

The reports could come from bugzilla or repo issue. I have kept a tracker here. After each report, I always relay the message to the responsible vendor.

I feel like it's missing some important data:

• Date when it was reported
• If feedback was sent to the vendor
• When it was fixed

[peiying2]

The reports could come from bugzilla or repo issue. I have kept a tracker here. After each report, I always relay the message to the responsible vendor.

I feel like it's missing some important data:

* Date when it was reported

* If feedback was sent to the vendor

* When it was fixed

Good points. I can add columns to include the information. The bugs are recorded in chronological order. The links of issues/bugs have dates in them - when the bugs were reported and fixed/closed. The vendor feedback is not documented explicitly.

Also the fix on our side means different things for different type of content. Mozilla.org is right away, and app store description would take longer.

[peiying2]

Let me know if this is close to what you have in mind: - click on QualityIssues tab

[flodolo]

What is the difference between "l10n update" and "production update"?

"Production" is misleading, it should be "Date fixed" (i.e. when the user started seeing the fixed version).

[flodolo]

Note: the rest looks good to me, I think that covers everything we need.

[peiying2]

What is the difference between "l10n update" and "production update"?

"Production" is misleading, it should be "Date fixed" (i.e. when the user started seeing the fixed version).

We will discuss this at our 1on1. There is fix at the source (on the Smartling platform so TM is up to date) and the final fix that goes to production, which we don't always have complete control or visibility.