Crowdin Connector

[plamarque]

Rationale

Content translation is a very accessible and popular activity in communities. One of the best solutions for translation management is Crowdin.com. It has 120K projects and 2 million users.

Meeds is currently running a rewarding program for translation, but it requires significant review effort and can only work in batches. A technical integration would allow the capture and point allocation to be automated.

1. Functional Requirements

The Crowdin Connector should be capable of:

• capturing individual translation submissions on designated Crowdin projects
• authenticate and map the Corwdin user account to the Meeds hub account
• capture when a translation is added
• capture when a translation is approved
• capture when a translation is upvoted
• capture when a translation is commented

Top User Stories

Contributor

• As a contributor, I can connect my Crowdin account through my Hub settings by doing a simple authentication with Crowdin.

• As a contributor, I can earn points in my Meeds hub only by working in the right project with the right Crowdin account

Program Owner

• As a program owner, I can add an automated action to reward a fixed amount of points when users on Crowdin :

  • Add translations
  • Comment on strings
  • Approve translations
  • Receive approval for their translations
  • Vote on translations
  • Receive upvotes on their translations

• As a program owner, I can create an automated action that will track translations on a specific folder of the Crowdin project

• As a program owner, I can create an automated action that will track translations on a specific language of the Crowdin project

• As a program owner, I can create an automated action that will track only human translations (rather than MT or TM)

Connector Administration

• As a hub administrator, I can allow/disallow users to attach their Crowdin account to their hub settings through oAuth authentication.

• As an administrator, I can select the Crowdin projects I want to gamify

• As an administrator, I can manage the following events, and their corresponding canceling events are available

Event Name	Description	Contribution Owner	Cancel Event
Add Translation	Suggest a translation	user who suggested the translation	The translation is deleted
Approve Translation	Approve a translation	user who approved the translation	Approval is removed
Translation Approved	User translation is approved	user who suggested the approved translation	Approval is removed
Comment String	Comment on a string	user who commented	String comment or issue is deleted

• As an administrator, I can enable/disable some gamification events on a per-project basis

Impacts

Gamification

Notifications

Analytics

• Track actions triggered
• Track languages used

Unified Search

2. Technical Requirements

Expected Volume & Performance

Terminology

• The same as described in Github connector

Security

*The Oauth Tokens don't have to be stored in the Meeds database, but only used to validate user accounts on external provider

• The API key (secret) used by Crowdin to transmit triggered events into the Meeds server has to be generated by the rewarding manager using a specific UI in the Gamification administration UI inside Crowdin connector settings. This Meeds API Key has to be encrypted using CodecInitializer before storing it in the database.

• A Crowdin oAuth Client & Secret keys have to be provided by rewarding admin in a dedicated UI. Those oAuth Keys have to be encrypted using CodecInitializer before storing it in the database. Those Keys will be used to validate the oAuth Token provided by the user from the Meeds Server to the Crowdin Server.

Extensibility

• The same as described in Github connector

Configurability

• The same as described in Github connector

Upgradability

NA

Existing Features

NA

Feature Flags

No feature flag is needed.

Other Non Functional Requirements

Source code The Crowdin Connector should have a dedicated code repository.

Packaging The Crowdin Connector should be packaged as a separate add-on.

Webhooks

https://developer.crowdin.com/api/v2/#operation/api.projects.webhooks.post	Event	Contribution user	Crowdin Webhook Event	Cancel Event
Add Translation	translation.user.username	suggestion.added	suggestion.deleted
Approve Translation	translation.user.username	suggestion.approved	suggestion.disapproved
Translation Approved		suggestion.approved	suggestion.disapproved
Comment on a String	comment.user.username	stringComment.created	stringComment.deleted

4. Software Architecture

Security

• The Crowdin Webhook will have to connect to the REST using an authentication Bearer:

  Authorization: Bearer <meeds_api_key>

Note: meeds_api_key: Key generated through UI that has to be personal and non-unique. This key will be encrypted before storing it inside Database, using CodecInitializer.

Warning: The Access Token of user must not be stored in internal database.

• The Crowdin oAuth Keys have to be encrypted using CodecInitialize before storing them in the database using SettingService with Global context and Crowdin Application Scope.

REST endpoints

Some REST endpoints have to be added to serve new assets:

• /crowdin/hooks : this will allow the admin to manage webhooks (add, update, delete)
• /crowdin/webhooks: this endpoint will manage external webhooks events.

Services & processing

• The external triggers handling has to be made asynchronously through a queued executor. When the WebHook sends events, it has to be queued and then asynchronously processed by gamification by invoking io.meeds.gamification.service.RealizationService.createRealizationsAsync. The queue has to hold the objects sent by WebHook without parsing data

Data and persistence

• Crowdin events will be stored in the existing GAMIFICATION_EVENTS gamification table
• User Crowdin account will be stored in the existingGAMIFICATION_CONNECTOR_ACCOUNTS gamification table
• Crowdin Connector Settings will be stored in the global setting
• Crowdin webhooks will be stored in a new table CROWDIN_WEBHOOKS

[plamarque]

@srenault-meeds @margondicco I'm submitting this new MIP for the Crowdin Connector, providing the ability to incentivize community translators

[srenault-meeds]

@plamarque I have a question. In order to boost translations for a language or a folder, it would be really interesting to add a parameter to the action like language, folder. WDYT?

[plamarque]

Yes, I have requirements for that but no mockup. I intend to implement them as additional fields in step 2 of the drawer

[srenault-meeds]

Ok yes it would be nice indeed. No blocker for me of course.

Go fonc, of course

[AzmiTouil]

Technical spec added. To check please @rdenarie

[rdenarie]

Hello, go tech

[AzmiTouil]

MIP ready for review by the DAO members (cc @rdenarie ). Acc up & running.

[rdenarie]

Ok, for me, you can proceed to merge