Use Weblate for continuous translation

[zdb999]

I've been looking at Weblate, and I think could serve as the translation framework discussed in CONTRIBUTING.md. It integrates directly with source control, monitoring for changes in the default strings. It then imports them into Web UI for translation, and alerts interested translators who speak a given language. Projects can define a quality control process if they wish, and Weblate supports defining glossaries and translation memories. After strings are translated, Weblate will then submit a pull request to the source code repository with the updates. This was briefly discussed in https://github.com/sandstorm-io/sandstorm/issues/557.

My initial idea would be that this would a three stage process: 1) Adapt and package Weblate as a Sandstorm app 2) Start a community instance, and configure it to work with Sandstorm's main repo. 3) Create a process for app makers to make their apps available for translation through the community instance. A single Weblate instance can handle many projects, and a (voluntary) centralized interface will help connect translators and and apps that need them.

Does this seem like a good idea? Would this be a good project to pursue given the current state of the project?

Also, would it actually make sense for Sandstorm to run this hypothetical service as an app, or would it make more sense to run the service in a more traditional container? The first option has the advantage of dogfooding, but it would also require giving all translators accounts on whatever Sandstorm server actually runs the instance, and it has all the drawbacks of Sandstorm's current problems hosting non-static content on the open internet.

[zenhack]

I like the idea of porting weblate and trying it out. Getting the PR functionality going will probably be a bit of work, since we'll need to adapt it to use the powerbox API, but I think it's also a good opportunity for us to start dogfooding that in earnest.

but it would also require giving all translators accounts on whatever Sandstorm server actually runs the instance

Not really; we just need to put a share link up somewhere. You don't need special access to a sandstorm box to use a grain that's been shared with you.

A single Weblate instance can handle many projects, and a (voluntary) centralized interface will help connect translators and and apps that need them.

The more sandstorm-ish way to do this would be to separate grains by project, and just create a grain per project. Often this requires a bit of work during porting, but there are many examples of apps where we've retrofitted the one-document model onto an app that is built to handle multiple docs.

For connecting translators with apps, I have an alternate idea: We could add a field that developers could set in their app's package definition, which would link to a place to help with translations. If defined, we'd display a "help translate ${app}" link next to the links we already display for the app's source code, website, etc. If we wanted to get fancy we could play with making it more prominent if the app isn't already translated into the user's language or something, but just the basic link I think is probably useful for discovery.

and it has all the drawbacks of Sandstorm's current problems hosting non-static content on the open internet.

I'm not sure this is really a problem here; I think especially since the goal is to offer folks a place to translate sandstorm (and sandstorm apps), it won't be too weird for users to follow a share link to get there, and have the usual sandstorm UI around it.

[zenhack]

Also, if we're interested in porting weblate, we might want to weigh in on https://github.com/WeblateOrg/weblate/issues/2825; continued support for sqlite would be useful for sandstorm.

[zenhack]

(Though the problems they're trying to address there are likely to have an even bigger impact on sandstorm, so... trade-offs)

[zdb999]

Not really; we just need to put a share link up somewhere. You don't need special access to a sandstorm box to use a grain that's been shared with you.

This is true, but giving translators accounts has advantages. They can subscribe to get notifications when there are needed translations in a language they speak, they get credit in Weblate and in version control, they can review each other's work, and there is more resistance to vandalism. (Not that the last issue is likely to be a problem in the near future, but I hope we get popular enough to worry about such things.)

Visitor accounts would provide all the necessary capacities though.

The more sandstorm-ish way to do this would be to separate grains by project, and just create a grain per project. Often this requires a bit of work during porting, but there are many examples of apps where we've retrofitted the one-document model onto an app that is built to handle multiple docs.

I agree this is a more Sandstorm-y approach, and I think it's worth supporting for other users. Weblate has a great dashboard interface that helps connect translators to apps they may initially not know or personally care about, but which they would like to be in their language. I think it would be great to replicate that experience as much as possible, and putting every project in a different grain would cut against that discoverability in my view.

I'm not sure this is really a problem here; I think especially since the goal is to offer folks a place to translate sandstorm (and sandstorm apps), it won't be too weird for users to follow a share link to get there, and have the usual sandstorm UI around it.

I agree with you here, if it's sufficiently well implemented.

[ocdtrekkie]

@zdb999 Anyone can sign into any Sandstorm server (unless it's explicitly disabled because it's an internal org server), but is not granted the ability to create their own grains. (As an example, you can go log into alpha.sandstorm.io right now.) There would be no reason to 'give every translator an account' on the Sandstorm server Weblate is on, in the way we think about giving someone an account. They can still sign in, receive notifications, get credit in Weblate, because they would still log into Sandstorm, and Sandstorm would still manage their user account and integration with apps.

While Sandstorm apps often miss out on dashboard interfaces and discoverability, I think it's still the right approach. We can invest effort in connecting people via Sandstorm's own app store and website, rather than inside the Weblate app. I think @zenhack's specific metadata entry for translation may or may not be a bit ambitious, I suppose it depends how many apps support translations. If it's generic and lets people link to anywhere, it will probably be worthwhile, as I know Wekan uses Transifex, Etherpad uses TranslateWiki, etc. But then we also need to make sure we're keeping pace with upstream versions (coughEtherpadcough).

I totally agree with Ian that Weblate operating within the Sandstorm frame should be fine. Sandstorm contributors expect Sandstorm links, and they may be managing multiple Weblate instances they view for different apps and all.

[xet7]

Wekan uses Transifex, because it has always used it already.

For Weblate, is there some reason already hosted Weblate can not be used for doing translations? https://hosted.weblate.org/

Or is this generally about porting Weblate to Sandstorm?

[ocdtrekkie]

It is probably a little of both. Presumably using a proper translation platform would be easier for drive-by translation contributors to add even partial work to. But we also really should try to dogfood the tools needed to develop Sandstorm on Sandstorm as much as possible.

[zenhack]

Yeah, I was thinking it would just be an arbitrary link.

[zdb999]

Anyone can sign into any Sandstorm server (unless it's explicitly disabled because it's an internal org server), but is not granted the ability to create their own grains.

I agree this is all translators would need.

Yeah, I was thinking it would just be an arbitrary link.

I think this should be an option, but there should also be a way for app packagers to easily get a Weblate grain on Alpha or another similar server. App makers/packagers should not need to run their own server to take advantage of the translation framework/system.

Down the road, we could hypothetically modify Weblate to export information about its status via api, and then use that information to create a discoverable dashboard on the app store itself.

[zdb999]

On additional thought. Weblate does assume the ability to have multiple projects in a single instance. The basic hierarchy is

Project > Repo > String,

Where each project can have many repos. Would it make sense to modify Weblate to have a single project per grain?