Add Betterbird localization

[mfschumann]

The main version currently only ships the en-US version of Betterbird, the version built from source (branch 3-build-from-source) ships unmodified Thunderbird localization.

-> Replace Thunderbird localization by the Betterbird-patched localization when available.

See https://github.com/Betterbird/thunderbird-patches/tree/main/102/scripts

[Betterbird]

Localisation is a problem. Those scripts are for Windows, but could easily be made to work on Linux since they use Unix commands.

Our aim is to cover the top 10 TB locales as can be seen here: https://stats.thunderbird.net/#platlang. en, de, ja, fr, pl, ru, it, es, pt-BR, nl. Off these we cover all but "it", pl an ru. We have a script for "it", but not shipping it yet (attached to https://github.com/Betterbird/thunderbird-patches/issues/24).

It's not really efficient to ship an entire 50+MB package just to change the language. TB offers the option to install language packs, but in BB we disabled the UI for that so people wouldn't install incompatible TB language packs which cause BB to malfunction and not even start. We get the occasional report and it's also clearly stated on your support page. We have prepared a de language pack for 91 for a single (paying!) user who wanted it for Mac where we're not localising at all due to lack of (crosss platform) tools to handle HFS disk images.

All in all, we could discuss how to handle localised versions best on Linux. The Arch people ship lots of packages which is equally inefficient.

I've never used TB much on Linux. However, I remember that some distros ship many/all languages in a single build which makes language menus overflow and is hard to manoeuvre. And frankly, most people speak one language, or maybe two or even three or four, so why flood them with UI for many. Also, unless the PC is really used by a close-knit family, we don't see why you would want to change the language of the UI just because you can 😉

Let's get in touch and work out something.

[mfschumann]

The flatpak manifest for Thunderbird downloads all the localisation XPI files and includes them in the flatpak. As the Betterbird manifest is based on the one for Thunderbird, using a version of the XPI files that is adapted for Betterbird would be easiest to implement on my side. We would then have to configure Betterbird such that it uses the Betterbird XPI according to the locale if available, and falls back to en-US otherwise (if that's not already the case).

As far as I understand the scripts in the Betterbird repo, most of the adaption work is already done there. It's just that the scripts do not generate modified XPI files.

[Betterbird]

We have https://github.com/Betterbird/thunderbird-patches/blob/main/91/scripts/langpack.cmd. That produces the language pack XPIs for BB, but we don't ship them. The script is a total-hack-one-off, it expects the en-US branding-chrome and branding-localization in the directory where you execute it.

[mfschumann]

I have updated apply-patches.sh to add the Betterbird-specific localization strings to the Thunderbird localization XPIs and ship those in the flatpak, resulting in the following build with working localization: https://github.com/flathub/eu.betterbird.Betterbird/pull/5#issuecomment-1241560324

I have set up packaging such that only those locales are shipped that fully support Betterbird (judging from the names of the scripts in https://github.com/Betterbird/thunderbird-patches/tree/main/102/scripts).

Installation and selection of the locale with flatpak works like this:

1. Users can configure flatpak to install specific locales for all apps that they install. This setting defaults to the system locale, but it can also be set to a list of arbitrary locales.
2. When the user installs a new flatpak app with a locale extension available, it downloads and installs the locales that match the user configuration and are available in the locale extension (meaning for Betterbird: only those fully supported).
3. Betterbird uses the locale that matches the settings in the current profile if available.

@Betterbird I understand that you switch off the UI for selecting the locale in https://github.com/Betterbird/thunderbird-patches/blob/main/102/misc/05-misc-no-multi-lingual.patch because it lures users into downloading incompatible Thunderbird language packs via the UI. If downloading is the main problem here, could we instead set

pref("intl.multilingual.enabled", true);
pref("intl.multilingual.downloadEnabled", false);

? That would disable the download part of the UI, but still show the part for selecting which of the already installed locales shall be used. Like this, users that have configured flatpak to install multiple locales would be able to switch between them via the UI.

Example with flatpak set to install de and fr locales:

[Betterbird]

Thanks for investing so much time into this. Two comments:

You wrote: "... to add the Betterbird-specific localization strings to the Thunderbird localization XPIs", That's only half of what it does. Sadly the TB folks constantly make localisation errors which they correct later at such short notice that the localisers can't catch up.

Previously, we had a lot more error corrections, now we have these ones which are specific to certain languages: https://github.com/Betterbird/thunderbird-patches/blob/497c37a256caa8271643185dd0e61658e4033a48/102/scripts/ja.cmd#L52 https://github.com/Betterbird/thunderbird-patches/blob/497c37a256caa8271643185dd0e61658e4033a48/102/scripts/fr.cmd#L55 https://github.com/Betterbird/thunderbird-patches/blob/main/102/scripts/de.cmd#L55-L67 In Japanese, a keyboard shortcut is missing leading to the rejection of the entire string, in French localisers somehow missed two strings when translating four new ones. German is particularly bad since the German localiser adheres to the strict localisation rules and doesn't make any changes after the initial localisation for the first release of TB NNN.0 ESR. So if more string changes ship in .y with y > 0, they are missing. If you run stock-standard TB in German, you'll notice those strings in English.

Also note that we take the TB language packs from a particular build, see here: https://github.com/Betterbird/thunderbird-patches/blob/497c37a256caa8271643185dd0e61658e4033a48/102/scripts/repack-linux.cmd#L13 Like for the source, that could be noted in the artifacts file.

The second issue is the language pack switch. Yes, as you said, that's switched off to NOT "lure users into downloading incompatible Thunderbird language packs via the UI". I've had some requests to public a BB-compatible set of language packs, so I think I'll go with the simple solution you suggested. For the time being, feel free to modify or replace the patch for testing purposes.

[Betterbird]

Here you go (untested): https://github.com/Betterbird/thunderbird-patches/commit/7a29c1c8cb59f5abe46139af48e4f4dc4ff4009d

[mfschumann]

Thank you for the detailed explanation. When packaging, I actually execute the *.cmd scripts (after making them compatible with bash), so I do get all your error corrections, too.

Regarding the build I am assuming that you use the same build for the Thunderbird source and the language packs. If that is not always true we should note the build for the language packs in the .appdata.xml as you suggested.

[Betterbird]

Damn, forgot the main part: https://github.com/Betterbird/thunderbird-patches/commit/4d10fa2f7e97423fb84c70a1727b0379f46b17dc

In the past I haven't always used most recent language packs since they don't always change. I could start doing that though.

So from the source entry in .appdata.xml you navigate to the language pack location? http://ftp.mozilla.org/pub/thunderbird/candidates/102.2.2-candidates/build1/source/thunderbird-102.2.2.source.tar.xz becomes http://ftp.mozilla.org/pub/thunderbird/candidates/102.2.2-candidates/build1/linux-x86_64/xpi/LLLL.xpi ?

P.S.: I'll produce some language packs myself and publish them so users can switch.

[mfschumann]

So from the source entry in .appdata.xml you navigate to the language pack location? http://ftp.mozilla.org/pub/thunderbird/candidates/102.2.2-candidates/build1/source/thunderbird-102.2.2.source.tar.xz becomes http://ftp.mozilla.org/pub/thunderbird/candidates/102.2.2-candidates/build1/linux-x86_64/xpi/LLLL.xpi

Yes.

P.S.: I'll produce some language packs myself and publish them so users can switch.

Cool. Let me know when they are ready and I will get them from you instead of applying the patches to the Thunderbird langpacks myself (that part was quite hacky).

[Betterbird]

Well, I'll publish them mostly for Windows and Mac users. You're aim is to "build from source", so you wouldn't want the ones I'll be building.

[mfschumann]

Ah ok, I wasn't aware the language packs were platform specific. Would it be possible to turn the *.cmd scripts adding the Betterbird magic to the Thunderbird language packs into something more platform-independent (e.g. a patch file)? That would remove the hackiness on my side.

[Betterbird]

We've just added https://github.com/Betterbird/thunderbird-patches/blob/main/102/scripts/langpack.cmd based on the earlier 91 version. It successfully created a language pack. We'll produce and publish the packs for the supported languages later today.

As for your questions: Language packs aren't platform specific, I was just saying that we'll publish them for the largest user group on Windows and for Mac users since there is no localisation for Mac at all.

Still, your aim is to build everything from source, so it wouldn't make sense to include "foreign" language packs in your otherwise "own/native" build.

The current approach is to localise BB from the TB language packs whose content is patched accordingly. That is a little hacky, and even more so for you since you bend the .cmd scripts into shape. But that work is done now. Additionally, as of today, we'll also provide patched language packs and will allow switching again in the next release (can be done how by flipping the prefs).

[mfschumann]

Ok, then let's keep the current process. Do we need an additional artifact in the .appdata.xml to communicate a possibly different build for the language packs or are you comfortable with always using the same build for source and language pack?

[Betterbird]

Yes, the source link from .appdata.xml plus our .cmd scripts will always give a good result when the language packs are taken from the same build as the source.