Duplicated translation alert

[liminspace]

I always get Duplicated translation alert and can't understand how to fix it.

Used django generated PO-files: en, da, de, et, etc. The main (source) language is EN. All (except EN) PO-files contain some translations. I created new component using from version control and gettext PO file. PO files are imported successfully. The source language of the translation project is English.

In Repository changes block I get messages: None Found duplicated language en (en, en)

In Alerts tab I get messages: Duplicated translation. The component contains several translation files mapped to single language in Weblate. The following occurrences were found: Language Language codes English en, en Please fix this by removing one of the duplicated strings from the translation files.

BUT. When I change source language into some non-existing language, this alert is disappeared.

 * Weblate: 3.10.3
 * Django: 3.0.2
 * siphashc: 1.3
 * Whoosh: 2.7.4
 * translate-toolkit: 2.5.0
 * lxml: 4.3.2
 * Pillow: 7.0.0
 * bleach: 3.1.0
 * six: 1.14.0
 * python-dateutil: 2.8.1
 * social-auth-core: 3.2.0
 * social-auth-app-django: 3.1.0
 * django-crispy-forms: 1.8.1
 * oauthlib: 3.1.0
 * django-compressor: 2.4
 * djangorestframework: 3.11.0
 * django-appconf: 1.0.3
 * user-agents: 2.0
 * filelock: 3.0.12
 * setuptools: 40.8.0
 * jellyfish: 0.7.2
 * openpyxl: 3.0.1
 * celery: 4.4.0
 * kombu: 4.6.7
 * celery-batches: 0.2
 * translation-finder: 1.7
 * html2text: 2020.1.16
 * pycairo: 1.16.2
 * pygobject: 3.30.4
 * diff-match-patch: 20181111
 * requests: 2.22.0
 * django-redis: 4.11.0
 * hiredis: 1.0.1
 * sentry_sdk: 0.14.0
 * Cython: 0.29.14
 * misaka: 2.1.1
 * GitPython: 3.0.5
 * borgbackup: 1.1.10
 * Python: 3.7.3
 * Git: 2.20.1
 * psycopg2: 2.7.7
 * phply: 1.2.5
 * chardet: 3.0.4
 * ruamel.yaml: 0.16.5
 * tesserocr: 2.5.0
 * akismet: 1.0.1
 * boto3: 1.11.5
 * zeep: 3.4.0
 * aeidon: 1.6.0
 * Mercurial: 4.8.2
 * git-svn: 2.20.1
 * Database backends: django.db.backends.postgresql
 * Cache backends: default:RedisCache, avatar:FileBasedCache
 * Email setup: django.core.mail.backends.smtp.EmailBackend: 172.212.0.1
 * Celery: redis://cache:6379/1, redis://cache:6379/1, regular
 * Platform: Linux 4.9.0-12-amd64 (x86_64)

System check identified some issues:

WARNINGS:
?: (security.W004) You have not set a value for the SECURE_HSTS_SECONDS setting. If your entire site is served only over SSL, you may want to consider setting a value and enabling HTTP Strict Transport Security. Be sure to read the documentation first; enabling HSTS carelessly can cause serious, irreversible problems.

INFOS:
?: (weblate.I021) Error collection is not configured, it is highly recommended for production use
    HINT: https://docs.weblate.org/en/weblate-3.10.3/admin/install.html#collecting-errors
?: (weblate.I028) Backups are not configured, it is highly recommended for production use
    HINT: https://docs.weblate.org/en/weblate-3.10.3/admin/backup.html

[nijel]

The problem is that your source language is "en" and you also have translation for "en", what doesn't make much sense. Either remove the English translation or set Weblate to ignore it using language regex. I've added such explanation to the alert in 60e14ceaa63faf0d69bf83847e4ef152e4a71290.

[github-actions[bot]]

Thank you for your report, the issue you have reported has just been fixed.

• In case you see a problem with the fix, please comment on this issue.
• In case you see similar problem, please open separate issue.
• If you are happy with the outcome, consider supporting Weblate by donating.

[vladox]

@nijel how can you enable translation from "en" to "en" to override the strings used in the code for the base language? is that possible at all?

[nijel]

It's not possible for bilingual translations. The problem is that this would change the English translation, but not the source string for all other languages, what could easily lead to inconsistencies in them. The better approach for this is to fix the strings in the source code (for example with help of source string review) or switching to monolingual translations, where the source code contains just identifiers and the English translation is at same level as others.

[vladox]

But there a case when you are not editing the source strings, that's when you generate a PO file for the source language in a bilingual setup. For instance if the source language is English then the locale/en/LC_MESSAGES/django.po file is generated. We have that setup that helps us to reduce development time by letting the translators correct strings in the source by themselves. We can implement this as a opt-in option if you see this as a valid use case.

[nijel]

What you can do in this case is to change source language on the project. Creating something like English (developer) (en@devel) will make it clear what is going on and will avoid conflict with existing English translation.

[vladox]

@nijel thanks for the suggestion, after some trial and error is clear that the solution we are looking for is achieved by using a monolingual setup. It's working perfectly, the only part that is not very easy to understand is the concept of "intermediate file", what would be a use case for that setting?

[nijel]

Something is described at https://docs.weblate.org/en/latest/admin/projects.html#intermediate-language-file, I will try to extend it

[hnyman]

I think that we at OpenWrt have stumbled to a bit similar situation with weblate. We have

• about 50 components/submodules in GUI to be translated
• gettext based bilingual translation files
• English defined as source language, as the project source code is in English
• source string templates are is po/templates/xxx.pot at the upstream project
• language translations are in language specific po/DE/xxx.po files
• we have own scripts upstream that keep the source strings regularly in sync in the bilingual translation files upstream. (Thus we do not actually need weblate's automatics in updating source strings in the language specific .po files)
• There is also a possible "translation" for English is case we later want to change punctuation or something in the display also for English. So there is also language "en"

That "en" English confuses weblate, and we get "duplicate language alerts".

The mostly unused English also seems to lock strings in case somebody has accidentally a edited a string in the "English translation" of a component. Then all empty strings in "en" are evaluated as untranslated source strings and are shown as "Source in review" in other languages and locked from translation to other languages.

Based on the discussion above,

• one solution might be to define an unused language as the source, or even invent a language like "English (developer)". But I can't figure out where that new language could be created in the weblate user interface. Any hints for that? Or could there be that kind of choice in the default lanaguage selection list?
• Other solution mentioned above might be to "set Weblate to ignore it using language regex". We currently have just the default regex ^[^.]+$ there. What would be the acceptable regex to exclude "en" English from the scanned translation files? (as far as I understand, that setting would need to be done separately for each component, right?)

[nijel]

I've made changes to the alert text and documentations to cover your questions.

See https://hosted.weblate.org/projects/openwrt/luciapplicationsopkg/#alerts

[hnyman]

Thanks. I tried changing the source language to "English (Developer)".

But since then I have noticed that some components cause errors when I try visiting translation strings in them, and in some components part of the strings are shown as "read only". Almost like weblate still remembers the status it set for the string when the source language was previously "English". That might still be due to something like no upstream git change, so weblate has not updated its source string table, or something like that.

So I wonder if I need to perform some weblate-wide cleanup or refresh action after the change of "source language".

Additionally, can we somehow lock the "English (developer)", so that nobody can translate to that? Worst case would be that somebody clicks the visible offer of translating the "English (developer)" and we end up in the same situation as earlier. "English (developer)" should remain disabled as translation language for casual users..

(And naturally we don't have any actual po/en_devel/xxx.po files at our repo, so we would not like weblate to start to introduce such ones.)

Or should we also edit the language filter somehow? Would hiding the English devel file be a better option?

[hnyman]

Thanks. Your commit on preventing caching of English seems to have cleared most problems, so far.

I still wonder how to prevent polluting "English (developer)" from translations. Sooner or later somebody used the "pen" icon next to source strings, or otherwise semi-accidentally starts to translate that.

Could it possible to declare the source language "read-only" in the project settings? (or any language?)

[nijel]

The source language is read only in bilingual translations. It was messed up before because Weblate didn't properly handle migration when actual translation becomes source one.