Android string download should not include newlines

tipa commented 1 week ago

Describe the issue

When downloading Android translation strings (either through the website or the API), strings that include a line break have - additionally to the expected and required "\n" - a newline character in the export XML. This causes the string, when loaded in an Android app, to have an additional, unwanted space character: https://stackoverflow.com/a/31683548/3585218

I already tried

[X] I've read and searched the documentation.
[X] I've searched for similar filed issues in this repository.

Steps to reproduce the behavior

Download "Android String Resource" from base_url/projects/my_proj/my_component/de/#download (or alternatively the API)
Open in text editor

Expected behavior

No newlines inside the string-tags in XML files, only \n

Screenshots

Current behavior:

Expected behavior:

Exception traceback

No response

How do you run Weblate?

Other

Weblate versions

Weblate: 5.7.2
- Django: 5.0.8
- siphashc: 2.3
- translate-toolkit: 3.13.1
- lxml: 5.2.1
- pillow: 10.3.0
- nh3: 0.2.14
- python-dateutil: 2.8.2
- social-auth-core: 4.5.3
- social-auth-app-django: 5.4.0
- django-crispy-forms: 2.1
- oauthlib: 3.2.2
- django-compressor: 4.4
- djangorestframework: 3.15.1
- django-filter: 24.2
- django-appconf: 1.0.5
- user-agents: 2.2.0
- filelock: 3.12.4
- rapidfuzz: 3.4.0
- openpyxl: 3.1.2
- celery: 5.4.0
- django-celery-beat: 2.6.0
- kombu: 5.3.4
- translation-finder: 2.16
- weblate-language-data: 2024.6
- html2text: 2024.2.26
- pycairo: 1.26.0
- PyGObject: 3.44.1
- diff-match-patch: 20230430
- requests: 2.32.3
- django-redis: 5.4.0
- hiredis: 2.2.3
- sentry-sdk: 2.5.1
- Cython: 3.0.4
- mistletoe: 1.3.0
- GitPython: 3.1.40
- borgbackup: 1.2.6
- pyparsing: 3.1.1
- ahocorasick_rs: 0.20.0
- python-redis-lock: 4.0.0
- charset-normalizer: 3.3.0
- cyrtranslit: 1.1.1
- Python: 3.11.10
- Git: 2.36.6
- phply: 1.2.6
- ruamel.yaml: 0.17.40
- tesserocr: 2.6.2
- boto3: 1.34.94
- zeep: 4.2.1
- aeidon: 1.14.1
- iniparse: 0.5
- Mercurial: 6.7.2
- git-svn: 2.36.6
- PostgreSQL server: 15.7
- Database backends: django.db.backends.postgresql
- PostgreSQL implementation: psycopg2
- Cache backends: default:RedisCache, avatar:FileBasedCache
- Email setup: django.core.mail.backends.smtp.EmailBackend: mrkos.uberspace.de
- OS encoding: filesystem=utf-8, default=utf-8
- Celery: redis+socket:///home/tipako/.redis/sock, redis+socket:///home/tipako/.redis/sock, regular
- Platform: Linux 3.10.0-1160.119.1.el7.tuxcare.els2.x86_64 (x86_64)

Weblate deploy checks

System check identified some issues:

WARNINGS:
?: (security.W008) Your SECURE_SSL_REDIRECT setting is not set to True. Unless your site should be available over both SSL and non-SSL connections, you may want to either set this setting True or configure a load balancer or reverse-proxy server to redirect all connections to HTTPS.
?: (weblate.W033.Gerrit) Failure in loading VCS module for Gerrit: git: 'review' is not a git command. See 'git --help'.
 (1)
        HINT: https://docs.weblate.org/en/weblate-5.7.2/vcs.html
?: (weblate.W039.AlibabaTranslation) Error importing module 'weblate.machinery.alibaba' in WEBLATE_MACHINERY: No module named 'aliyunsdkalimt'
        HINT: https://docs.weblate.org/en/weblate-5.7.2/admin/machine.html

System check identified 3 issues (1 silenced).

Additional context

No response

nijel commented 1 week ago

Indeed, translate-toolkit (which Weblate uses to manipulate with the translation files) adds newline to improve XML readability. The additional whitespace due to the newline is added at the end of the string, what should be pretty harmless. To be precise, translate-toolkit does what SO answer you've shared suggests as a correct approach to this since https://github.com/translate/translate/pull/3606.

tipa commented 1 week ago

I don't agree with the "solution" in the SO post I linked. This comment also highlights the problem with the newlines.

Yes, the additional whitespace at the end of each line is harmless or unnoticeable in most of the cases (especially when used in uneditable, unselectable labels), but not in all of them. I noticed it in my app in a case where the multi-line localized string resource is the default for a user-editable value, shown in an editable text field. In the end, it's not the exact string that was translated in Weblate - and that should be the expected result imo.

github-actions[bot] commented 1 week ago

The issue you've reported needs to be addressed in the translate-toolkit. Please file the issue there, and include links to any relevant specifications about the formats (if applicable).

nijel commented 1 week ago

While I understand you, I'm a bit hesitant in breaking behavior translate-toolkit has for years. Let's see if somebody objects to https://github.com/translate/translate/pull/5352

WeblateOrg / weblate