[Bug]: invalid template parameters in translations prevent substitution

dvtate commented 2 months ago

⚠️ This issue respects the following points: ⚠️

[X] This is a bug, not a question or a configuration/webserver/proxy issue.
[X] This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
[X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
[X] I agree to follow Nextcloud's Code of Conduct.

Bug description

I was uploading some files to my Nextcloud instance which I have the language set to Danish and noticed this message "{sekunder} sekunder tilbage". I assumed the translator mistakenly translated the template parameter causing it to not get substituted.

After seeing this I didn't find it in the any of the l10n/*.json files but did find it in some compiled javascript files in dist/ which seems suspicious:

[server]$ grep -Rl "{sekunder} sekunder tilbage" .
./dist/files-main.js.map
./dist/files-init.js
./dist/files-main.js
./dist/files-init.js.map

Why does this happen??

I also found some problems in the l10n json files. (for example)

So I threw together a script to get all the translations with fields not in the original English version.

I think we should include this script (or some better-written version of it) as a CI/unit test before releasing.

check_translations.py

```python import subprocess import glob import os import json import re translation_dirs = (subprocess .check_output([ 'find', '.', '-type', 'd', '-name', 'l10n' ]) .decode('utf-8') .split('\n')) files = [ # [file path, parsed json] ] for d in translation_dirs: for path in glob.glob(d + "/*.json"): if os.stat(path).st_size != 0: with open(path) as file: try: files.append([path, json.load(file)]) except Exception as e: print('json load failed: ', e) for t in files: for t_en in t[1]['translations']: t_to = t[1]['translations'][t_en] en_fields = re.findall('\\{([^}]+)\\}', t_en) if type(t_to) == list: # plurals? for tt in t_to: for f in re.findall('\\{([^}]+)\\}', tt): if f not in en_fields: print(t[0] + ': invalid field {' + f + '}') else: for f in re.findall('\\{([^}]+)\\}', t_to): if f not in en_fields: print(t[0] + ': invalid field {' + f + '}') ```

Steps to reproduce

set language to danish
upload a bunch of files on a slow machine in the files app
time remaining translation is wrong (probably others too)

Expected behavior

Translations should probably not translate the template params

Installation method

Other Community project

Nextcloud Server version

29

Operating system

Other

PHP engine version

PHP 8.2

Web server

Nginx

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

[X] Default user-backend (database)
[ ] LDAP/ Active Directory
[ ] SSO - SAML
[ ] Other

Configuration report

{
    "system": {
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "logfile": "\/var\/log\/nextcloud\/nextcloud.log",
        "loglevel": 3,
        "apps_paths": [
            {
                "path": "\/usr\/share\/webapps\/nextcloud\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/lib\/nextcloud\/apps",
                "url": "\/wapps",
                "writable": true
            }
        ],
        "trusted_domains": [
            "localhost",
            "***REMOVED SENSITIVE VALUE***"
        ],
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "htaccess.RewriteBase": "\/",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "29.0.3.4",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "enable_previews": true,
        "enabledPreviewProviders": [
            "OC\\Preview\\Movie",
            "OC\\Preview\\PNG",
            "OC\\Preview\\JPEG",
            "OC\\Preview\\GIF",
            "OC\\Preview\\BMP",
            "OC\\Preview\\XBitmap",
            "OC\\Preview\\MP3",
            "OC\\Preview\\MP4",
            "OC\\Preview\\TXT",
            "OC\\Preview\\MKV",
            "OC\\Preview\\AVI",
            "OC\\Preview\\mov",
            "OC\\Preview\\Image"
        ],
        "maintenance": false,
        "theme": "",
        "memories.vod.ffmpeg": "\/usr\/bin\/ffmpeg",
        "memories.vod.ffprobe": "\/usr\/bin\/ffprobe",
        "memories.vod.path": "\/var\/lib\/nextcloud\/apps\/memories\/bin-ext\/go-vod-amd64",
        "memories.vod.disable": false,
        "memories.exiftool_no_local": true,
        "memories.db.triggers.fcu": true
    }
}

List of activated Apps

No response

Nextcloud Signing status

No response

Nextcloud Logs

No response

Additional info

I have installed nextcloud from the Arch Linux repos

kesselb commented 2 months ago

Good finding, thank you :+1:

Maybe that problem could be solved by adding a translation check on transifex: https://help.transifex.com/en/articles/6241794-setting-translation-checks#h_317a8b70f5

rakekniven commented 2 months ago

@dvtate Can you provide the list of defective placeholders? I will fix them. Minutes ago I added translation checks and if it works like intended it should fix the issue.

dvtate commented 2 months ago

I pulled and re-ran the script and here's the list of invalid placeholders: list.txt

I'm not sure how to check the bundles in the dist/ folder but it seems there's there's still some problems there.

I used the attached python script to generate this list translation_check.py.txt

dvtate commented 1 month ago

My script might not be checking all the translations

nextcloud / server