XLIFF <file> tag not handled correctly for monolingual setup

mclei-asw commented 4 years ago

Describe the bug When monolingual XLIFF setup is used, then newly translated texts ignores the "file" tag of source XLIFF and gets inserted in the first "file" tag in target XLIFF. It also generated non-valid XLIFF file with duplicated entries.

To Reproduce Steps to reproduce the behavior:

Source language file:

<?xml version='1.0' encoding='UTF-8'?>
<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" version="1.2" lang="en-US">
  <file original="APP.txt" source-language="en-US" datatype="x-asw">
    <body>
      <trans-unit id="text_title">
        <source>Title</source>
      </trans-unit>
    </body>
  </file>
  <file original="MENU.txt" source-language="en-US" datatype="x-asw">
    <body>
      <trans-unit id="appmenu_Open">
        <source>Open</source>
      </trans-unit>
    </body>
  </file>
</xliff>

Translated file:

<?xml version='1.0' encoding='UTF-8'?>
<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" version="1.2" lang="cz-CZ">
  <file original="APP.txt" source-language="en-US" datatype="x-asw">
    <body>
      <trans-unit id="text_title">
        <source>Title</source>
      </trans-unit>
    </body>
  </file>
  <file original="MENU.txt" source-language="en-US" datatype="x-asw">
    <body>
      <trans-unit id="appmenu_Open">
        <source>Open</source>
      </trans-unit>
    </body>
  </file>
</xliff>

Create component with these files in Weblate

Add new source text in the source file under second "file" tag (added appmenu_Close trans-unit):

<?xml version='1.0' encoding='UTF-8'?>
<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" version="1.2" lang="en-US">
<file original="APP.txt" source-language="en-US" datatype="x-asw">
<body>
  <trans-unit id="text_title">
    <source>Title</source>
  </trans-unit>
</body>
</file>
<file original="MENU.txt" source-language="en-US" datatype="x-asw">
<body>
  <trans-unit id="appmenu_Open">
    <source>Open</source>
  </trans-unit>
  <trans-unit id="appmenu_Close">
    <source>Close</source>
  </trans-unit>
</body>
</file>
</xliff>

Translate all texts and commit.

The wrong result is: The appmenu_Close trans-unit gets inserted under first "file" tag.

<?xml version='1.0' encoding='UTF-8'?>
<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" version="1.2" lang="cz-CZ">
  <file original="APP.txt" source-language="en-US" datatype="x-asw" target-language="cs-CZ">
    <body>
      <trans-unit id="text_title">
        <source>Title</source>
        <target>Titulek</target>
      </trans-unit>
      <trans-unit id="appmenu_Close">
        <source>Close</source>
        <target>Zavřít</target>
      </trans-unit>
    </body>
  </file>
  <file original="MENU.txt" source-language="en-US" datatype="x-asw">
    <body>
      <trans-unit id="appmenu_Open">
        <source>Open</source>
        <target>Otevřít</target>
      </trans-unit>
    </body>
  </file>
</xliff>

Expected result is: The appmenu_Close trans-unit must be inserted under second "file" tag.

<?xml version='1.0' encoding='UTF-8'?>
<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" version="1.2" lang="cz-CZ">
  <file original="APP.txt" source-language="en-US" datatype="x-asw" target-language="cs-CZ">
    <body>
      <trans-unit id="text_title">
        <source>Title</source>
        <target>Titulek</target>
      </trans-unit>
    </body>
  </file>
  <file original="MENU.txt" source-language="en-US" datatype="x-asw">
    <body>
      <trans-unit id="appmenu_Open">
        <source>Open</source>
        <target>Otevřít</target>
      </trans-unit>
      <trans-unit id="appmenu_Close">
        <source>Close</source>
        <target>Zavřít</target>
      </trans-unit>
    </body>
  </file>
</xliff>

BUT it is getting even worse and generates NON-valid XLIFF file

Add one more source text (appmenu_Quit trans-unit under second "file" tag):

<?xml version='1.0' encoding='UTF-8'?>
<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" version="1.2" lang="en-US">
<file original="APP.txt" source-language="en-US" datatype="x-asw">
<body>
  <trans-unit id="text_title">
    <source>Title</source>
  </trans-unit>
</body>
</file>
<file original="MENU.txt" source-language="en-US" datatype="x-asw">
<body>
  <trans-unit id="appmenu_Open">
    <source>Open</source>
  </trans-unit>
  <trans-unit id="appmenu_Close">
    <source>Close</source>
  </trans-unit>
  <trans-unit id="appmenu_Quit">
    <source>Quit</source>
  </trans-unit>
</body>
</file>
</xliff>

pull the changes in Weblate

Now the already translate trans-unit "appmenu_Close" shows as untranslated.

translate again all texts and commit

The WRONG result is:

<?xml version='1.0' encoding='UTF-8'?>
<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" version="1.2" lang="cz-CZ">
  <file original="APP.txt" source-language="en-US" datatype="x-asw" target-language="cs-CZ">
    <body>
      <trans-unit id="text_title">
        <source>Title</source>
        <target>Titulek</target>
      </trans-unit>
      <trans-unit id="appmenu_Close">
        <source>Close</source>
        <target>Zavřít</target>
      </trans-unit>
      <trans-unit id="appmenu_Quit">
        <source>Quit</source>
        <target>Ukončit</target>
      </trans-unit>
      <trans-unit id="appmenu_Close">
        <source>Close</source>
        <target>Zavřít</target>
      </trans-unit>
    </body>
  </file>
  <file original="MENU.txt" source-language="en-US" datatype="x-asw">
    <body>
      <trans-unit id="appmenu_Open">
        <source>Open</source>
        <target>Otevřít</target>
      </trans-unit>
    </body>
  </file>
</xliff>

It means, that the trans-unit "appmenu_Close" gets duplicated!!!!

Server configuration and status The component configured as monolingual with source file in en-US.

weblate list_versions

Weblate: 3.10.3
Django: 3.0.2
siphashc: 1.3
Whoosh: 2.7.4
translate-toolkit: 2.5.0
lxml: 4.4.2
Pillow: 7.0.0
bleach: 3.1.0
six: 1.14.0
python-dateutil: 2.8.1
social-auth-core: 3.2.0
social-auth-app-django: 3.1.0
django-crispy-forms: 1.8.1
oauthlib: 3.1.0
django-compressor: 2.4
djangorestframework: 3.11.0
django-appconf: 1.0.3
user-agents: 2.0
filelock: 3.0.12
setuptools: 45.1.0
jellyfish: 0.7.2
openpyxl: 3.0.3
celery: 4.4.0
kombu: 4.6.7
celery-batches: 0.2
translation-finder: 1.7
html2text: 2020.1.16
pycairo: 1.18.2
pygobject: 3.34.0
diff-match-patch: 20181111
requests: 2.22.0
django-redis: 4.11.0
hiredis: 1.0.1
sentry_sdk: 0.14.0
Cython: 0.29.14
misaka: 2.1.1
GitPython: 3.0.5
borgbackup: 1.1.10
Python: 3.7.3
Git: 2.20.1
psycopg2-binary: 2.8.4
phply: 1.2.5
chardet: 3.0.4
ruamel.yaml: 0.16.6
tesserocr: 2.5.0
boto3: 1.11.6
zeep: 3.4.0
aeidon: 1.6.0
git-svn: 2.20.1
Database backends: django.db.backends.postgresql
Cache backends: default:RedisCache, avatar:FileBasedCache
Email setup: django.core.mail.backends.smtp.EmailBackend: localhost
Celery: redis://localhost:6379, redis://localhost:6379, regular
Platform: Linux 4.19.0-6-amd64 (x86_64)

nijel commented 4 years ago

Duplicate of #2482

mclei-asw commented 4 years ago

Why was this just closed? In the #2482, there is even not mentioned that the similar problem is also with "file" tag. There is only "group" tag discussed.

nijel commented 4 years ago

Because it's technically same issue, it doesn't matter whether it is file or group tag.

WeblateOrg / weblate

XLIFF <file> tag not handled correctly for monolingual setup #3522