search

serge-community / serge

Continuous localization platform
https://serge.io/
Other
237 stars 53 forks source link

Localized file generation failed #124

Closed AkshayRana92 closed 4 years ago

AkshayRana92 commented 4 years ago

Hi,

While using serge, I came across this strange issue. While running serge localize, I get an error

WARNING: Localized file generation failed; the file will not be saved
Reason:
not well-formed (invalid token) at line 1, column 15, byte 16 at /usr/local/lib/x86_64-linux-gnu/perl/5.26.1/XML/Parser.pm line 187.
at /usr/local/share/perl/5.26.1/XML/Twig.pm line 5245.

This happens only because of one field, that is "Meters and Users". So, the following xliff 1.2 file fails : 

<?xml version="1.0" encoding="UTF-8"?>
<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" version="1.2">
    <file source-language="en" datatype="plaintext" original="ng2.template">
        <body>
            <trans-unit id="some_unique_id" datatype="html">
                <source>Meters &amp; Users</source>
            </trans-unit>
        </body>
    </file>
</xliff>

Changing "Meters and Users" to anything else works.

iafan commented 4 years ago

This usually happens when the source file contains some parsing errors, but in your case it seems to fail only on a second pass (when generating the localized files, it re-reads the source file second time, and this is where it fails).

This might be simply due to caching. Try running serge localize --force and see if it starts to fail also during the first parsing pass (you will then get an error that says File parsing failed; the file will not be processed). If that's the case, then your source file is invalid; you can try online validators to see what the problem is, or upload the file somewhere and share it in this issue.

Another possibility is that you're using some file pre-processing logic in after_load_source_file_for_generating callback phase that modifies the source before the second parsing pass.

In any case, if you can't find a problem right away, please share a minimal project (the source file + your Serge config) so that I could reproduce the issue on my side.

AkshayRana92 commented 4 years ago

Thank you for getting back so quickly.

I tried serge localize --force, still same issue. And no, I'm not using after_load_source_file_for_generating.

Here you can find my config and translation file : https://github.com/AkshayRana92/serge-error Again, changing the text in the source file to something else works.

iafan commented 4 years ago

I tried to reproduce this on your test project (had to tweak a few paths to actually make it work), and file parsing/generation works as expected. I believe what happens here is that as soon as you add this string, a translation is being reused for it either from the Serge database or from ts files, and this translation is a malformed XML (most likely, it has & instead of &amp;). And this breaks the final file. I'd recommend looking at a corresponding ts file (e.g. ../ts/de/error.admin.messages.xlf.po) and see what translation for this string is there.

iafan commented 4 years ago

On a side note: is there a reason to use XLIFF files as source ones? XLIFF is typically used as an interchange file format; with Serge, it might be easier to skip XLIFF altogether and localize the actual final file format that is used in your application.

AkshayRana92 commented 4 years ago

Yes, '&' was the issue in the .po file. Thank you for your help.

The application I'm working on is on Angular, and Angular only provide 2 options, XLIFF or XMB.