search

maschmann / TranslationLoaderBundle

Symfony2 bundle with database translation loader
23 stars 9 forks source link

[asm:translations:import] Duplicate entry for key 'PRIMARY' #25

Closed christinloehner closed 9 years ago

christinloehner commented 9 years ago

Hello,

I'm using Symfony 2.6.6 and the TranslationLoaderBundler (dev-master) for putting my translations to the database in order to easily be able to provide some translation tool to my community.

Everything works as expected with one exception:

I have an own command for updating the translations, this is the code:

protected function configure()
{
    $this
        ->setName('syw:update:translations')
        ->setDescription('Updates Translations from templates into database');
}

/**
 * {@inheritDoc}
 */
protected function execute(InputInterface $input, OutputInterface $output)
{
    $db = $this->getContainer()->get('doctrine.dbal.default_connection');
    $bundle_array = $this->getContainer()->getParameter('kernel.bundles');
    $bundles = array();
    foreach ($bundle_array as $key => $val) {
        $bundles[] = $key;
    }
    $rows    = $db->fetchAll('SELECT l.locale FROM languages l ORDER BY l.locale ASC');
    $locales = array();
    foreach ($rows as $row) {
        $locales[] = $row['locale'];
    }
    foreach ($locales as $locale) {
        @passthru('php app/console translation:update --prefix "" --force ' . $locale . ' 2>/dev/null 3>&2 4>&2');
        foreach ($bundles as $bundle) {
            @passthru('php app/console translation:update --prefix "" --force ' . $locale . ' ' . $bundle . ' 2>/dev/null 3>&2 4>&2');
        }
    }
    @passthru('php app/console asm:translations:import -v');

    $output->writeln('<comment>finished!</comment>');
}

In general it simply parses the templates for the translations and then it puts all into the database.

This works like a charm when I run it on an empty translation table. But when I RE-run this after I've changed some things in my templates or added new translations to my templates, it always throws the following exception:

[Doctrine\DBAL\DBALException]                                                                                                                                                                                                                                                                                               
An exception occurred while executing 'INSERT INTO translation (translation, date_created, date_updated, trans_key, trans_locale, message_domain) VALUES (?, ?, ?, ?, ?, ?)' with params ["<a href=\"https:\/\/www.first-colo.net\/\" target=\"_blank\">First Colo<\/a> operates as an IT infrastructure supplier high ava  
ilability data processing center with the core specialties in rack housing, server hosting, managed services, and DDoS-protection, at German and European locations.", "2015-04-15 13:11:26", "2015-04-15 13:11:26", "<a href=\"https:\/\/www.first-colo.net\/\" target=\"_blank\">First Colo<\/a> operates as an IT infra  
structure supplier high availability data processing center with the core specialties in rack housing, server hosting, managed services, and DDoS-protection, at German and European locations.", "vi", "syw_front_main_main_sponsor"]:                                                                                     
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '<a href="https://www.first-colo.net/" target="_blank">First Colo' for key 'PRIMARY'

This seems to be caused by html-code with quotes and maybe resulting slashes or something. So....

in /vendor/asm/translation-loader-bundle/Command/ImportTranslationsCommand.php on lines 182:204 it seems to pass the "if (!$translation) {" because of additional slashes or whatever.

I would urgently need a fix for this or at least an idea how to fix it by myself in an overwriting command.

maschmann commented 9 years ago

Hello Alex, I'll have a look into the issue later tonight.

I'd strongly suggest not to use @passthru, but to do it nicely like e.g. http://symfony.com/doc/current/components/console/introduction.html#calling-an-existing-command Also having HTML in your translations might not be the best way ;-)

maschmann commented 9 years ago

Hello @alexloehner ,

is there any way you can provide some testdata for me to import? I couldn't reproduce your error yet...

christinloehner commented 9 years ago

Hello @maschmann ,

well, you could clone my https://github.com/alexloehner/linuxcounter.new and try to get it to work. Don't forget to load all the dependencies with composer update. :P When it works, you would have to run my command syw:update:translations twice as explained above.

maschmann commented 9 years ago

Hello @alexloehner ,

I've cloned your project and ran the "syw:update:translations" command several times and still could not reproduce your error. Maybe it is related to your server setup or previous database contents?

What you could try is: backup your translation table, do a fresh import - and the do a fresh import again to check if the error persists. I could imagine two possibilities: Either the translation key that causes the error was not in the repository or you've got old entries from a previous version which might behave inconsistently.

Most of the time you wouldn't like "asm/translation-loader-bundle": "dev-master", in your database, but stable versions. Keep in mind that "dev-master" ist at best "bleeding edge" and far from stable!

(master)wwwdev@devbox:~/htdocs/linuxcounter.new$ app/console syw:update:translations
nice: kann Priorität nicht setzen: Keine Berechtigung
--------------------------------------------------------------------------------
Translation file importer
--------------------------------------------------------------------------------
importing all available translation files ...
searching AsmTranslationLoaderBundle translations
--------------------------------------------------------------------------------
loading messages.de_DE.yml with locale de_DE and domain messages
loading messages.en_US.yml with locale en_US and domain messages

searching EasyAdminBundle translations
--------------------------------------------------------------------------------

searching MopaBootstrapBundle translations
--------------------------------------------------------------------------------
loading pagination.it.yml with locale it and domain pagination
loading pagination.uk.yml with locale uk and domain pagination
loading pagination.ar.yml with locale ar and domain pagination
loading pagination.pt.yml with locale pt and domain pagination
loading pagination.sk.yml with locale sk and domain pagination
loading pagination.de.yml with locale de and domain pagination
loading pagination.sl.yml with locale sl and domain pagination
loading pagination.fr.yml with locale fr and domain pagination
loading pagination.es.yml with locale es and domain pagination
loading pagination.ja.yml with locale ja and domain pagination
loading pagination.en.yml with locale en and domain pagination
loading pagination.sv.yml with locale sv and domain pagination
loading pagination.ru.yml with locale ru and domain pagination
loading pagination.lt.yml with locale lt and domain pagination
loading pagination.nl.yml with locale nl and domain pagination
loading pagination.pl.yml with locale pl and domain pagination

searching RedmonsterAnnouncementBundle translations
--------------------------------------------------------------------------------
loading messages.fr.xlf with locale fr and domain messages

inserting all translations
--------------------------------------------------------------------------------
de_DE:  ... messages.de_DE
en_US:  ... messages.en_US
it:  ... pagination.it
uk:  ... pagination.uk
ar:  ... pagination.ar
pt:  ... pagination.pt
sk:  ... pagination.sk
de:  ... pagination.de
sl:  ... pagination.sl
fr:  ... pagination.fr ... messages.fr
es:  ... pagination.es
ja:  ... pagination.ja
en:  ... pagination.en
sv:  ... pagination.sv
ru:  ... pagination.ru
lt:  ... pagination.lt
nl:  ... pagination.nl
pl:  ... pagination.pl
--------------------------------------------------------------------------------
finished!
finished!
Sie haben neue Post in /var/mail/wwwdev.
(master)wwwdev@devbox:~/htdocs/linuxcounter.new$ app/console syw:update:translations
nice: kann Priorität nicht setzen: Keine Berechtigung
--------------------------------------------------------------------------------
Translation file importer
--------------------------------------------------------------------------------
importing all available translation files ...
searching AsmTranslationLoaderBundle translations
--------------------------------------------------------------------------------
loading messages.de_DE.yml with locale de_DE and domain messages
loading messages.en_US.yml with locale en_US and domain messages

searching EasyAdminBundle translations
--------------------------------------------------------------------------------

searching MopaBootstrapBundle translations
--------------------------------------------------------------------------------
loading pagination.it.yml with locale it and domain pagination
loading pagination.uk.yml with locale uk and domain pagination
loading pagination.ar.yml with locale ar and domain pagination
loading pagination.pt.yml with locale pt and domain pagination
loading pagination.sk.yml with locale sk and domain pagination
loading pagination.de.yml with locale de and domain pagination
loading pagination.sl.yml with locale sl and domain pagination
loading pagination.fr.yml with locale fr and domain pagination
loading pagination.es.yml with locale es and domain pagination
loading pagination.ja.yml with locale ja and domain pagination
loading pagination.en.yml with locale en and domain pagination
loading pagination.sv.yml with locale sv and domain pagination
loading pagination.ru.yml with locale ru and domain pagination
loading pagination.lt.yml with locale lt and domain pagination
loading pagination.nl.yml with locale nl and domain pagination
loading pagination.pl.yml with locale pl and domain pagination

searching RedmonsterAnnouncementBundle translations
--------------------------------------------------------------------------------
loading messages.fr.xlf with locale fr and domain messages

inserting all translations
--------------------------------------------------------------------------------
de_DE:  ... messages.de_DE
en_US:  ... messages.en_US
it:  ... pagination.it
uk:  ... pagination.uk
ar:  ... pagination.ar
pt:  ... pagination.pt
sk:  ... pagination.sk
de:  ... pagination.de
sl:  ... pagination.sl
fr:  ... pagination.fr ... messages.fr
es:  ... pagination.es
ja:  ... pagination.ja
en:  ... pagination.en
sv:  ... pagination.sv
ru:  ... pagination.ru
lt:  ... pagination.lt
nl:  ... pagination.nl
pl:  ... pagination.pl
--------------------------------------------------------------------------------
finished!
finished!
christinloehner commented 9 years ago

Hello @maschmann

That is strange... especially because it seems you have not all translations imported. For me, importing all translations took about 24 hours... :P

well, I'll try to downgrade my stuff a bit and then clearing the database up.

Will respond later then. Thanks for now.

christinloehner commented 9 years ago

by the way... in the UpdateTranslationsCommand.php have you UN-commented the lines 66-71? Otherwise not all translations, especially my own template translations will not get imported.

maschmann commented 9 years ago

Hello Alex, even with un-commenting the lines in the UpdateTranslationsCommand, I had no increase in translations and no error, too. I cloned your project, added an nginx config for a local host, fired composer update, created database and schema via doctrine - and then started your command.

After looking into your code: Why are you copying e.g. the command, or to be more precise, just the code, into your command? I'd strongly suggest to use the symfony methodology to call other commands from within your commands: http://symfony.com/doc/current/components/console/introduction.html#calling-an-existing-command You're running into loads of problems if internal workings of the referenced code change and you didn't use the provided APIs or extension points to hook in. I can only guess what's going wrong during the import, since I can't fathom how the TranslationLoaderBundle code will behave outside the tested boundaries.

maschmann commented 9 years ago

@alexloehner since this behavior could not be reproduced and is most likely caused by custom changes in the code, I'm closing this issue.