SQL error when running update_stores on an XLIFF project

[dwaynebailey]

I'm assuming that the central issue may be that this related to XLIFF support on Pootle 2.7.0. But I'm not sure if the issue is just the XLIFF file.

2015-07-21 16:12:29,073 ERROR Failed to run update_stores over /cy/ios/firefox-ios.xliffTraceback (most recent call last):
  File "/var/www/sites/mozilla/src/pootle/apps/pootle_app/management/commands/__init__.py", line 65, in do_translation_project
    self.handle_store(store, **options)
  File "/var/www/sites/mozilla/src/pootle/apps/pootle_app/management/commands/update_stores.py", line 50, in handle_store
    store.update(overwrite=overwrite, only_newer=not force)
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/transaction.py", line 394, in inner
    return func(*args, **kwargs)
  File "/var/www/sites/mozilla/src/pootle/apps/pootle_store/models.py", line 1746, in update
    self.save()
  File "/var/www/sites/mozilla/src/pootle/apps/pootle_store/models.py", line 1421, in save
    super(Store, self).save(*args, **kwargs)
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 589, in save
    force_update=force_update, update_fields=update_fields)
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 617, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 679, in _save_table
    forced_update)
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 723, in _do_update
    return filtered._update(values) > 0
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/models/query.py", line 600, in _update
    return query.get_compiler(self.db).execute_sql(CURSOR)
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 1005, in execute_sql
    cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 787, in execute_sql
    cursor.execute(sql, params)
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 59, in execute
    self.db.validate_no_broken_transaction()
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/backends/__init__.py", line 386, in validate_no_broken_transaction
    "An error occurred in the current transaction. You can't "
TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.

[dwaynebailey]

Actually it's likely not the files itself. The traceback is preceeded by the following:

...
[2015-07-21T16:12:26]   system  UO      cy      18069500        /cy/ios/fi
refox-ios.xliff  Nid ydych wedi eich mewngofnodi i'ch Cyfrif Firefox.
[2015-07-21T16:12:26]   system  UO      cy      18069509        /cy/ios/fi
refox-ios.xliff  Nid oes gennych unrhyw ddyfais arall wedi ei gysylltu i Fir
efox Sync.

UO means An existing unit made obsolete

[jleclanche]

Anything in the sql log?

[dwaynebailey]

@jleclanche error logs are empty

[dwaynebailey]

After removing @transaction.atomic from def update as suggested by @ta2-1 I get this traceback:

[2015-07-22T08:29:34]   system  UO      cy      18069509        /cy/ios/firefox-ios.xliff    Nid oes gennych unrhyw ddyfais arall wedi ei gysylltu i Firefox Sync.
[2015-07-22T08:29:36]   [update] added 8, obsolete 19 units in /cy/ios/firefox-ios.xliff [revision: 24382]
2015-07-22 08:29:36,852 ERROR Failed to run update_stores over /cy/ios/firefox-ios.xliff
Traceback (most recent call last):
  File "/var/www/sites/mozilla/src/pootle/apps/pootle_app/management/commands/__init__.py", line 65, in do_translation_project
    self.handle_store(store, **options)
  File "/var/www/sites/mozilla/src/pootle/apps/pootle_app/management/commands/update_stores.py", line 50, in handle_store
    store.update(overwrite=overwrite, only_newer=not force)
  File "/var/www/sites/mozilla/src/pootle/apps/pootle_store/models.py", line 1655, in update
    newunit = self.addunit(unit, unit.index, user=system)
  File "/var/www/sites/mozilla/src/pootle/apps/pootle_store/models.py", line 1930, in addunit
    newunit.update(unit, user=user)
  File "/var/www/sites/mozilla/src/pootle/apps/pootle_store/models.py", line 742, in update
    self.add_suggestion(suggestion.target, user=user, touch=False)
  File "/var/www/sites/mozilla/src/pootle/apps/pootle_store/models.py", line 1113, in add_suggestion
    suggestion.save()
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 589, in save
    force_update=force_update, update_fields=update_fields)
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 617, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 698, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 731, in _do_insert
    using=using, raw=raw)
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/models/manager.py", line 92, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/models/query.py", line 921, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 921, in execute_sql
    cursor.execute(sql, params)
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 134, in execute
    six.reraise(utils.IntegrityError, utils.IntegrityError(*tuple(e.args)), sys.exc_info()[2])
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 129, in execute
    return self.cursor.execute(query, args)
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
    self.errorhandler(self, exc, value)
  File "/var/www/sites/mozilla/env/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
IntegrityError: (1048, "Column 'unit_id' cannot be null")

[jleclanche]

This is fixed by #3915.

[julen]

Mind elaborating what was the root cause? the commit message doesn't clarify much :confused:

[jleclanche]

@julen FWIU, alt-trans support was causing us to sometimes duplicate existing translations/suggestions which, down the line, caused us to create a suggestion on a unit that hadn't been saved yet.

I couldn't reproduce the issue myself nor with @dwaynebailey on our staging server, but he could reliably reproduce it on MLO, so I don't think I fully understand what's happening. Regardless, we decided that it's causing more trouble than it's worth and that we don't need the feature anyway, so we dropped it.

[julen]

:+1: thanks for clarifying this!