TransactionManagementError

[rafagarciac]

Description

After upgrade my huge project from Python2.7/Django1.8 to Python3.7/Django2.2.1 I got the following error while I ran my tests. And I supposed that is blame of FactoryBoy (latest version).

To Reproduce

When I run my pytest command I just get a waterfall with all my tests getting the same error django.db.transaction.TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block. Also I'm using DjangoRestFramework (I don't think that make any relevance...)

Model / Factory code

class StorageFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = 'storage.Storage'

    company = factory.SubFactory(CompanyFactory)
    secure = False
    storage_type = 1 # Primary
    path = '/path/to/storage/'
    user = 'test_user'
    password = 'test_password'

Core Tests - Merge Factories

class BaseUser(APITestCase):
    # https://docs.djangoproject.com/en/2.2/topics/testing/tools/#django.test.TransactionTestCase.multi_db
    databases = '__all__'

    def setUp(self):
        # Companies
        self._client = factories.ClientFactory()
        self.company = factories.CompanyFactory()
        self.iprange = factories.IpRangeFactory()
        self.service = factories.ServiceFactory(company=(self.company,))
        self.company_disconnected = factories.CompanyFactory()  # In order to test Permissions.
        self.site = factories.SiteFactory(company=self.company)
        self.alert = factories.AlertFactory(company=self.company)
        self.invalidation = factories.InvalidationFactory(company=self.company)
        self.ratelimit = factories.RateLimitFactory(company=self.company)
        self.warmupcache = factories.WarmUpCaheFactory(company=self.company)
        # Storage
        self.storage = factories.StorageFactory(company=self.company)
        self.consumption = factories.ConsumptionFactory(storage=self.storage)
        . . .

And then I used this BaseUser in all my unit tests.

The issue

___________________________________________________________ StorageSuperAdminTests.test_storage_list_post_bad_request ____________________________________________________________
core/utils_tests.py:104: in setUp
    super(SuperAdminUser, self).setUp()
core/utils_tests.py:34: in setUp
    self.company = factories.CompanyFactory()
/usr/local/lib/python3.7/site-packages/factory/base.py:46: in __call__
    return cls.create(**kwargs)
/usr/local/lib/python3.7/site-packages/factory/base.py:564: in create
    return cls._generate(enums.CREATE_STRATEGY, kwargs)
/usr/local/lib/python3.7/site-packages/factory/django.py:141: in _generate
    return super(DjangoModelFactory, cls)._generate(strategy, params)
/usr/local/lib/python3.7/site-packages/factory/base.py:501: in _generate
    return step.build()
/usr/local/lib/python3.7/site-packages/factory/builder.py:279: in build
    kwargs=kwargs,
/usr/local/lib/python3.7/site-packages/factory/base.py:315: in instantiate
    return self.factory._create(model, *args, **kwargs)
/usr/local/lib/python3.7/site-packages/factory/django.py:185: in _create
    return manager.create(*args, **kwargs)
/usr/local/lib/python3.7/site-packages/django/db/models/manager.py:82: in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
/usr/local/lib/python3.7/site-packages/django/db/models/query.py:422: in create
    obj.save(force_insert=True, using=self.db)
companies/models.py:159: in save
    self._create_consolidated_table()
companies/models.py:168: in _create_consolidated_table
    response = cursor.execute("SHOW TABLES like '%s'" % tname)
/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py:67: in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py:76: in _execute_with_wrappers
    return executor(sql, params, many, context)
/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py:79: in _execute
    self.db.validate_no_broken_transaction()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <django.db.backends.mysql.base.DatabaseWrapper object at 0x7fdf591a0f98>

    def validate_no_broken_transaction(self):
        if self.needs_rollback:
            raise TransactionManagementError(
>               "An error occurred in the current transaction. You can't "
                "execute queries until the end of the 'atomic' block.")
E           django.db.transaction.TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.

/usr/local/lib/python3.7/site-packages/django/db/backends/base/base.py:438: TransactionManagementError

[federicobond]

Without more data it's almost impossible to reproduce your problem. Can you step into the code and find which database query produced an error in the current transaction, as described by the stack trace?

[rbarrois]

@rafagarciac, can you take a look at the logs or errors at the top of the stack or on the DB size?

The bug seems to indicate that some invalid data has been written to the database, but the DatabaseError has been ignored without rollbacking the transaction. The connection is in an inconsistent state, and Django will prevent anything from touching it.

I'm not aware of any factory_boy code that would "swallow" database exceptions, but that's still a possibility — could you look at the first errors in your stack trace?

[manjul-ridecell]

@rafagarciac Can you share the fix that you have used to solve the issue as we are also facing something similar for our django test cases.

[rafagarciac]

HI @manjul-ridecell ! 👋🏼 I'm not longer working with this codebase and consequently don't have access to reproduce it anymore. However, If I don't remember wrong, this was due to some miss configuration at the BaseUser class and/or the ClientFactory class level. I would recommend you to try to review all the low level of your configuration dependencies.

Hope this helps. Thanks in advance. 😄