psycopg2.IntegrityError: duplicate key value violates unique constraint pkey

[rubnov]

Description

Following a dependency update of my Django project (django, factory_boy, pytest-factoryboy), a lot of my tests are failing with a duplicate key error.

The error occurs when using a user fixture based on a UserFactory. For some reason factory_boy is calling create() twice on the object with the same primary_key, and I cannot understand why or what is causing this.

To Reproduce

E   psycopg2.IntegrityError: duplicate key value violates unique constraint "users_user_pkey"
E   DETAIL:  Key (id)=(1) already exists.

Model / Factory code

@python_2_unicode_compatible
class User(AbstractBaseUser, PermissionsMixin):

    DEFAULT_LANGUAGE = 'en'
    SOCIAL_FIELDS = ['first_name', 'last_name', 'gender']

    email = models.EmailField(
        _('email'), max_length=255, unique=True, db_index=True,
        help_text='User email id (Unique identification of user)')

    username = models.CharField(
        _('name'), max_length=255, blank=True, null=True,
        help_text='User name')

    first_name = models.CharField(
        _('first name'), max_length=255, blank=True, null=True, db_index=True,
        help_text='Initial name of user')
    last_name = models.CharField(
        _('last name'), max_length=255, blank=True, null=True, db_index=True,
        help_text='Surname of user')

    gender = models.CharField(
        _('gender'), max_length=255, blank=True, null=True)

    token = models.CharField(
        _('api key'), max_length=255, blank=True, null=True, db_index=True, unique=True,
        help_text='API token key. Unique for each user.'
        'Without token user cant make API calls')

    objects = UserManager()

    def save(self, force_insert=False, force_update=False, using=None, update_fields=None):
        if not self.token:
            self.token = self.generate_token()
            if update_fields:
                update_fields.append('token')

        if not self.last_login:
            self.last_login = timezone.now()
            if update_fields:
                update_fields.append('last_login')

        return super(User, self).save(force_insert=force_insert, force_update=force_update, using=using,
                                      update_fields=update_fields)

    def generate_token(self):
        new_uuid = uuid.uuid4()
        return hmac.new(new_uuid.bytes, digestmod=sha256).hexdigest()

@python_2_unicode_compatible
class UserVehicle(models.Model):

    """User's vehicle information."""

    user = models.ForeignKey(
        'users.User',
        on_delete=models.CASCADE,
        related_name='user_vehicles')

    vehicle = models.ForeignKey(
        'core.Vehicle',
        null=True,
        on_delete=models.SET_NULL,
        related_name='user_vehicles',
        help_text='Search parameter sequence: Make, Year, Model, Fuel type, Transmission, Displacement',
        blank=True
    )
    ...

class UserFactory(factory.django.DjangoModelFactory):
    """A user factory."""

    class Meta:
        model = User
        django_get_or_create = ('email',)

    first_name = u'Chloé'
    last_name = u'Curaçao'
    email = factory.Sequence(lambda n: 'user{0}@example.com'.format(n))
    username = 'chloe.curacao'
    gender = 'female'
    bio = 'bio'
    password = factory.PostGenerationMethodCall('set_password', DEFAULT_PASSWORD)
    save = factory.PostGenerationMethodCall('save')
    last_login = timezone.datetime(
        year=2015, month=8, day=17, hour=8, minute=30, tzinfo=timezone.get_default_timezone())
    date_active = last_login
    confirmed = True
    is_active = True
    is_staff = True

    country = 'NL'
    settings = factory.Sequence(lambda n: dict(**SETTINGS_DEFAULT))

register(UserFactory)

class UserVehicleFactory(factory.DjangoModelFactory):

    class Meta:
        model = planner_models.UserVehicle

    user = factory.SubFactory(UserFactory)
    vehicle = factory.SubFactory(VehicleFactory)
    vclass = factory.SubFactory(VehicleClassFactory)
    hidden = False

    country = 'NL'
    state = None

register(UserVehicleFactory, 'user_vehicle')
register(UserVehicleFactory, 'other_user_vehicle')

The issue

Add a short description along with your code One example failing test is test_auth_broken_check_hmac_expected_user_id below:

@pytest.mark.urls(__name__)
@pytest.mark.django_db
class TestSecuredViewHmacEnabled(object):
    @pytest.fixture(autouse=True)
    def setup(self, user, user_vehicle, other_user_vehicle):
        self.user = user
        self.user_vehicle = user_vehicle
        self.other_user_vehicle = other_user_vehicle

    def test_auth_success(self, client, settings):
        uri = reverse('user_vehicles', kwargs={'id': self.user.id})
        full_url = ''.join((settings.SITE_DOMAIN, uri))

        hmac = SecuredViewHmacEnabled.generate_hmac_url(user_token=self.user.token, full_request_path=full_url)

        response = client.get(hmac, content_type='application/json')

        assert response.status_code == http.client.OK
        assert response.content.decode() == SecuredViewHmacEnabled.success_message

    def test_auth_broken_check_hmac_expected_user_id(self, client, settings):
        if self.other_user_vehicle.id == self.user.id:
            user_vehicle = self.user_vehicle
        else:
            user_vehicle = self.other_user_vehicle

        uri = reverse('vehicle_users', kwargs={'id': user_vehicle.id})
        full_url = ''.join((settings.SITE_DOMAIN, uri))

        hmac = SecuredViewHmacEnabled.generate_hmac_url(user_token=self.user.token, full_request_path=full_url)

        response = client.get(hmac, content_type='application/json')

        assert response.status_code == http.client.NOT_FOUND

Stack Trace:

――――――――――――――――――――――――――――――――― ERROR at setup of TestSecuredViewHmacEnabled.test_auth_broken_check_hmac_expected_user_id ――――――――――――――――――――――――――――――――――
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/django/db/backends/utils.py:84: in _execute
    return self.cursor.execute(sql, params)
E   psycopg2.IntegrityError: duplicate key value violates unique constraint "users_user_pkey"
E   DETAIL:  Key (id)=(1) already exists.

The above exception was the direct cause of the following exception:
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/pytest_factoryboy/fixture.py:256: in model_fixture
    factoryboy_request.evaluate(request)
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/pytest_factoryboy/plugin.py:83: in evaluate
    self.execute(request, function, deferred)
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/pytest_factoryboy/plugin.py:65: in execute
    self.results[model][attr] = function(request)
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/pytest_factoryboy/fixture.py:296: in deferred
    declaration.call(instance, step, context)
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/factory/declarations.py:743: in call
    return method(*args, **kwargs)
gopublic/apps/users/models.py:1059: in save
    super(User, self).save(*args, **kwargs)
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/django/contrib/auth/base_user.py:66: in save
    super().save(*args, **kwargs)
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/django/db/models/base.py:741: in save
    force_update=force_update, update_fields=update_fields)
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/django/db/models/base.py:779: in save_base
    force_update, using, update_fields,
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/django/db/models/base.py:870: in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/django/db/models/base.py:908: in _do_insert
    using=using, raw=raw)
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/django/db/models/manager.py:82: in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/django/db/models/query.py:1186: in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/django/db/models/sql/compiler.py:1335: in execute_sql
    cursor.execute(sql, params)
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/django/db/backends/utils.py:67: in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/django/db/backends/utils.py:76: in _execute_with_wrappers
    return executor(sql, params, many, context)
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/django/db/backends/utils.py:84: in _execute
    return self.cursor.execute(sql, params)
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/django/db/utils.py:89: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
../../../.pyenv/versions/3.6.8/envs/venv368/lib/python3.6/site-packages/django/db/backends/utils.py:84: in _execute
    return self.cursor.execute(sql, params)
E   django.db.utils.IntegrityError: duplicate key value violates unique constraint "users_user_pkey"
E   DETAIL:  Key (id)=(1) already exists.
Destroying test database for alias 'default'...

Notes

The code above worked before updating the dependencies, which puzzles me. To be honest this was a major update from Django 1.11 to 2.2.6, along with factory_boy from 2.5.2 to 2.12.0 and pytest-factoryboy from 1.1.0 to 2.0.3.

[matejkloska]

Same issue here. Any idea how to fix it?

[rbarrois]

@matejkloska Can you post a minimal code example reproducing the issue?

@rubnov Can you try it in a minimal script without using pytest-factory_boy? The way fixtures are injected by pytest makes reasoning about the code path quite harder...

[HerlanAssis]

I have this same problem, any update?

[HerlanAssis]

hi, some news. After many time trying fix, i notice my mistake is because i changed method save in model for calling super save twice in one only call for save

[shacker]

In my case, I have a call to super().save(*args, **kwargs) within the User.save() method. It is intentional, not a mistake, and I really do need it to establish M2M relations. But now I've discovered that it's broken my UserFactory, and am not sure how to proceed.

[Braintelligence]

Same issue here... this still isn't fixed.

[rbarrois]

@rubnov sorry for the very late response, but the issue stemmed from this line:

    save = factory.PostGenerationMethodCall('save')

It did cause a second .save(), which might cause issues with your overridden def save(...); moreover, that extra method call isn't required, as factory.django.DjangoModelFactory will automatically call .save() after all PostGenerationMethodCalls have run.

[rbarrois]

For other participants to this discussion, this error came from a specific implementation of the factory; which is unlikely to be the cause of the error you would see. Could you please open new issues with your specific code and stacktrace? This will allow us to look into each special case specifically.

Thanks!

[francoisfreitag]

Closed for inactivity.