IntegrityError: null value in column "gm2m_src_id"

[tkhyn]

Original report by Ata Ali Kilicli (Bitbucket: ataalik, GitHub: ataalik).

---

Hello, I wanted to use gm2m for creating manytomany generic relations because one of our models called entity is abstract and will be inherited by many classes. I tried a very basic setup where I create a GM2MField() on a third class and try to instance it with a PoliticalEntity inheriting from the abstract entity. That's when I get this error.

#!python
======================================================================
ERROR: setUpClass (api.tests.APITest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
psycopg2.IntegrityError: null value in column "gm2m_src_id" violates not-null constraint
DETAIL:  Failing row contains (null, 1, 12, 1).

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/django/test/testcases.py", line 1002, in setUpClass
    cls.setUpTestData()
  File "/src/api/tests.py", line 69, in setUpTestData
    geo=GEOSGeometry('{"type": "MultiPolygon","coordinates": [[[ [102.0, 2.0], [103.0, 2.0], [103.0, 3.0], [102.0, 3.0], [102.0, 2.0] ]],[[ [100.0, 0.0], [101.0, 0.0], [101.0, 1.0], [100.0, 1.0], [100.0, 0.0] ],[ [100.2, 0.2], [100.8, 0.2], [100.8, 0.8], [100.2, 0.8], [100.2, 0.2] ]]]}'))
  File "/usr/local/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 415, in create
    obj = self.model(**kwargs)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/base.py", line 490, in __init__
    _setattr(self, prop, kwargs[prop])
  File "/usr/local/lib/python3.6/site-packages/gm2m/descriptors.py", line 71, in __set__
    super(SourceGM2MDescriptor, self).__set__(instance, value)
  File "/usr/local/lib/python3.6/site-packages/gm2m/descriptors.py", line 24, in __set__
    manager.set(value)
  File "/usr/local/lib/python3.6/site-packages/gm2m/managers.py", line 140, in set
    self._do_add(db, to_add)
  File "/usr/local/lib/python3.6/site-packages/gm2m/managers.py", line 76, in _do_add
    self.through._default_manager.using(db).bulk_create(through_objs)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 466, in bulk_create
    ids = self._batched_insert(objs_without_pk, fields, batch_size)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 1142, in _batched_insert
    inserted_id = self._insert(item, fields=fields, using=self.db, return_id=True)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 1125, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1285, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: null value in column "gm2m_src_id" violates not-null constraint
DETAIL:  Failing row contains (null, 1, 12, 1).

----------------------------------------------------------------------
Ran 0 tests in 0.091s

I'm including a basic setup of mentioned models. You can find the whole code here

#!python

from json import loads

from django.core.exceptions import ValidationError
from django.contrib.gis.db import models
from django.contrib.postgres.fields import ArrayField
from simple_history.models import HistoricalRecords
from colorfield.fields import ColorField
from gm2m import GM2MField

# Create your models here.

class Entity(models.Model):
    name = models.TextField(max_length=100,
                            help_text="Canonical name, should not include any epithets, must be unique",
                            unique=True)
    url_id = models.SlugField(max_length=75,
                              help_text="Identifier used to lookup Entities in the URL, "
                                        "should be kept short and must be unique",
                              unique=True)
    references = ArrayField(
        models.TextField(max_length=150),
    )
    links = ArrayField(
        models.URLField(),
        blank=True,
    )
    description = models.TextField(help_text="Flavor text, brief history, etc.",
                                   blank=True)
    aliases = ArrayField(
        models.TextField(max_length=100),
        help_text="Alternative names this state may be known by",
        blank=True,
    )

    class Meta:
        abstract = True

class PoliticalEntity(Entity):
    """
    Cultural/governmental entity. Serves as foreign key for most Territories
    """
    color = ColorField(help_text="Color to display on map",
                       unique=True,
                       null=True,
                       blank=True)
    history = HistoricalRecords()

    CONTROL_TYPE_CHOICES = (
        ("CC", "Complete Control"),
        ("DT", "Disputed Territory"),
        # TODO: Add more types later
    )
    control_type = models.TextField(
        max_length=2,
        choices=CONTROL_TYPE_CHOICES,
        default="CC",
        blank=True,
    )

    #History fields

    # Foreign key to auth.User which will be updated every time the model is changed,
    # and is this stored in history as the user to update a specific revision
    # Consider other metadata (DateTime) for the revision (may be handled by django-simple-history)
    # TODO: implement this

    def __str__(self):
        return self.name

class Territory(models.Model):
    """
    Defines the borders and controlled territories associated with an Entity.
    """
    class Meta:
        verbose_name_plural = "territories"

    start_date = models.DateField(help_text="When this border takes effect")
    end_date = models.DateField(help_text="When this border ceases to exist")
    geo = models.GeometryField()
    # entity = models.ForeignKey(Entity,
    #                            related_name="territories",
    #                            on_delete=models.CASCADE,
    #                            null=True,
    #                            blank=True
    #                            )
    entity = GM2MField()
    references = ArrayField(
        models.TextField(max_length=150),
    )
    history = HistoricalRecords()

    def clean(self, *args, **kwargs):
        if self.start_date > self.end_date:
            raise ValidationError("Start date cannot be later than end date")
        if loads(self.geo.json)["type"] != "Polygon" and loads(self.geo.json)["type"] != "MultiPolygon":
            raise ValidationError("Only Polygon and MultiPolygon objects are acceptable geometry types")
        super(Territory, self).clean(*args, **kwargs)

    def save(self, *args, **kwargs):
        self.full_clean()
        super(Territory, self).save(*args, **kwargs)

    def __str__(self):
        return "%s: %s - %s" % (self.entity.name,
                                self.start_date.strftime("%m/%d/%Y"),
self.end_date.strftime("%m/%d/%Y"))

[tkhyn]

Original comment by Thomas Khyn (Bitbucket: tkhyn, GitHub: tkhyn).

---

Hi, GM2MField is like django's built-in ManyToManyField. You cannot initialise it when you create the instance. And if you could, you would need to pass a list, not a single instance (https://github.com/chronoscio/backend/blob/wh-Entity/project/api/tests.py#L69)

The way to provide initial values to a GM2MField is to create the model instance first, then populate the field.

See https://stackoverflow.com/a/6996358/1445719 or https://docs.djangoproject.com/en/2.1/topics/db/examples/many_to_many/

[tkhyn]

Original comment by Ata Ali Kilicli (Bitbucket: ataalik, GitHub: ataalik).

---

I see. Initially the question here was different but I edited it to ask what I was really meaning to ask. You answered both of the questions so thank you.

[tkhyn]

Original comment by Thomas Khyn (Bitbucket: tkhyn, GitHub: tkhyn).

---

@ataalik no worries, thanks for reporting, hopefully you'll make it work now!

Closing as invalid.