django.db.utils.InterfaceError: (0, '') in MySQL

[artemistomaras]

Python Version: 3.5 Architect Version: 0.5.5 (same with 0.5.6) Django Version: 1.10.5 Mysql Version: 5.7.16 ( also tested with 5.7.18) mysqlclient Version: 1.3.9 (same with 1.3.10) Operating System: Windows 7 64bit (also tested on Ubuntu 16.04 LTS docker image)

My model:

@architect.install('partition', type='range', subtype='date', constraint='year', column='ontime')
class Measurement(models.Model):
    id = models.BigAutoField(primary_key=True)
    service_id = models.IntegerField(db_index=True)
    value = models.FloatField(blank=False, null=True)
    ontime = models.DateTimeField(blank=False, null=False)

I have made sure that architect partition command is run beforehand.

I am getting the following error when I try to create an Measurement objects in any of my views:

[2017-05-12 12:14:12,265: ERROR/MainProcess] Task api.tasks.create_measurements[c2f94f7c-0ce2-48f0-9b60-f22a2dc21cb8] raised unexpected: InterfaceError(0, '')
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/celery/app/trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/celery/app/trace.py", line 438, in __protected_call__
    return self.run(*args, **kwargs)
  File "/opt/path/to/my/service/tasks.py", line 43, in create_measurements
    Measurement.objects.create(service_id=measurement['service_id'], value=measurement['value'], ontime=timestamp)
  File "/usr/local/lib/python3.5/dist-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/django/db/models/query.py", line 399, in create
    obj.save(force_insert=True, using=self.db)
  File "/usr/local/lib/python3.5/dist-packages/architect/orms/django/features.py", line 103, in wrapper
    if not partition.exists():
  File "/usr/local/lib/python3.5/dist-packages/architect/databases/mysql/partition.py", line 36, in exists
    """.format(parent_table=self.table, name=self._get_name()))
  File "/usr/local/lib/python3.5/dist-packages/architect/orms/django/features.py", line 47, in select_one
    self.execute(sql)
  File "/usr/local/lib/python3.5/dist-packages/architect/orms/django/features.py", line 44, in execute
    return self.connection.execute(sql)
  File "/usr/local/lib/python3.5/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.5/dist-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python3.5/dist-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.5/dist-packages/django/db/backends/utils.py", line 62, in execute
    return self.cursor.execute(sql)
  File "/usr/local/lib/python3.5/dist-packages/django/db/backends/mysql/base.py", line 110, in execute
    return self.cursor.execute(query, args)
  File "/usr/local/lib/python3.5/dist-packages/MySQLdb/cursors.py", line 250, in execute
    self.errorhandler(self, exc, value)
  File "/usr/local/lib/python3.5/dist-packages/MySQLdb/connections.py", line 42, in defaulterrorhandler
    raise errorvalue
  File "/usr/local/lib/python3.5/dist-packages/MySQLdb/cursors.py", line 247, in execute
    res = self._query(query)
  File "/usr/local/lib/python3.5/dist-packages/MySQLdb/cursors.py", line 411, in _query
    rowcount = self._do_query(q)
  File "/usr/local/lib/python3.5/dist-packages/MySQLdb/cursors.py", line 374, in _do_query
    db.query(q)
  File "/usr/local/lib/python3.5/dist-packages/MySQLdb/connections.py", line 270, in query
    _mysql.connection.query(self, query)
django.db.utils.InterfaceError: (0, '')

The problem exists when invoking object creation in any of the two ways:

1. Measurement.objects.create(....)
2. m = Measurement(...) and then m.save()

and the creation is performed in any of my views, or a celery task. When I create the object via the shell (by running python manage.py shell ...) the object is created with no problem. More specifically:

Windows 7 (mysql 5.7.16)

Object creation in a view : fails Object creation in a celery task: succeeds Object creation via the shell: succeeds

Ubuntu 16.04 LTS docker image ( mysql 5.7.18)

Object creation in a view : fails Object creation in a celery task: fails Object creation via the shell: succeeds

[maxtepkeev]

That's very interesting, thanks for such a detailed report, I'll try to have a look at it in the upcoming days and see what's causing this.

[artemistomaras]

Thank you.

Anyway, if you require any more information, more details, I am happy to provide them. I'm sure though that you have been able to reproduce that weird behavior by now. If not, let me know.

[maxtepkeev]

I failed to reproduce the issue.

I created a django project from scratch with one app, added your model to models.py, partitioned it, then created a view and added a Measurement.objects.create(....) to it, everything worked perfectly fine. Can you try to do the same and if that works, then there should be some other library/code that is causing this behaviour.

[maxtepkeev]

Ah, no wait, I reproduced it, I have to call the view several times and then I was able to reproduce it.

[maxtepkeev]

Apologies for being silent, I'm busy with a lot of other stuff at the moment, the quick fix you can make for now is to set the CONN_MAX_AGE to None in your settings.py which will make Django to use persistent connections (docs) and the error will go away.

In the meantime I have to come up with a proper connection handling for Django to fix this error ones and for all.

[artemistomaras]

Thank you for your time and your quick workaround.

[Menci]

Well, persistent connections will involve another issue - MySQL by default has a 8 hours timeout for every connection. So if our persistent connections keep for over 8 hours, the next query will raise a MySQL server has gone away error.

[akellehe]

I'm seeing a recurrence of this issue.

[chenluyong]

I'm seeing a recurrence of this issue.

Django==3.0.7