We have a production setup with lots of workers running in parallel and a deadlock error occur from time to time, I think when two workers are trying the update the same task at the same moment. Here is the traceback of the error:
DeadlockDetected: deadlock detected
DETAIL: Process 3256555 waits for ShareLock on transaction 3220907091; blocked by process 3256561.
Process 3256561 waits for ShareLock on transaction 3220907090; blocked by process 3256555.
HINT: See server log for query details.
CONTEXT: while updating tuple (1239,37) in relation "huey_monitor_taskmodel"
File "django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
OperationalError: deadlock detected
DETAIL: Process 3256555 waits for ShareLock on transaction 3220907091; blocked by process 3256561.
Process 3256561 waits for ShareLock on transaction 3220907090; blocked by process 3256555.
HINT: See server log for query details.
CONTEXT: while updating tuple (1239,37) in relation "huey_monitor_taskmodel"
File "huey/consumer.py", line 94, in initialize
startup_hook()
File "huey_monitor/tasks.py", line 122, in startup_handler
update_task_instance(
File "huey_monitor/tasks.py", line 45, in update_task_instance
instance.save(update_fields=update_fields)
File "bx_django_utils/models/timetracking.py", line 59, in save
super().save(**kwargs)
File "django/db/models/base.py", line 814, in save
self.save_base(
File "django/db/models/base.py", line 877, in save_base
updated = self._save_table(
File "django/db/models/base.py", line 990, in _save_table
updated = self._do_update(
File "django/db/models/base.py", line 1054, in _do_update
return filtered._update(values) > 0
File "django/db/models/query.py", line 1231, in _update
return query.get_compiler(self.db).execute_sql(CURSOR)
File "django/db/models/sql/compiler.py", line 1982, in execute_sql
cursor = super().execute_sql(result_type)
File "django/db/models/sql/compiler.py", line 1560, in execute_sql
cursor.execute(sql, params)
File "django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(
File "django/db/backends/utils.py", line 80, in _execute_with_wrappers
return executor(sql, params, many, context)
File "django/db/backends/utils.py", line 84, in _execute
with self.db.wrap_database_errors:
File "django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
Any idea on how to prevent this?
We are using postgreSQL if it matters.
Hello,
We have a production setup with lots of workers running in parallel and a deadlock error occur from time to time, I think when two workers are trying the update the same task at the same moment. Here is the traceback of the error:
Any idea on how to prevent this? We are using postgreSQL if it matters.