With the additional logging provided in the latest release, I do see multiple managers
pulling down the same tasks. This is due to a race condition in queue_get_next where
that code allows multiple managers to select the same records before updating them.
This PR hopefully solves this issue by using FOR UPDATE, which locks the selected rows
until a commit is made.
Unfortunately, this is a really hard thing to test (requires multiple workers and multiple running servers connected to the same db) and so no test is added.
Changelog description
Fixes race condition resulting in multiple managers computing the same tasks
codecov[bot]
commented
3 years ago
Description
With the additional logging provided in the latest release, I do see multiple managers pulling down the same tasks. This is due to a race condition in
queue_get_nextwhere that code allows multiple managers toselectthe same records before updating them.This PR hopefully solves this issue by using
FOR UPDATE, which locks the selected rows until a commit is made.See: Postgresql docs and sqlalchemy docs
Unfortunately, this is a really hard thing to test (requires multiple workers and multiple running servers connected to the same db) and so no test is added.
Changelog description
Fixes race condition resulting in multiple managers computing the same tasks
Status