rails / solid_queue

Database-backed Active Job backend
MIT License
1.84k stars 110 forks source link

Release any orphaned claimed executions as part of the process prune #244

Closed rosa closed 3 months ago

rosa commented 3 months ago

Orphaned executions shouldn't happen as part of regular operation, but some people have reported them (#159). The only way I can think of this happening could be that a worker gets deregistered while still working normally so that it happens to be claiming executions while the process record is deleted. This means that the callback to release claimed executions might not yet see the executions that are being claimed at the same time, and as such, it won't release any. Or, in other words, those executions would get claimed by a process that's getting deleted and that once they're committed to the DB, it no longer exists.

To recover from this scenario, as well as checking for processes with an expired heartbeat, the supervisor will also check for orphaned executions and release them.