search

collectiveidea / delayed_job_active_record

ActiveRecord backend integration for DelayedJob 3.0+
MIT License
346 stars 336 forks source link

Clear locks using `reserve_sql_strategy` #211

Open njakobsen opened 1 year ago

njakobsen commented 1 year ago

When shutting down and releasing jobs, we should still respect the ActiveRecord.configuration.reserve_sql_strategy and clear locks using explicit job ids if the strategy is :default_sql. This can help avoid transaction isolation conflicts when shutting down several job runners.

albus522 commented 1 year ago

If locked_by is hitting a conflict you have misconfigured something and will have major issues with duplicate job runs.

njakobsen commented 1 year ago

It's not so much that locked_by is causing a conflict but that the non-primary key query to obtain a lock on a row uses a range lock, which must wait behind transactions that have created a job but have not committed yet. If those transactions are long running and plentiful, it can cause the delayed job shutdown process to take a long time. Therefore this change is intended to apply the same mechanism to unlocking as is used for locking.