Open ajvondrak opened 9 months ago
Due to resque/resque-scheduler#767, resque-scheduler 4.9+ now wraps all enqueues within a Redis transaction. This means that Redis commands in
before_enqueue
hooks (inadvertently) no longer run synchronously under redis 4.x, which breaks gems like resque-lock-timeout (since it usesbefore_enqueue
hooks for loner lock checks).
Do you happen to have a fork with a mitigation? I just started looking at this issue, as we just ran into it, and looking for any prior art before I go spelunking.
No forks I'm afraid, but the mitigations I see:
Thanks, yep. I'm currently rolling back scheduler & redis to mitigate a live issue now, and will be looking into option 2. Option 3 I think would require 'injecting' Resque.enqueue with a given pipeline'd instance, which looks... not easy right now but maybe a longer term fix.
FWIW, I'll probably publish both that and our fork of resque-lock-timeout (which incorporates a few of the other fork fixes that have been around).
Nice. Hugops and will be curious to see to any other tweaks you have lying around. :)
Now that I'm spooling context back up on option 3 (credit to this comment), I think even that refactor still wouldn't resolve https://github.com/resque/resque-scheduler/issues/779 since enqueue hooks would still be executed in the middle of an open transaction so...option 2 seems like honestly the most feasible thing anyway.
Due to https://github.com/resque/resque-scheduler/pull/767, resque-scheduler 4.9+ now wraps all enqueues within a Redis transaction. This means that Redis commands in
before_enqueue
hooks (inadvertently) no longer run synchronously under redis 4.x, which breaks gems like resque-lock-timeout (since it usesbefore_enqueue
hooks for loner lock checks).Repro:
This seems to not be an issue on Redis 5.x, although there are other problems with resque-scheduler's transactions there, since some commands inadvertently "spill out" of the ongoing transaction (cf. https://github.com/resque/resque-scheduler/issues/787).
I think this is more a bug for resque-scheduler to fix (specifically https://github.com/resque/resque-scheduler/issues/779), but it affects this gem. So, heads up.
Further reading: