This PR is a PoC for an optimised way to poll queues. Tests were deliberately omitted at this stage.
The way it works is that the key t:queued_cache_token:<queue root> (I welcome better naming here) is updated with a random token whenever a task is added to that queue.
The workers will monitor the tokens related to the queues they're interested in and do the expensive queue poll only when any of them change (or if an hour elapsed since the last poll, as a safeguard).
This PR is a PoC for an optimised way to poll queues. Tests were deliberately omitted at this stage.
The way it works is that the key
t:queued_cache_token:<queue root>(I welcome better naming here) is updated with a random token whenever a task is added to that queue.The workers will monitor the tokens related to the queues they're interested in and do the expensive queue poll only when any of them change (or if an hour elapsed since the last poll, as a safeguard).