Use Jesqueue's build in delayed job feature

[peh]

Since quite some time now the jesque library supports delayed enqueuing of Jobs. the grails plugin still uses it's own delay implementation which is not super different from the "original" one. I guess switching to the original one is not to awfully hard no?

[edwardotis]

This would also provide access to Client#removeDelayedEnqueue() for when we need to cancel a scheduled job. In the current grails implemenation, I don't see a way to destroy or remove a scheduled job.

net.greghaines.jesque.client.Client#removeDelayedEnqueue
net.greghaines.jesque.client.Client#delayedEnqueue

[edwardotis]

I just got around to testing it, and there was no problem using the underlying jesque client for delayedEnqueue and removeDelayedEnqueue.

My test job executed at the correct time. I was also able to delete a delayed test job correctly.

Using: compile "org.grails.plugins:jesque:1.2.1" grails 3.1.16

Here's examples using the jesque client syntax:

        jesqueService.jesqueClient.delayedEnqueue("myqueue",
                new Job(MyJesqueJob.simpleName, [user.id, myOtherArg]), myScheduledDateTime.getMillis()
        )

      jesqueService.jesqueClient.removeDelayedEnqueue("myqueue",
            new Job(MyJesqueJob.simpleName, [user.id, myOtherArg])
        )

So, it appears that it would be straight forward to use the jesque client in the next release.

[edwardotis]

Actually, it would also require the grails plugin to integrate the 2.1.1 jesque project implementation of WorkerImpl.pop(), which uses a lua script to ensure that only a single worker runs a delayed and/or scheduled task. Otherwise, user is exposed to a pre jesque 2.1.1 bug where multiple workers execute a single delayed or recurring task. (This can be a disaster if the task is not explicitly, defensively coded to protect against this case. I've had to implement custom locking logic to ensure this in my app.) https://github.com/gresrun/jesque/issues/91

In the meantime, the current grails plugin doesn't support removing delayed tasks, which can be a deal breaker for many apps, and is natively supported in jesque.

[edwardotis]

fyi, I confirmed that v1.2.1 grails plugin version of recurring jobs does not suffer from the multiple workers concurrently executing the recurring job jesque bug, even in a distributed environment. (Very good!) i.e. Only one worker thread on one server executes the job at each recurring interval.

[bp-FLN]

I just worked on this a bit. See #27

Actually the jesque delayed job feature works a bit differently: A delayed queue can not be used as a regular queue and vice versa. Workers have to be configured to also poll delayed queues. The workers pop method can handle both regular and delayed queues.

I managed to keep the plugins old behaviour by transparently handling of the jesque delayed queues. This means that using the same queue name for enqueue and enqueueAt will still work.

Input welcome!