Open ashley-walsh opened 1 year ago
I think you should use this middleware to wrap your enqueuer: https://pkg.go.dev/github.com/taylorchu/work@v0.2.9/middleware/unique.
It maps a job into a unique key and a unique duration.
The way how you did it will have race conditions.
Thank you so much for the fast response! The issue is, we have varying jobs already in existence that will require different unique durations (i.e. the cron job above runs daily, but others run every 3 days). Is there a non-duration solution? For example, is it possible to just base it off of whether the job ID is unique and whether a job with that same ID is currently in the queue?
Or if we are going about this incorrectly, what is the best way to proceed?
uniq function can switch by enqueue options https://pkg.go.dev/github.com/taylorchu/work@v0.2.9#EnqueueOptions too.
Hi there,
I am trying to have a cron job that only runs once every X hours (or daily, etc.) despite there being multiple instances of my application. When the job is instantaneous (or close, i.e. prints a line and returns) it works as expected (we run the job with a
UniqueJobId
and our uniqueness constraint is triggered (see below).But when the job is longer (our job takes several minutes to complete), the unique constraint is ignored across instances and the job is dequeued several times and wrongly runs several times (once per instance). We've tried to use
InvisibleSec
but we have found that other jobs just run after that time period-- i.e. if the job is set to run at 5:00 andInvisibleSec
is 60, one instance's job runs (correctly) at 5:00 and another runs at 5:01. We've also tried to see what we can do withEnqueueDelay
but that does not seem to be working either.Any help/insight would be greatly appreciated! See below for how we are setting up our cron service.