distribworks / dkron

Dkron - Distributed, fault tolerant job scheduling system https://dkron.io
GNU Lesser General Public License v3.0
4.23k stars 374 forks source link

Job should execute when schedule in the past #460

Open congphan opened 5 years ago

congphan commented 5 years ago

I encounter an issue that is when I try to create a job which will be execue at specific time. But some time due to network latency,... then the job is created at time greater than specific time the job should be run. So the schedule is in the past and it can not be executed. Is there any option can make dkron auto execute a job if the schedule put in the past?

Thanks.

vcastellm commented 5 years ago

Currently there is no such option, some feature could be introduced like the epsilon concept that gives a time window to execute the job it is not executed for some reason but it's not currently implemented.

Anyway, jobs are created quite fast in Dkron, what time difference from the job's time do you use to create jobs? seems that it should be small. Could you post some examples?

vcastellm commented 5 years ago

Closing for no answer

congphan commented 5 years ago

Hi Victor,

I don't know why I did not receive any replied email from you until you closed this issue. In my situation when I schedule a job throuh microservice ảchitecture-drop message to rabitmq and somehouse the service would consume the message down at that time. When the service is up but the message contains the scheduled time is in the past now so the job can not be execute by dkron. I think it will be greate if have an option to run a job frome the past if we want :)

Thank & best regards

On Wed, May 15, 2019 at 21:57 Victor Castell notifications@github.com wrote:

Closed #460 https://github.com/victorcoder/dkron/issues/460.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/victorcoder/dkron/issues/460?email_source=notifications&email_token=AB7QY462D7WYATPTWJH7IHLPVQQGNA5CNFSM4GE5EW42YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOROYKR5Y#event-2343610615, or mute the thread https://github.com/notifications/unsubscribe-auth/AB7QY45TVECSBX4SSHKNAALPVQQGNANCNFSM4GE5EW4Q .

--


Cong Phan | Senior Full Stack Engineer

Address VN Ehome3 Condominium – Room A8-610, Ho Hoc Lam Street,Ward An Lac, District Binh

Tan, Ho Chi Minh city, Vietnam

E-mail thanhcongsp@gmail.com thanhcongsp@gmail.com

Mobile +84 (0) 949 187 087

Skype thanhcong.phan

Connect. Get ready. We deliver.

yvanoers commented 5 years ago

It seems to me like dkron is being used somewhat like a task queue processor in this scenario. That could actually be very useful, but it is not the intended use of a task scheduler. So IMHO there is a risk here of adding a feature that violates the philosophy of doing one thing well.

However, as a suggestion for implementing such a thing, @victorcoder : What about an additional schedule keyword: @after [RFC3339 datetime] <[ISO8601 duration] For instance, a job with a scheduled like this: @after 2020-01-01T00:00:00Z <P2H would run at 2020-01-01 00:00:00 if created before then, would run immediately if created anywhere between 00:00 and 02:00 on 2020-01-01, would never run if created after 2020-01-01 02:00:00.

Of course, the first question to answer is whether the use case should be supported at all.

drauschenbach commented 3 years ago

Currently Dkron does not run jobs if there is no leader at the time when a scheduled job should be kicked off. My team struggles with this on a regular basis, since our private network has a nightly maintenance outage. This issue sounds very related. If a Dkron node becomes a leader, shouldn't it consider kicking off a job that was supposed to be launched a minute ago?

vcastellm commented 1 year ago

I can consider implementing something like @yvanoers suggested to fix the use case of missing a job creation using a job queue.

In your case @drauschenbach, it's related but a bit different, do you think that re-triggering jobs with missed executions on becoming leader will cover your use case?

Why "a minute ago"? and not any amount of time?