search

Rykian / clockwork

A scheduler process to replace cron.
MIT License
543 stars 66 forks source link

A job can be triggered twice if the process is restarted at the wron moment #10

Open vddgil opened 7 years ago

vddgil commented 7 years ago

Hello, I have the following job definition: every 1.hour, 'Follower Flow Follow Emails', at: '**:00' So this job will be triggered at 01:00:00, 02:00:00, ...

The problem is I'm deploying with capistrano and if I deploy between 01:00:00 and 01:00:59. The job is trigerred again.

So I ended up with the folowing solution: every 1.hour, 'Follower Flow Follow Emails', at: '**:00', if: ->(time) { time.sec.zero? }

Is that ok or there is an other way to do this ?

Thanks in advance,

Gil

blackjid commented 7 years ago

I have a similar problem, what is the reasoning of triggering all the jobs when the clockwork process start??

blackjid commented 7 years ago

https://github.com/Rykian/clockwork/pull/8 maybe this would work

blackjid commented 7 years ago

mm, well, I don't think I had the same problem... Now I think I understand better how clockwork work.

dup2 commented 7 years ago

We have the same issue.

We run jobs from database events (as documented with "sync_database_events"), the jobs have a "at" attribute like "Friday 11:00" and "every" is set to "week", so this jobs should run at 11:00 every Friday.

If the process running clockwork is restarted between 11:00:00 and 11:00:59, the job is triggered again.

Epigene commented 6 years ago

This touches an architectural pattern.
Consider these solutions:

One: Update the at option syntax to allow precision in seconds. Downside - if you deploy on the specified second, the job is never executed. A smart workaround to this would require either clockwork keeping a log of executed jobs and providing some ensurement functionality or developers making jobs indempotent and calling them at least two times to make sure that if one execution falls through due to restart, the other is executed. This runs into "fallbacks all the way down" architecture, yuck.

Two: Change nothing in clockwork - jobs are specified with a maximum at specificity of a minute. Developers realize that this exposes them to situations where jobs can get executed twice if a restart happens on the triggering minute. Developers write idempotent jobs or jobs that halt their execution if triggered to soon ("cooldown" behavior).

Recommendation: Make clockwork readme explicitly state that double execution is a design choice.
Urge developers to write idempotent/cooldown jobs.