search

dagu-org / dagu

Developer-friendly, minimalism Cron alternative, but with much more capabilities. It aims to solve greater problems.
https://dagu.readthedocs.io
GNU General Public License v3.0
1.44k stars 137 forks source link

[Feature Request] High availibility / cluster mode #361

Open cernoel opened 1 year ago

cernoel commented 1 year ago

Make it possible to run the server/schedulers multiple times (High availibility) in Kubernets, Swarm,.. but so that jobs do not run twice (do not get executed from other servers)

I saw some other simpler projects using: Raft or redis to atomicly lock jobs and distribute configuration. Also finding other servers/schedulers dynamically with: go-discover

yohamta commented 1 year ago

Thanks for making the issue and providing the library information. I suppose Raft would be viable for implementing HA/Cluster mode.

gedw99 commented 1 year ago

The easy way to do this is with NATS.

then you can allow jobs to run as singletons and also pipikibeninto each other across servers.

NATS is golsng .

Nats can also see all jobs and also store and diatribe the job descriptions files

Mut also has built in distributed security too .

i know I am making it sound perfect but a lot of people use nats for exactly this situation .

gedw99 commented 1 year ago

Btw Nats under the hood uses raft to do its thing ..

@yohamta