quantum-elixir / quantum-core

:watch: Cron-like job scheduler for Elixir
https://hexdocs.pm/quantum/
Apache License 2.0
2.3k stars 147 forks source link

In cluster mode jobs are scheduled multiple times #478

Closed rradz closed 3 years ago

rradz commented 3 years ago

Hello, I have been testing Quantum 3.3 and I noticed an issue when running in cluster mode. I defined a function which was basically just a log entry "[time] Hello from #{Node.self()}" and made it run every minute. Then I run my tiny test project as 3 separate nodes on my computer.

Results were unexpected.

What I expected:

What I got:

Basically, in cluster mode, using default strategy, jobs are executed on random node, but they are scheduled once per each node. That seems to be defying the purpose and I'd like to know how to solve this.

maennchen commented 3 years ago

@rradz Quantum 3.0 removed that mode:

https://github.com/quantum-elixir/quantum-core/blob/master/CHANGELOG.md#changed-1

See https://github.com/quantum-elixir/quantum-swarm

dantswain commented 3 years ago

Hi I ran into this exact same problem. I think the design is probably good, but the documentation could really be improved. The quantum-core documentation kind of leads me to expect that this will "just work" when I set the run policy to {Quantum.RunStrategy.Random, :cluster}. It's possible that I'm thrown off a little because I haven't used this lib in a while and last time I did the global feature was still supported. The README for quantum-swarm could be a lot more useful - I had to dial in to the source code for quantum-swarm to see how to use it.

I'd be glad to submit PRs to help with the docs (and maybe quantum-core could/should emit a warning if you have global: true), but I'm super busy :( In about a month I might have some time, so if it's still an issue then maybe I can help.

maennchen commented 3 years ago

@dantswain I'll happily accept PRs :)