ajvb / kala

Modern Job Scheduler
MIT License
2.13k stars 187 forks source link

Job Dependencies #148

Open shunte88 opened 7 years ago

shunte88 commented 7 years ago

With all of the scheduling engines I've come across if I have a list of dependencies then all of these dependencies should be met for a job to execute.

I setup a job with 3 parent dependencies expecting the job to execute when all three hard run to success

What occurred was the initial job ran to success and the job executed It also executed following parent two And again following parent three

Well I suppose that's one use case, and I can see uses for that if I had granular jobs with distinct functions that I could mix and match

I guess my line of thinking was flavored by working with several enterprise scheduler where if I'd a list of jobs to follow this was out of the box a hard dependency; most of these enterprise level scheduler give the flexibility for defining other levels, MUST, REQUIRES, FOLLOWS, NEEDS, etc that further refine these dependencies on both jobs and resources.

So, it looks like a list of dependencies in the Kala world is not a hard requirement list, the sibling job will execute following each parent specified

Rather than relying on the assumed logic that all parents MUST complete in a given scheduled process in order to achieve the end goal the jobs should be chained and the sibling job should follow the penultimate job in the chain.

Or better still an attribute could be surfaced that adds a simple bool switch to specify all_parents_required; all parent dependencies are required for a sibling job to execute

Could this functionality be added?

LordBrain commented 6 years ago

I could really use something like this as well. I want to trigger a child job after all the of the parents repeat runs are complete as well.

Are these on the radar at all?

gwoo commented 6 years ago

We could add this feature. First would be to create a test case that clearly represents the goal. Then we can work on adding the functionality to solve it.