search

seomoz / qless-core

Core Lua Scripts for qless
MIT License
85 stars 34 forks source link

Resolving dependencies on job failures #33

Closed wr0ngway closed 11 years ago

wr0ngway commented 11 years ago

From what I can tell, if a job fails, none of its dependent jobs get triggered. Is this expected behavior? If so, it would be nice if the behavior could be configured so that I can depend on a job completing, be it successful or not.

myronmarston commented 11 years ago

It's 100% the way dependencies were designed to work. The use case we had was multiple jobs that operate on an external resource in sequence. Job B depends on Job A putting that resource into a particular state. If Job A fails, job B can't proceed because it's unknown if the resource is in the state it requires.

What is your use case? Dependencies may not be the right tool for it. Alternately, you can use a middleware to undepend jobs on failure, in order to get the behavior you're looking for

wr0ngway commented 11 years ago

Thats fine, just wanted to make sure I wasn't missing something, I have something in my middleware to remove dependencies on a failure to work around this.

We are using the dependency mechanism to implement a poor-mans job concurrency, and completion rather than success is the important event for determining if a dependent job can run. Once #27 is completed, we'll probably switch to that instead.