search

seomoz / qless

Queue / Pipeline Management
MIT License
294 stars 76 forks source link

Conditionally Chaining Jobs together #191

Open StephenOTT opened 10 years ago

StephenOTT commented 10 years ago

Do you have a recommendation on best practice for conditionally chaining jobs together? I was just running if statements at the end of a job to call the next job. But just wanted some feedback

dlecocq commented 10 years ago

Sorry for the late reply. Off hand, having some ifs at the end seems reasonable. There may exist a more elegant mechanism, but one doesn't jump to mind.

myronmarston commented 10 years ago

@StephenOTT -- you may want to check out plines. It may not do exactly what you want but it should give you ideas, at least. It provides an API for constructing job batch pipelines that conditionally fan out and use dependencies based on the data that goes into the batch.