chaps-io / gush

Fast and distributed workflow runner using ActiveJob and Redis
MIT License
1.05k stars 106 forks source link

Conditionally execution #42

Open KirillSuhodolov opened 7 years ago

KirillSuhodolov commented 7 years ago

In continuation of the conversation at issue https://github.com/chaps-io/gush/issues/41

May be it possible to run Job only that satisfied condition. Due to in runtime, it may be not need to execute. The api can looks like:

class SupplierWorkflow < Gush::Workflow
  def configure
    run AskSupplierAboutAvailableAmount
    run TryToOrderProducts, should(or condition): ->(previous_result, passed_params) { true }
  end
end

All this questions about to adopt this gem for complex workflows, that I have in project, where they are not linear, and branching based on conditions or failures.

P.S. When do jobs create? When method .start! on workflow ran all described jobs creates or after previous jobs executed, next creates and put to the queue?

pokonski commented 7 years ago

Can you share a real life example on when one would need dynamic branching? Because this goes against the basic design of Gush.

Jobs are created when workflow is first created and persisted. But they are put on queue when any job's descendants finish successfully.

scarfacedeb commented 7 years ago

@pokonski I have a similar situation:

I have a pdf manual pipeline that looks like this:

database row with url -> DownloadJob -(manual.id)-> ProcessingJob -(manual.id)-> PublishJob

Gush seems like a good fit for my case, but unfortunately I also need to deal with errors on every step.

e.g. if DownloadJob failed to download the manual, I'll need to break out the workflow and update the source with the given error.

The best way to deal with it would've been a conditional execution, like:

success: DownloadJob -(:ok, manual.id)-> ProcessingJob
error: DownloadJob -(:error, ["Wrong mime type"] -> ErrorInSourceJob
pokonski commented 7 years ago

Okay thanks for feedback! I'll look into that but at the same time I do not want break compatibility with older versions, so this would be most likely opt in