Note: There is a project page and documentation being build in url http://dynflow.github.io/. It's still work in progress but you may find useful information there. It'll eventually replace this README.
Dynflow [DYN(amic work)FLOW] is a workflow engine written in Ruby that allows to:
Dynflow doesn't try to choose the best tool for the jobs, as the right tool depends on the context. Instead, it provides interfaces for persistence, transaction layer or executor implementation, giving you the last word in choosing the right one (providing default implementations as well).
Dynflow has been under heavy development for several months to be able to support the services orchestration in the Katello and Foreman projects, getting to production-ready state in couple of weeks.
In traditional workflow engines, you specify a static workflow and then run it with various inputs. Dynflow takes different approach. You specify the inputs and the workflow is generated on the fly. You can either specify the steps explicitly or subscribe one action to another. This is suitable for plugin architecture, where you can't write the whole process on one place.
Dynflow doesn't differentiate between workflow and action. Instead, every action can populate another actions. This allows composing more simpler workflows into a big one.
The whole execution is done in three phases:
Plan phase
Construct the execution plan for the workflow. Two mechanisms are used to get the set of actions to be executed:
a. explicit calls of plan_action
methods in the plan
method
b. implicit associations: an action A subscribes to an action B, which means that the action A is executed whenever the action B occurs.
The output of this phase is a set of actions and their inputs.
Run phase
The plan is being executed step by step, calling the run method of an action with corresponding input. The results of every action are written into output attribute.
The run method should be stateless, with all the needed information included in the input from planning phase. This allows us to control the workflow execution: the state of every action can be serialized therefore the workflow itself can be persisted. This makes it easy to recover from failed actions by rerunning it.
Finalize phase
Take the results from the execution phase and perform some additional tasks. This is suitable for example for recording the results into database.
Every action can participate in every phase.
The examples
directory contains simple ruby scripts different
features in action. You can just run the example files and see the Dynflow
in action.
orchestrate.rb
- example worlflow of getting some infrastructure
up and running, with ability to rescue from some error states.
orchestrate_evented.rb
- the same workflow using the ability to
suspend/wakeup actions while waiting for some external event.
It also demonstrates the ability to cancel actions that support it.
remote_executor.rb
- example of executing the flows in external
process
# every action needs to inherit from Dynflow::Action
class Action < Dynflow::Action
# OPTIONAL: the input format for the execution phase of this action
# (https://github.com/iNecas/apipie-params for more details.
# Validations can be performed against this description (turned off
# for now)
input_format do
param :id, Integer
param :name, String
end
# OPTIONAL: every action can produce an output in the execution
# phase. This allows to describe the output.
output_format do
param :uuid, String
end
# OPTIONAL: this specifies that this action should be performed when
# AnotherAction is triggered.
def self.subscribe
AnotherAction
end
# OPTIONAL: executed during the planning phase. It's possible to
# specify explicitly the workflow here. By default it schedules just
# this action.
def plan(object_1, object_2)
# +plan_action+ schedules the SubAction to be part of this
# workflow
# the +object_1+ is passed to the +SubAction#plan+ method.
plan_action SubAction, object_1
# we can specify, where in the workflow this action should be
# placed, as well as prepare the input.
plan_self { id: object_2.id, name: object_2.name}
end
# OPTIONAL: run the execution part of this action. Transform the
# data from +input+ to +output+. When not specified, the action is
# not used in the execution phase.
def run
output[:uuid] = "#{input[:name]}-#{input[:id]}"
end
# OPTIONAL: finalize the action after the execution phase finishes.
# in the +input+ and +output+ attributes are available the data from
# execution phase. in the +outputs+ argument, all the execution
# phase actions are available, each providing its input and output.
def finalize
puts output[:uuid]
end
end
Every action should be as atomic as possible, providing better granularity when manipulating the process. Since every action can be subscribed by another one, adding new behaviour to an existing workflow is really simple.
The input and output format can be used for defining the interface that other developers can use when extending the workflows.
Dynflow::Action
. Defines code to be run in
plan/run/finalize phase. It has defined input and output data.plan
method of the action. The execution follows immediately.plan
method into
action input, that can be accessed from the run
/finalize
phase.plan
method of the actionForeman - lifecycle management tool for physical and virtual servers
Katello - content management plugin for Foreman: integrates couple of REST services for managing the software updates in the infrastructure.
Foreman-tasks - Foreman plugin providing the tasks management with Dynflow on the back-end
Dyntask - generic Rails engine providing the tasks management features with Dynflow on the back-end
Sysflow - set of reusable tools for running system tasks with Dynflow, comes with simple Web-UI for testing it
MIT
Ivan Nečas, Petr Chalupa