Workflow 调研

[ninehills]

Workflow 调研

workflow 有两层含义，一种叫做工作流（pipeline），还有一种可以叫做审批流（Process Management）。

工作流（Pipeline）

Airflow

Airflow is a platform to programmatically author, schedule and monitor workflows.

• 主页: https://airflow.incubator.apache.org/
• GitHub: https://github.com/apache/incubator-airflow
• Language: Python

luigi

Luigi is a Python (2.7, 3.3, 3.4, 3.5) package that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, handling failures, command line integration, and much more.

• GitHub: https://github.com/spotify/luigi
• Language: Python

Azkaban

Azkaban is a batch workflow job scheduler created at LinkedIn to run Hadoop jobs. Azkaban resolves the ordering through job dependencies and provides an easy to use web user interface to maintain and track your workflows.

• 主页: https://azkaban.github.io/
• GitHub: https://github.com/azkaban/azkaban
• Language: Java

Pinball

Pinball is a scalable workflow manager

• GitHub: https://github.com/pinterest/pinball
• Language: Python

Spinnaker

Spinnaker is an open source, multi-cloud continuous delivery platform for releasing software changes with high velocity and confidence. 其中的Pipeline部分，专门进行程序部署。

• 主页：https://www.spinnaker.io/
• Pipeline文档：https://www.spinnaker.io/concepts/pipelines/
• GitHub: https://github.com/spinnaker

Mistral

Mistral 是OpenStack的子项目，它的workflow定义的比较清晰明了。

• 主页：https://docs.openstack.org/mistral/latest/overview.html

CloudSlang

CloudSlang is an open source tool for orchestrating cutting edge technologies. It can orchestrate anything you can imagine in an agentless manner. You can use or customize ready-made YAML based workflows.

• 主页：http://cloudslang.io/
• 特性：
  • Process-Based，基于进程而不是状态机
  • Ready-made Content，社区良好的现成代码
  • Agentless，不需要单独的Agent，可以远程执行

其中比较亮点的应该是第一个，Process-Based。

审批流（Process Management）

django-lb-workflow

Reusable workflow library for Django.

• GitHub: https://github.com/vicalloy/django-lb-workflow
• Language: Python

Viewflow

Reusable workflow library for Django

• GitHub: https://github.com/viewflow/viewflow/
• Language: Python

SpiffWorkflow

• Github: https://github.com/knipknap/SpiffWorkflow
• Language: Python

参考文档

• 译-解密 Airbnb 的数据流编程神器：Airflow 中的技巧和陷阱
• Luigi vs Airflow vs Pinball
• Pinball review
• awesome-pipeline