robinhood / faust

Python Stream Processing
Other
6.7k stars 538 forks source link

Dagger - A Faust Based Workflow Engine #767

Open patkivikram opened 1 year ago

patkivikram commented 1 year ago

Building on the concepts of faust, we built out a micro-service orchestration/workflow engine - Dagger. Dagger operates at Faust scale to handle millions of long running tasks. It works similar to Java based workflow engines like Cadence or Camunda but integrates directly with Kafka