search

portals-project / portals

Portals is a framework for stateful serverless apps, unifying dataflow streaming with actors
https://www.portals-project.org/
Apache License 2.0
19 stars 2 forks source link

Dataparallel, distributed execution #147

Open jspenger opened 1 year ago

jspenger commented 1 year ago

There should be a distributed execution mode for Portals. The workloads should be split into partitions, and executed across distributed nodes. The distributed execution should provide fault-tolerance. The nodes are to communicate via Kafka (or similar) queues. Ask @jonasspenger for more details.