UnderstandLingBV / Tuktu

Big Data Science Swiss Army Knife - http://www.tuktu.io --
http://www.understandling.com
Other
60 stars 16 forks source link

Add DP anchoring on distributed flows #36

Closed ErikTromp closed 8 years ago

ErikTromp commented 8 years ago

When a flow is ran on multiple nodes, for aggregation (and other) purposes, it is highly valuable to have data with a specific key always flow through the same node. This applies to generators that can receive data (JS Generator, (A)Sync Generator) but also to all the meta processors that make use of this.