search

vasia / gelly-streaming

An experimental Graph Streaming API for Apache Flink
Apache License 2.0
140 stars 44 forks source link

Add single pass aggregation support for global graph properties #9

Closed senorcarbone closed 7 years ago

senorcarbone commented 8 years ago

A preliminary aggregation scheme for global property computation can be the following:

  1. EdgeStream keyBy partitionID
  2. Parallel window fold for incremental local partial aggregation (e.g. bipartition groups, connected components etc)
  3. (Optional) combiner for partial states
  4. (Optional) merge function that converts partial state to final aggregation state
  5. (Optional) flag for transient/persistent state across windows