vasia / gelly-streaming

An experimental Graph Streaming API for Apache Flink
Apache License 2.0
138 stars 44 forks source link

Introduce merge-tree abstraction #4

Closed vasia closed 9 years ago

vasia commented 9 years ago

For the bipartite detection algorithm and the top-k degrees.

balidani commented 9 years ago

So far we have two implementations that use a merge-tree and here is what it looks like when we create one:

edges
        .map(new InitCandidateMapper())
        .map(new BipartitenessMapper())
        .groupBy(new MergeTreeKeySelector(0))
        .map(new BipartitenessMapper())
        .groupBy(new MergeTreeKeySelector(1))
        // Repeat the above 2 lines log(DoP) times
        .map(new BipartitenessMapper());

The initial mapper just converts the edge to a format that is used by the main one. What do you think, what would be a good way to express this in gelly-streaming?

edges
        .mergeTree(new InitialMapper())
        .with(new SomeMergeTreeMapper())
        .on(0);

I was thinking of something like this, probably with different wording for the function names. The last part of it just receives the position of the key to later group by to form the tree.

balidani commented 9 years ago

This is implemented now