Improve PartitionManager to cleverly merge partitions that can be joined

snuspl / cruise

Cruise: A Distributed Machine Learning Framework with Automatic System Configuration

Apache License 2.0

26 stars 2 forks source link

Improve PartitionManager to cleverly merge partitions that can be joined #111

Closed jsjason closed 8 years ago

jsjason commented 9 years ago

The PartitionManager should be smart enough to merge two adjacent partitions that belong to the same evaluator, e.g. Range[0, 10] and Range[10, 20], instead of regarding them as two separate partitions.

bgchun commented 9 years ago

A data structure similar to MerkleTree. MerkleTree is used when we deal with self-certifying data.

wynot12 commented 8 years ago

I'm closing it, since PartitionManager is deprecated.