Improved partition to datastreams assignment:

ActianCorp / spark-vector

Repository for the Spark-Vector connector

Apache License 2.0

20 stars 9 forks source link

Closed and-costea closed 8 years ago

and-costea commented 8 years ago

take into account when the number of datastreams / node was unbalanced, by having a target # of partitions / node (instead of overall)
make sure that any assignment is not heavily skewed: the difference between the maximum number of partitions and the minimum should be smaller than a fixed threshold

cbarca commented 8 years ago

nothing else to add

ship it