bbcarchdev / libcluster

Clustering support library (originally part of anansi)
Apache License 2.0
0 stars 2 forks source link

Add partitioning #15

Closed nevali closed 8 years ago

nevali commented 8 years ago

Currently a cluster with a given name operates as a single multi-worker unit. It can be advantageous, particularly when libcluster is used in conjunction with libmq, to allow arbitrary application-defined partitioning of the cluster.

This can be implemented by way of cluster_partition() and cluster_set_partition() APIs, which set a short (16-character) string naming the cluster partition this member belongs to, and engines will balance different partitions distinct from one another (e.g., the default partition may have 200 workers, while a low-priority partition might only have four). The name of the partition, when non-default, will be available to applications for whatever distinguishing purpose they might require.

For example, libcluster might be used along with libmq to load-balance a stream of incoming messages. If the libmq engine supports it, the partition name can be used to differentiate particular classes of message and direct them only to the cluster configured to deal with them.