Closed xanderdunn closed 11 months ago
The Redis INCR
operation would work well here. On each iteration of the nodes, each replica can increment a replica_id
value in the DB and INCR will atomically return the current value and increment it by 1.
This would require #3.
The current implementation with separate Watcher and Processor pods does not rely on any pod naming for its partitioning of work across nodes. Hence, we're using Deployments rather than StatefulSets. Each Processor randomly chooses a node to process and then performs leader election with a lease on that node's name. If some other Processor is already operating on that node, then it fails leader election and randomly chooses a different node to work on.
Currently the service is deployed as a StatefulSet so that each node has a name with an incrementing identifier:
kube-state-rs-0
,kube-state-rs-1
, ...This is so that we can choose which partition of nodes belongs to this particular replica:
However, the service itself is stateless and makes use of no other feature of a StatefulSet. It would therefore be desirable to use a different mechanism for knowing the replica ID so that we can use a stateless
Deployment
.