search

liftbridge-io / go-liftbridge

Go client for Liftbridge. https://github.com/liftbridge-io/liftbridge
Apache License 2.0
68 stars 18 forks source link

assign parition number in kuberntes #103

Closed mehran-prs closed 3 years ago

mehran-prs commented 3 years ago

How should we assign partition numbers in Kubernetes which has multiple pods of our program and each pod should get a partition number?

tylertreat commented 3 years ago

Are you referring to partitioning rolling updates for StatefulSets in Kubernetes or something else?

mehran-prs commented 3 years ago

no, I mean I deploy my app to Kubernetes (using deployment) with three replicas, now I have three running instance of my app. Let say I have a subject named "order.created" with 4 partitions, so we need to subscribe to these 4 partitions, how should I detect partition number to subject in each instance?

tylertreat commented 3 years ago

Stream partition assignments are automatically managed by the Liftbridge cluster. The client should just subscribe to, e.g. "order.created, partition 2" and the Go client library will resolve which node in the cluster is the leader of this partition and make the subscribe request to this node.

mehran-prs commented 3 years ago

Thank @tylertreat, but I mean something like consumerGroup, to do not set partition number explicitly.

tylertreat commented 3 years ago

I understand now. Consumer group functionality is still in development (see roadmap). Until then, you must manually assign partitions to consumers.