Multitennant ZK ensembles?

[tombentley]

It may be possible for a single ZK ensemble to support multiple Kafka clusters. To know whether that would be worthwhile we need a better understanding of:

1. How heavily utilised the ensemble is by a single cluster of a given size (in particular, peak utilisation), and thus some idea of "sharing ratio".
2. How we could migrate a Kafka cluster using a multi tennant ensemble to a dedicated one.

[scholzj]

Some further points to consider / keep in mind:

• The weak security model should be taken into consideration
• The migration would IMHO not work without complete cluster shutdown
• What are the fixed costs for running ZK cluster versus the variable costs caused by the user (Kafka in this case). Since the pods can be vertically scaled we need to know how much more efficient is one big ZK cluster with X tenants compared to X small single-tenant clusters.

[tombentley]

• The weak security model should be taken into consideration

ZK has per-znode ACLs. Or did you mean some other security aspect?

[scholzj]

Well, the ACLs set up by Kafka are AFAIK only about write access, right? So everyone has read access. We either need to address this or use the multi-tenant ZK only in cases where it doesn't matter, such as:

• All tenants are the same customer, just different Kafka clusters. So the read access might not be a problem for them.
• Actual users have no access to ZK. They either manage things through Kafka or through our management console. ZK is only used directly by Kafka and our own tools.

[scholzj]

The cluster controller currently deploys both Kafka and Zookeeper from single ConfigMap. In order to make ZK multitenant, we would need to split the ConfigMap into two - one for Kafka and one for Zookeeper and add a "link" to the zookeeper node to the Kafka config map.

[scholzj]

This idea has been abandoned as there is currently no need for it. I will close it and we can raise a new one later if needed.