Priming kafka-manager with 'zkhosts' of a known set of kafka clusters

[vvutharkar]

Hi,

We want to automate some of the kafka management by

• deploying a few kafka-manager instances via automation (puppet etc)
• Each instance of kafka-manager is responsible for only a known set of kafka clusters (a bare bones compartmentalization)
• We also intend to disable the 'add cluster' feature so that our customers cannot modify the scope of a given kafka-manager (what clusters you can manage via that particular instance)

To achieve this, we also need to be able to prime (pre-populate) a kafka-manager instance with a set of kafka clusters' 'zkhost' lists. We intend to provide the list via the application conf (As a list of lists) and set that value in the conf during puppet install of the kafka-manager.

But I am struggling a bit to find a nice 'bootstrap' (Startup) point in the lifecycle of the kafka-manager app to insert a populate_kafka_clusters() method. May be because I am very new to Scala and Play. Can you please point me to such a location or if there is none, can it be added so that other such initializations can be added by others at that interception point? Also can you point me to the code that takes the 'zkhosts' for a cluster and adds them to the backend store (kafka-manager's Zookeeper namespace /kafka-manager I presume)

[patelh]

We may be able to introduce a way to make the kafka manager "read-only" in terms of adding/modifying/deleting clusters. And for bootstrap, we can just bundle a CLI utility which you can use to define a new cluster.

[sherzberg]

I would like a way to specify the clusters in the application.conf as well. It would also be nice to have a read only option too.

[miketheman]

I believe this effort has started inh ttps://github.com/yahoo/kafka-manager/commit/18d370dd60896100927dc17ced766bd358b56645 and has been getting updates as needed. The conf file has an entry for "enabled features", so an admin of KM can choose behaviors.