search

Mongey / terraform-provider-kafka

Terraform provider for managing Apache Kafka Topics + ACLs
MIT License
517 stars 129 forks source link

Provision topics with a Zookeeper provider #57

Closed edify42 closed 5 years ago

edify42 commented 5 years ago

The tooling only supports connecting to an array of brokers. We use zookeeper with JAAS auth to create topics (internally I think kafka-acl.sh gets a kafka broker directly to create the topic?)

Would be nice if this TF provider was able to use Zookeeper instead to provision topics and ACLs to replaces our commands:

kafka-acls.sh --authorizer-properties zookeeper.connect=$ZOOKEEPER --list kafka-topics.sh --create --zookeeper $ZOOKEEPER --replication-factor 3 --partitions 1 --topic my-topic

Mongey commented 5 years ago

👋 Hey @edify42, sorry for the delay. Can you explain why you would want to connect via zookeeper, rather than talking directly to Kafka?

edify42 commented 5 years ago

Hey @Mongey - we have a multi broker kafka deployment which we setup from Confluent documentation The kafka metadata is managed by Zookeeper and the out-of-the-box scripts provided in the kafka docs suggest we connect with zookeeper to manage things like topic creation and ACLs. Ideally we'd do it all via this TF provider so I think there's a requirement to connect to Zookeeper instead of Kafka directly.

Mongey commented 5 years ago

Topics and ACLs can be managed through the Kafka broker, and therefore via this provider, there’s no need to connect to zookeeper.

I’m going to close this out, I didn’t intend to support anything that is enabled by fudging configs in zookeeper to make Kafka do something. Those feature requests should happen in apache/Kafka and expose functionality via the api for others (this provider) to use.