search

devshawn / kafka-gitops

🚀Manage Apache Kafka topics and generate ACLs through a desired state file.
https://devshawn.github.io/kafka-gitops
Apache License 2.0
322 stars 71 forks source link

How to setup kafka-gitops with Kafka cluster running in AWS EKS? #40

Closed sergialonsaco closed 4 years ago

sergialonsaco commented 4 years ago

Hi!

Is there any way to setup kafka-gitops to work against a kafka cluster inside an AWS EKS cluster? Right now I am creating a docker image with the kafka-broker + kafka-gitops and running it on EKS, but that means that every time I want to execute a kafka-gitops operation I need to connect inside the EKS cluster. Is there any option to run the kafka-gitops from outside the EKS?

Thanks so much!

devshawn commented 4 years ago

Hi @sergialonsaco, sorry for the late response!!

I'm not super knowledgeable with EKS specifically, but kafka-gitops just needs access to the kafka broker. So, you need to run kafka-gitops from a machine or container that is able to connect to the kafka broker. You could run kafka-gitops from outside of EKS, inside an EC2 instance for example, as long as the networking is set up correctly inside of your VPC. You could also expose your kafka to the public internet or an internal VPN and use kafka-gitops from your local machine for example.

sergialonsaco commented 4 years ago

Thanks! totally agree! Will go for exposing the broker to an internal VPN :) thanks