search

izzyacademy / kafka-in-a-box

Kafka in a Container
Apache License 2.0
14 stars 6 forks source link

Kafka commands responding very slowly #3

Open bharathreddy901 opened 2 years ago

bharathreddy901 commented 2 years ago

Hi All,

I have deployed Apache Kafka 3.0 on Kubernetes Locally with image(izzyacademy/kafka:3.0.0) in leagcy mode, when i logged in to kafka pod and running creating topics, console producer & console consumer commands its time taking around 30 seconds to 1 minute. Can you please help on this.

kafka-topics.sh kafka-console-producer.sh kafka-console-consumer.sh ./kafka-log-dirs.sh --describe --bootstrap-server localhost:9092 ./kafka-broker-api-versions.sh --bootstrap-server localhost:9092

Thanks.

izzyacademy commented 2 years ago

Hi @bharathreddy901

I think it is because of the settings in the values file.

https://github.com/izzyacademy/kafka-in-a-box/blob/main/kubernetes/helm-charts/desktop/values.yaml

The default values are meant to run it on smaller machines

Try upgrading the memory and CPU numbers for the Zookeeper and Brokers

Current Settings Intended for Smaller Machines

This allocates about a quarter of CPU time to the docker image and very little memory


zookeeper.zookeeperCPU=250m
zookeeper.zookeeperMemory=900Mi
zookeeper.zookeeperHeap="-Xms128m -Xmx512m"

broker.brokerCPU=250m
broker.brokerMemory=900Mi
broker.brokerCPU="-Xms128m -Xmx512m"

Try out these Larger Settings

This should provide you with sufficient CPU time and memory, so increase it to any number you have resources for locally on your machine.


zookeeper.zookeeperCPU=1000m
zookeeper.zookeeperMemory=2048Mi
zookeeper.zookeeperHeap="-Xms1024m -Xmx2000m"

broker.brokerCPU=1000m
broker.brokerMemory=2048Mi
broker.brokerCPU="-Xms1024m -Xmx2000m"

After you have increased these numbers, please let me know how it went.

Thanks.

bharathreddy901 commented 2 years ago

Thank you Israel Ekpo for your reply, I will increase the cpu & memory and let you know.

bharathreddy901 commented 2 years ago

Hi @izzyacademy,

I have increased the numbers for CPU, memory & heap memory as suggested. Now the commands are executing fast when compared to previously. But still i could see there is a lag about 5 seconds,may be I need to need to increase more cpu,memory & heap memory.

Thanks.