search

banzaicloud / koperator

Oh no! Yet another Apache Kafka operator for Kubernetes
Apache License 2.0
782 stars 195 forks source link

Replace ZooKeeper with a Self-Managed Metadata Quorum #867

Open TritoXid opened 1 year ago

TritoXid commented 1 year ago

Hello, I have one question about kafkacluster and zookeeper. I think that kafka from version 2.8.0 no need more zookeeper because they implemented new solution for kafka metadata. I checked helm charts for KackaCluster, but I didn't see some solution how it's possible removed zookeeper cluster, because kind: KafkaCluster contains zkAddresses variable. Have you some solution for this or are you plan it add it in future?

Docs: KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum More information here: ZooKeeper Removal from Apache Kafka

Thx for response.

panyuenlau commented 1 year ago

Hey @TritoXid, thank you for bringing this up.

We did notice that Kafka Raft was introduced to replace the dependency on Zookeeper when Kafka 2.8.0 was released, however, we also noticed that Kafka Raft was only for development use and not yet production-ready, and this is why we haven't incorporated it into Koperator.

We have just planned to incorporate KRaft into the Koperator as the Kafka community marked KRaft as production-ready and planned to release it in Kafka 3.3 (thanks to @hi-im-aren's finding on this), so, please stay tuned 😉

Reference about the plan on KRaft by the Kafka Community:KIP-833: Mark KRaft as Production Ready

phuc1998 commented 1 year ago

Kafka 3.3.1 has been released and it marks KRaft as production-ready. Do you have any plan to support it? https://downloads.apache.org/kafka/3.3.1/RELEASE_NOTES.html

panyuenlau commented 1 year ago

Kafka 3.3.1 has been released and it marks KRaft as production-ready. Do you have any plan to support it? https://downloads.apache.org/kafka/3.3.1/RELEASE_NOTES.html

Thanks for the info @phuc1998, we do have a plan to incorporate this into Koperator, however, we don't really have an estimate date at the moment. It would be appreciated if you'd like to take a lead on this

viktorsomogyi commented 1 year ago

@panyuenlau do you have any more recent news when will this come to Koperator? (Also we would be happy to contribute and help out if you think you're short on time or resources.)

mingmingshiliyu commented 1 month ago

Kafka 3.3.1 has been released and it marks KRaft as production-ready. Do you have any plan to support it? https://downloads.apache.org/kafka/3.3.1/RELEASE_NOTES.html

Thanks for the info @phuc1998, we do have a plan to incorporate this into Koperator, however, we don't really have an estimate date at the moment. It would be appreciated if you'd like to take a lead on this

has the Kraft mode milestone for kafka 4.0 until April 2024 shown in your doc released in time?