Not able to get zookeeper partition .Its throwing error as NoNodeError.

[souravroy0708]

I am created kafka topic streaming1 with 2 partition and my kafka ,zookeeper server are running.I am able to publish and consume message.I am expecting zk.get_children('/consumers/streaming1') should give the output but its throwing error as NoNodeError .Please anyone can look into it.

I want to implement as per below link: https://stackoverflow.com/questions/44110027/pyspark-kafka-direct-streaming-update-zookeeper-kafka-offset

Code to reproduce the issue:

def get_zookeeper_instance(): from kazoo.client import KazooClient if 'KazooSingletonInstance' not in globals(): globals()['KazooSingletonInstance'] = KazooClient(ZOOKEEPER_SERVERS) globals()['KazooSingletonInstance'].start() return globals()['KazooSingletonInstance']

partition = zk.get_children('/consumers/streaming1') # streaming1 is my topic name

Error:

NoNodeError

Python and kazoo library version:

Python 2.7.15 kazoo==2.6.1

start the Zookeeper server: bin/zookeeper-server-start.sh config/zookeeper.properties

start the Kafka server: bin/kafka-server-start.sh config/server.properties

create a topic: bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 2 --topic streaming1

start producer: bin/kafka-console-producer.sh \ --broker-list localhost:9092 \ --topic streaming1

start consumer: bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic streaming1 --from-beginning

[StephenSorriaux]

Hi,

I don't see any issue here: you are creating a topic in Kafka and then producing some messages to it. There is no reason to see the znode /consumers/streaming1 being created somehow since Kafka uses Zookeeper only for the controller election, ACLs, brokers information and topics configuration.

In the link you provided, Kazoo is used to save the offsets of each Kafka consumer in Zookeeper for monitoring purposes, via pyspark.