how to write Topology for kafka

[zhaohanqing]

Hi guys, When I write a Topology for kafka , I got an error when build:

_pyleus build: error: [VirtualenvError] Failed to execute Python module: kafka_spout_example.msg_print. Error: /tmp/tmpllh9zN/resources/pyleus_venv/bin/python: No module named kafka_spoutexample

it seems bolt module can not be found , however , I create my directory:

kafka_spout_example

pyleus_topology.yaml kafka_spout_example

msg_print.py

My topology:

name: kafka_spout_example topology: ``` - spout: name: kafka-my_topic type: kafka options: # The Kafka topic to stream from. # Required. topic: my-replicated-topic5 # ZooKeeper connection string. Comma-separated list of ZooKeeper # servers. # Required. zk_hosts: localhost:2181 # Root path in ZooKeeper to store consumer offsets. # Defaults to: /pyleus-kafka-offsets/ zk_root: /pyleus-kafka-offsets/kafka_spout_example # Kafka consumer ID. # Defaults to: pyleus- consumer_id: pyleus-kafka_spout_example # Whether the initial offset should be that specified by # start_offset_time (true) or the head of the stream (false). # Defaults to false. from_start: false # The offset time to start with if from_start is true. # Defaults to the earliest offset time in the stream. start_offset_time: 2147483648 - bolt: name: kafka_msg_print module: kafka_spout_example.msg_print parallelism_hint: 1 groupings: - shuffle_grouping: kafka-my_topic ```

[jswetzen]

Hi @zhaohanqing, I think your problem is that you are missing an empty file named "init.py" in your inner kafka_spout_example folder. That file makes it a python module. Just create it and you should be set.

[zhaohanqing]

Hi @jswetzen , you are right , after add init.py it worked . Thank you very much !

[jswetzen]

You're welcome!