Yelp / pyleus

Pyleus is a Python framework for developing and launching Storm topologies.
Apache License 2.0
404 stars 113 forks source link

How can I include other python modules in pyleus build? #175

Closed jayadevanm closed 7 years ago

jayadevanm commented 8 years ago

My spout is a kafka consumer, using pykafka. When I issue pyleus build I get an error

  File "/tmp/tmpM_0bw1/resources/blue_kava_topology/test_word_spout.py", line 3, in <module>
    from pykafka import KafkaClient
ImportError: No module named pykafka

If I write a simple python script using pykafka, it works. If I just do a pip freeze from commandline, I can see pykafka. I assume there is some way to include pykafka when pyleus creates its pip and virtualenv and creates the jar. But how?

jayadevanm commented 8 years ago

To answer my own question - add a file named requirements.txt in the folder where the yaml file resides and make the necessary entries. In my case, the entries are

psycopg2
pykafka