Yelp / pyleus

Pyleus is a Python framework for developing and launching Storm topologies.
Apache License 2.0
404 stars 113 forks source link

Error when running storm topology using new storm release-1.0.0 and pyleus-0.3.0 #181

Open dzyubanv opened 8 years ago

dzyubanv commented 8 years ago

I'm running storm topology "pyleus --verbose local xyz_topology.jar" using storm-1.0.0, pyleus-0.3.0, python 2.7.9, centos-6.6 and getting the Error "Exception in thread "main" java.lang.NoClassDefFoundError: backtype/storm/topology/IRichBolt". This is because the classes in new storm 1.0.0 release are no longer in backtype.storm.topology. The classes are in the org.apache.storm.topology and probably that is why pyleus can't handle them properly. The same topology worked well and without errors when using the old version of storm-0.10.0 and pyleus-0.3.0.

Is there any solution to resolve the issue and being able to run the topology on new storm-1.0.0 release with pyleus-0.3.0 ?

Also, I'm using the SimpleBolts class in the bolts written in python 2.7.9 by importing it from pyleus like "from pyleus.storm import SimpleBolt". Below is the output from the shell: Running: java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/local/apache-storm-1.0.0 -Dstorm.log.dir=/usr/local/apache-storm-1.0.0/logs -Djava.library.path=/usr/local/ lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/local/apache-storm-1.0.0/lib/asm-5.0.3.jar: /usr/local/apache-storm-1.0.0/lib/clojure-1.7.0.jar:/usr/local/apache-storm-1.0.0/lib/disruptor-3.3.2.jar: /usr/local/apache-storm-1.0.0/lib/kryo-3.0.3.jar: /usr/local/apache-storm-1.0.0/lib/log4j-api-2.1.jar:/usr/local/apache-storm-1.0.0/lib/log4j-core-2.1.jar
:/usr/local/apache-storm-1.0.0/lib/log4j-over-slf4j-1.6.6.jar: /usr/local/apache-storm-1.0.0/lib/log4j-slf4j-impl-2.1.jar:/usr/local/apache-storm-1.0.0/lib/minlog-1.3.0.jar: /usr/local/apache-storm-1.0.0/lib/objenesis-2.1.jar:/usr/local/apache-storm-1.0.0/lib/reflectasm-1.10.1.jar: /usr/local/apache-storm-1.0.0/lib/servlet-api-2.5.jar:/usr/local/apache-storm-1.0.0/lib/slf4j-api-1.7.7.jar: /usr/local/apache-storm-1.0.0/lib/storm-core-1.0.0.jar: /usr/local/apache-storm-1.0.0/lib/storm-rename-hack-1.0.0.jar:xyzTopology.jar: /usr/local/storm/conf:/usr/local/apache-storm-1.0.0/bin -Dstorm.jar=xyz_topology.jar com.yelp.pyleus.PyleusTopologyBuilder --local Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" java.lang.NoClassDefFoundError: backtype/storm/topology/IRichBolt at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) at java.lang.Class.privateGetMethodRecursive(Class.java:3048) at java.lang.Class.getMethod0(Class.java:3018) at java.lang.Class.getMethod(Class.java:1784) at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544) at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526) Caused by: java.lang.ClassNotFoundException: backtype.storm.topology.IRichBolt at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 7 more pyleus local: error: [StormError] Storm command failed. Run with --verbose for more info.

Thanks in advance Vadim Dzyuban

jizhouli commented 7 years ago

I have also encountered the same problem

sudip-haz commented 6 years ago

so how you solve it.

westover commented 6 years ago

This whole code base has been considered archived by Yelp. We are maintaining a mirror which supports Python3 but no future versions of storm yet.

westover commented 6 years ago

Streamparse supports more recent versions of Storm