Could not find the main class: com.yelp.pyleus.PyleusTopologyBuilder

[zhaohanqing]

Hi guys, I'm a newbie of pyleus . I use "pip install pyleus" install it in my ubuntu-10.04 . when I follow guide in https://github.com/Yelp/pyleus to run the exclamation_topology.jar by "pyleus local exclamation_topology.jar" , however , throwout below exception: Exception in thread "main" java.lang.NoClassDefFoundError: backtype/storm/spout/MultiScheme Caused by: java.lang.ClassNotFoundException: backtype.storm.spout.MultiScheme at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) Could not find the main class: com.yelp.pyleus.PyleusTopologyBuilder. Program will exit.

I'm sure I successfully install pyleus with its dependence PyYaml , six , msgpack-python etc . I use "java -jar pyleus-base.jar" to test and also raise the exception ! I find it is really do not have PyleusTopologyBuilder.jar in my pyleus directory . What's wrong with it? How can I get or generate the depend jar of pyleus-base.jar need ? Do I need install mvn first before pip install pyleus ?

[zhaohanqing]

I had extract the pyleus-base.jar and find PyleusTopologyBuilder.class ,however why it is can not be found? I had config CLASSPATH to path of pyleus directory , it not work .

[zhaohanqing]

it seems not cp the pyleus-base.jar when execute command with log: Running: java -client -Dstorm.options= -Dstorm.home=/usr/lib/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -cp /usr/lib/storm/storm-0.8.1.jar:/usr/lib/storm/lib/tools.logging-0.2.3.jar:/usr/lib/storm/lib/httpcore-4.1.jar:/usr/lib/storm/lib/joda-time-2.0.jar:/usr/lib/storm/lib/jzmq-2.1.0.jar:/usr/lib/storm/lib/carbonite-1.5.0.jar:/usr/lib/storm/lib/clj-time-0.4.1.jar:/usr/lib/storm/lib/objenesis-1.2.jar:/usr/lib/storm/lib/reflectasm-1.07-shaded.jar:/usr/lib/storm/lib/commons-codec-1.4.jar:/usr/lib/storm/lib/snakeyaml-1.9.jar:/usr/lib/storm/lib/minlog-1.2.jar:/usr/lib/storm/lib/commons-logging-1.1.1.jar:/usr/lib/storm/lib/ring-servlet-0.3.11.jar:/usr/lib/storm/lib/core.incubator-0.1.0.jar:/usr/lib/storm/lib/jetty-util-6.1.26.jar:/usr/lib/storm/lib/ring-jetty-adapter-0.3.11.jar:/usr/lib/storm/lib/clojure-1.4.0.jar:/usr/lib/storm/lib/guava-13.0.jar:/usr/lib/storm/lib/slf4j-log4j12-1.5.8.jar:/usr/lib/storm/lib/kryo-2.17.jar:/usr/lib/storm/lib/disruptor-2.10.1.jar:/usr/lib/storm/lib/commons-io-1.4.jar:/usr/lib/storm/lib/servlet-api-2.5-20081211.jar:/usr/lib/storm/lib/jline-0.9.94.jar:/usr/lib/storm/lib/hiccup-0.3.6.jar:/usr/lib/storm/lib/ring-core-0.3.10.jar:/usr/lib/storm/lib/libthrift7-0.7.0.jar:/usr/lib/storm/lib/commons-exec-1.1.jar:/usr/lib/storm/lib/jetty-6.1.26.jar:/usr/lib/storm/lib/math.numeric-tower-0.0.1.jar:/usr/lib/storm/lib/jgrapht-0.8.3.jar:/usr/lib/storm/lib/clout-0.4.1.jar:/usr/lib/storm/lib/zookeeper-3.3.3.jar:/usr/lib/storm/lib/tools.cli-0.2.2.jar:/usr/lib/storm/lib/log4j-1.2.16.jar:/usr/lib/storm/lib/asm-4.0.jar:/usr/lib/storm/lib/commons-fileupload-1.2.1.jar:/usr/lib/storm/lib/curator-client-1.0.1.jar:/usr/lib/storm/lib/slf4j-api-1.5.8.jar:/usr/lib/storm/lib/servlet-api-2.5.jar:/usr/lib/storm/lib/tools.macro-0.1.0.jar:/usr/lib/storm/lib/curator-framework-1.0.1.jar:/usr/lib/storm/lib/httpclient-4.1.1.jar:/usr/lib/storm/lib/commons-lang-2.5.jar:/usr/lib/storm/lib/json-simple-1.1.jar:/usr/lib/storm/lib/compojure-0.6.4.jar:/usr/lib/storm/lib/junit-3.8.1.jar:exclamation_topology.jar:/root/.storm:/usr/lib/storm/bin -Dstorm.jar=exclamation_topology.jar com.yelp.pyleus.PyleusTopologyBuilder --local

what's wrong with it ?

[poros]

Have you tried installing Storm on your machine?

[zhaohanqing]

yes, but I install storm 0.8.1 , the version is too lower ?

[zhaohanqing]

yes, I replace storm-0.8.1 by storm-0.9.0.1 the class not found error disappear and I can load the jar to storm , however , another error :

java.lang.RuntimeException: Pipe to subprocess seems to be broken! Currently read output: ��pid���msg�
�Traceback (most recent call last): File "/usr/local/lib/python2.6/dist-packages/pyleus/storm

both spouts and bolts come across it

[zhaohanqing]

detail exception log below: Shell Process Exception:

at backtype.storm.utils.ShellProcess.readString(ShellProcess.java:118) ~[storm-core-0.9.0.1.jar:na]
at backtype.storm.utils.ShellProcess.readMessage(ShellProcess.java:64) ~[storm-core-0.9.0.1.jar:na]
at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:45) ~[storm-core-0.9.0.1.jar:na]
at backtype.storm.spout.ShellSpout.open(ShellSpout.java:36) ~[storm-core-0.9.0.1.jar:na]
at backtype.storm.daemon.executor$fn__3430$fn__3445.invoke(executor.clj:504) ~[storm-core-0.9.0.1.jar:na]
at backtype.storm.util$async_loop$fn__444.invoke(util.clj:401) ~[storm-core-0.9.0.1.jar:na]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
at java.lang.Thread.run(Thread.java:662) [na:1.6.0_41]

12892 [Thread-23-log-results] ERROR backtype.storm.util - Async loop died! java.lang.RuntimeException: Pipe to subprocess seems to be broken! Currently read output: ��pid�O��msg�
�Traceback (most recent call last): File "/usr/local/lib/python2.6/dist-packages/pyleus/storm/component.py", line 232, in run self.setup_component() File "/usr/local/lib/python2.6/dist-packages/pyleus/storm/component.py", line 192, in setup_component self.conf, self.context = self._init_component() File "/usr/local/lib/python2.6/dist-packages/pyleus/storm/component.py", line 307, in _init_component self._create_pidfile(setup_info['pidDir'], pid) TypeError: 'int' object is unsubscriptable

[zhaohanqing]

hi guys , I found the root cause of above exception . I came across the same situation : https://github.com/Yelp/pyleus/issues/103 After use the valid version of storm (0.9.2) , everything is OK . Thank goodness ! I suggest list the recommend of dependence at marked area of pyleus home page.

[poros]

It's a good suggestion. We'll do that, thanks.