search

Yelp / pyleus

Pyleus is a Python framework for developing and launching Storm topologies.
Apache License 2.0
404 stars 113 forks source link

Clarify README re: "You do NOT need to install pyleus on your Storm cluster" #95

Closed moandcompany closed 9 years ago

moandcompany commented 9 years ago

Hi,

The README might need an update on the remark below: "You do NOT need to install pyleus on your Storm cluster."

Installing Pyleus on Storm supervisors appears to address some Topology execution errors.

Here's the start of the error stack from my recent debugging experience:

  2015-01-28 23:36:21 b.s.t.ShellBolt [ERROR] Halting process: ShellBolt died.
  java.io.EOFException: null
    at org.msgpack.io.StreamInput.readByte(StreamInput.java:60) ~[stormjar.jar:na]
    at org.msgpack.unpacker.MessagePackUnpacker.getHeadByte(MessagePackUnpacker.java:66) ~[stormjar.jar:na]
    at org.msgpack.unpacker.MessagePackUnpacker.trySkipNil(MessagePackUnpacker.java:396) ~[stormjar.jar:na]
    at org.msgpack.template.MapTemplate.read(MapTemplate.java:59) ~[stormjar.jar:na]
    at org.msgpack.template.MapTemplate.read(MapTemplate.java:27) ~[stormjar.jar:na]
    at org.msgpack.template.AbstractTemplate.read(AbstractTemplate.java:31) ~[stormjar.jar:na]
    at org.msgpack.MessagePack.read(MessagePack.java:527) ~[stormjar.jar:na]
    at org.msgpack.MessagePack.read(MessagePack.java:496) ~[stormjar.jar:na]
    at com.yelp.pyleus.serializer.MessagePackSerializer.readMessage(MessagePackSerializer.java:198) ~[stormjar.jar:na]
    at com.yelp.pyleus.serializer.MessagePackSerializer.readShellMsg(MessagePackSerializer.java:74) ~[stormjar.jar:na]
    at backtype.storm.utils.ShellProcess.readShellMsg(ShellProcess.java:99) ~[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
    at backtype.storm.task.ShellBolt$1.run(ShellBolt.java:116) ~[storm-core-0.9.3.2.2.0.0-2041.jar:0.9.3.2.2.0.0-2041]
    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]