Yelp / pyleus

Pyleus is a Python framework for developing and launching Storm topologies.
Apache License 2.0
403 stars 113 forks source link

My bolt always fails due to org.msgpack.MessageTypeException: Expected map, but got integer value #154

Closed tellesnobrega closed 9 years ago

tellesnobrega commented 9 years ago

I'm trying to run a Kafka-spout example, very simple stuff. I want to send something via kafka and be able to process it in a bolt. I'm using kafka-python to send the messages, and at first i was using just the command line to send messages as needed, but the bolt always fails with this exception: org.msgpack.MessageTypeException: Expected map, but got integer value at org.msgpack.unpacker.Accept.acceptInteger(Accept.java:45) ~[storm_log_analyzer.jar:na] at org.msgpack.unpacker.MessagePackUnpacker.readOneWithoutStack(MessagePackUnpacker.java:91) ~[storm_log_analyzer.jar:na] at org.msgpack.unpacker.MessagePackUnpacker.readOne(MessagePackUnpacker.java:73) ~[storm_log_analyzer.jar:na] at org.msgpack.unpacker.MessagePackUnpacker.readMapBegin(MessagePackUnpacker.java:534) ~[storm_log_analyzer.jar:na] at org.msgpack.template.MapTemplate.read(MapTemplate.java:62) ~[storm_log_analyzer.jar:na] at org.msgpack.template.MapTemplate.read(MapTemplate.java:27) ~[storm_log_analyzer.jar:na] at org.msgpack.template.AbstractTemplate.read(AbstractTemplate.java:31) ~[storm_log_analyzer.jar:na] at org.msgpack.MessagePack.read(MessagePack.java:527) ~[storm_log_analyzer.jar:na] at org.msgpack.MessagePack.read(MessagePack.java:496) ~[storm_log_analyzer.jar:na] at com.yelp.pyleus.serializer.MessagePackSerializer.readMessage(MessagePackSerializer.java:205) ~[storm_log_analyzer.jar:na] at com.yelp.pyleus.serializer.MessagePackSerializer.readShellMsg(MessagePackSerializer.java:74) ~[storm_log_analyzer.jar:na] at backtype.storm.utils.ShellProcess.readShellMsg(ShellProcess.java:99) ~[storm-core-0.9.5.jar:0.9.5] at backtype.storm.task.ShellBolt$BoltReaderRunnable.run(ShellBolt.java:318) ~[storm-core-0.9.5.jar:0.9.5] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]

Can anyone help me on this? Am I missing something on the message or on the bolt? The bolt only reads the messages and sends it downstream (build your first topology example)

Thank you

poros commented 9 years ago

Could you try using the json serializer? This will rule out any problem with the msgpack serializer, if any.

tellesnobrega commented 9 years ago

I switched to json and it worked.

Thanks.

mnhbanagare commented 8 years ago

i was also facing same problem, now it solved. solution is i was printing data in process_tuple function. i removed the printing data and added the loger