Open n1epan opened 9 years ago
Can you try to run it locally and to debug the issue? I run a topology with some tuples in the order of the MB some time ago and it seemed to work fine (with some Storm tweaks).
The issues is because the msgs are too long for msgpack to handle. After I switch to json, it worked.
Ah, so this is actually a bug/limitation of our msgpack serializer. Good to know, thanks
I have a bolt A that continuously emits a namedtuple that contains a list that could have thousands of ids in it.
Fields = namedtuple("Fields", "table action ids")
, ids is a list that contains thousands of items in it.The bolt B in the downstream have the error below. My guess is that the namedtuple is overflowing the sys.stdin. Is this possible? What I should do in this case ?
I further checked what it is returned in read_tuple() to cmd. It is 56.
03/31/2015 03:55:13 PM - pyleus.storm.component - read_tuple - INFO: 56