Open victorpoluceno opened 9 years ago
We never explicitly designed Pyleus to be compatible with PyPy, so I am afraid I don't have more insight about that. As far as I know there might even be other issues arising when trying to run Pyleus on PyPy.
Can it be a conversion problem of the message id number due to messagepack or json/simplejson used with PyPy? That number seems kinda weird to me. Did you try with both serializers?
Yes, you are right! It works perfectly with json as serialize. It seems the problem is with msgpack and PyPy.
Also, to add more information to the issue I got this another exception when running with msgpack as serialize:
2015-02-09 22:27:58,798 29432 MainProcess 140567940040480 MainThread ERROR pyleus.storm.component Exception in bolt.run
Traceback (most recent call last):
File "/var/storm/supervisor/stormdist/topology-production-13-1423520840/resources/pyleus_venv/site-packages/pyleus/storm/component.py", line 233, in run
self.run_component()
File "/var/storm/supervisor/stormdist/topology-production-13-1423520840/resources/pyleus_venv/site-packages/pyleus/storm/bolt.py", line 46, in run_component
self._process_tuple(tup)
File "/var/storm/supervisor/stormdist/topology-production-13-1423520840/resources/pyleus_venv/site-packages/pyleus/storm/bolt.py", line 39, in _process_tuple
return self.process_tuple(tup)
File "/var/storm/supervisor/stormdist/topology-production-13-1423520840/resources/jabba/processor/topology/commit.py", line 20, in process_tuple
self.process(tup)
File "/var/storm/supervisor/stormdist/topology-production-13-1423520840/resources/jabba/processor/topology/commit.py", line 32, in process
self.update(tup)
File "/var/storm/supervisor/stormdist/topology-production-13-1423520840/resources/jabba/processor/topology/commit.py", line 67, in update
self.ack(tup)
File "/var/storm/supervisor/stormdist/topology-production-13-1423520840/resources/pyleus_venv/site-packages/pyleus/storm/bolt.py", line 64, in ack
'id': tup.id,
File "/var/storm/supervisor/stormdist/topology-production-13-1423520840/resources/pyleus_venv/site-packages/pyleus/storm/component.py", line 321, in send_command
self._serializer.send_msg(command_dict)
File "/var/storm/supervisor/stormdist/topology-production-13-1423520840/resources/pyleus_venv/site-packages/pyleus/storm/serializers/msgpack_serializer.py", line 50, in send_msg
self._output_stream.flush()
IOError: [Errno 32] Broken pipe: '<fdopen>'
I've a topology that works perfectly with
CPython
, but in order to achieve better performance I'm testing pyleus withPyPy
and I've stumbled in a very strange bug. When running this topology withPyPy
after a whilepyleus
breaks with this error:As far as I can tell this is the only problem preventing
pyleus
to work withPyPy
. Does anyone usesPyPy
withPyleus
, or knows what cause this error?Worth noticing that this error goes way when I reduce the size of each tuple that goes through
Storm
.