search

gmr / rabbitpy

A pure python, thread-safe, minimalistic and pythonic RabbitMQ client library
http://rabbitpy.readthedocs.org
BSD 3-Clause "New" or "Revised" License
242 stars 58 forks source link

Use Queue.get instead of time.sleep to delay AMQPFrame._wait_for_frame iterations. #27

Closed kristaps closed 10 years ago

kristaps commented 10 years ago

The example at https://gist.github.com/kristaps/10599120 can process ~10 msgs/s, due to the mandatory 0.1s sleep in _wait_on_frame, if there is no frame available on first iteration (happens on every message for me). I propose using the timeout param of the Queue.get call in AMQPQueue._read_from_queue, which will sleep only when there is no data and return immediately when data is available.

This change makes the example script's throughput rise to ~700 msgs/s on my machine.

gmr commented 10 years ago

How are you dealing with the fully async nature of state change if you move to fully blocking?

kristaps commented 10 years ago

Not sure I understood the question, could you elaborate a bit, please?

gmr commented 10 years ago

I'm refactoring the code a bit. RabbitMQ can send RPC messages to the client that should interrupt any existing behavior, such as Channel.Close, Basic.Return. Basic.Nack, etc. Moving to a fully blocking Queue.get() with the existing IO structure would break client behavior. I think I have a work around through.

gmr commented 10 years ago

Thanks for the patch, I've got a commit that changes how channels interact with the IO object which allows this behavior, so I'll be closing this with the spirit of your change commited.