Closed ericjmcd closed 6 months ago
Thanks for reporting! recv_string
(and json, etc.) does have one additional stringing of futures together. The warning happens here, because the underlying recv timeout and the cancellation have both completed in the same tick. The fix for this more common situation is to add a f.exception()
to consume the error. However, this reveals that there is a risk of lost messages, because chained futures do not propagate immediately, they propagate on the next event loop tick. So the warning is actually telling us that something isn't hooked up right!
FWIW, in this example, you're setting RCVTIMEO to 1 millisecond and sleeping for 1 second. So there are hundreds zmq.Agains
that are caught, and one that tries to cancel and fails.
It triggers the issue more reliably for me if I set RCVTIMEO = 100
, time.sleep(0.1)
, where both the cancel and the Again timeout are set to occur a the same time.
I think losing messages like this is quite unlikely, but not impossible. Missing error messages is far more likely, and easy to fix.
Really, what should happen here is that a completed operation shouldn't be allowed to cancel. The problem is that chaining that pyzmq does via future.add_done_callback
actually takes one additional event-loop tick to propagate the cancellation, which means that you can try to cancel an action and it won't actually be cancelled, but you've given up waiting for it.
zmq.Again
, which can be safely ignored.
What pyzmq version?
25.1.2
What libzmq version?
4.3.4
Python version (and how it was installed)
python 3.12 via ppa:deadsnakes/ppa
OS
ubuntu 20.04 (inside docker running on ubuntu 20.04 host)
What happened?
When cancelling an asyncio task that is looping a pull socket recv_string() with RCVTIMEO, a zmq Future exception (zmq.error.Again: Resource temporarily unavailable) is never retrieved in spite of wrapping the calls in try/except blocks. This does not happen with a simple "recv()" and I haven't checked if it's an issue with any other variants (e.g. "recv_json") or any other socket types.
Code to reproduce bug
Traceback, if applicable
More info
No response