Closed tcwalther closed 7 years ago
Yep. We have a dirty hack to get zmq to work at the moment this is going to replace that!
I'd consider the Curio support to be preliminary. Needs more testing. Still thinking about certain facets of the API. For example, to be consistent with the rest of Curio, should methods like connect()
, bind()
, close()
and such be defined as async
? Should the sockets support the async-context management protocol?
Is there any penalty to normal curio in using the ZMQSelector?
I haven't done a huge amount of benchmark testing on it, but I tried it just now. The penalty appears to be small if I try it with a simple echo server (on normal sockets). Using normal Curio, I get about 29500 request/sec. Using the ZMQSelector, it's about 27500 request/sec. This is on a Mac.
Big fan of both ZMQ and Curio. @dabeaz can you elaborate a bit on your 'vision' for the pairing going forward, with perhaps a concrete example or two?
Ah the vision. That is a really good question ;-). I think in the big picture, Curio is evolving into more of a library for Python concurrency generally. Yes, coroutines are at the center of it, but there are a variety of other interesting things happening with threads, processes, cancellation, timeouts, and more. ZMQ has a variety of interesting communication patterns. Coupling those two things together seems it could be a really interesting thing to play around with.
There is a part of me that thinks it might be interesting to have a pure-async implementation of ZMQ in Curio that is independent of the ZMQ C++ implementation. There are certain challenges in doing that, but it might be a fun side project.
Actually, "fun" might be the main vision of the whole thing if there was one. Is it possible to have a concurrency environment that's fun? Yes. Maybe.
So part of the reason I'm asking is I have this toy project https://github.com/SuperCowPowers/chains that processes network streams. I'm thinking about porting it to 3.6/curio and I'm trying to wrap my brain around the new ZMQ 'horn' you've added here and what fun possibilities that may add. Shrug.. my brain is full... I'll let you know how it goes.
This is not an issue, but just a thank-you note. I just noticed that @dabeaz , you committed https://github.com/dabeaz/curio/blob/master/curio/zmq.py 4 days ago, and I can't even begin to express how excited I am about this. Unfortunately, my own lack of understanding sockets doesn't make me confident enough in writing this myself, and so I just wanted to say a big, massive thanks for working on it. This is fantastic!