search

dabeaz / curio

Good Curio!
Other
4.04k stars 243 forks source link

[non-issue] Thanks for work on ZeroMQ! #181

Closed tcwalther closed 7 years ago

tcwalther commented 7 years ago

This is not an issue, but just a thank-you note. I just noticed that @dabeaz , you committed https://github.com/dabeaz/curio/blob/master/curio/zmq.py 4 days ago, and I can't even begin to express how excited I am about this. Unfortunately, my own lack of understanding sockets doesn't make me confident enough in writing this myself, and so I just wanted to say a big, massive thanks for working on it. This is fantastic!

jkbbwr commented 7 years ago

Yep. We have a dirty hack to get zmq to work at the moment this is going to replace that!

dabeaz commented 7 years ago

I'd consider the Curio support to be preliminary. Needs more testing. Still thinking about certain facets of the API. For example, to be consistent with the rest of Curio, should methods like connect(), bind(), close() and such be defined as async? Should the sockets support the async-context management protocol?

jkbbwr commented 7 years ago

Is there any penalty to normal curio in using the ZMQSelector?

dabeaz commented 7 years ago

I haven't done a huge amount of benchmark testing on it, but I tried it just now. The penalty appears to be small if I try it with a simple echo server (on normal sockets). Using normal Curio, I get about 29500 request/sec. Using the ZMQSelector, it's about 27500 request/sec. This is on a Mac.

brifordwylie commented 7 years ago

Big fan of both ZMQ and Curio. @dabeaz can you elaborate a bit on your 'vision' for the pairing going forward, with perhaps a concrete example or two?

dabeaz commented 7 years ago

Ah the vision. That is a really good question ;-). I think in the big picture, Curio is evolving into more of a library for Python concurrency generally. Yes, coroutines are at the center of it, but there are a variety of other interesting things happening with threads, processes, cancellation, timeouts, and more. ZMQ has a variety of interesting communication patterns. Coupling those two things together seems it could be a really interesting thing to play around with.

There is a part of me that thinks it might be interesting to have a pure-async implementation of ZMQ in Curio that is independent of the ZMQ C++ implementation. There are certain challenges in doing that, but it might be a fun side project.

Actually, "fun" might be the main vision of the whole thing if there was one. Is it possible to have a concurrency environment that's fun? Yes. Maybe.

brifordwylie commented 7 years ago

So part of the reason I'm asking is I have this toy project https://github.com/SuperCowPowers/chains that processes network streams. I'm thinking about porting it to 3.6/curio and I'm trying to wrap my brain around the new ZMQ 'horn' you've added here and what fun possibilities that may add. Shrug.. my brain is full... I'll let you know how it goes.