HTTP/1.1 client draft

[liormizr]

This is a preliminary draft for HTTP/1.1 support.

I have a few questions in mind:

• Do we allow external dependencies in curio? I've implemented an HTTP/1.1 client using h11 (https://github.com/njsmith/h11) and I'm unsure if this is the right approach for the project or if you'd prefer to implement the protocol as part of curio.

• How would you prefer to handle pull requests? Would you prefer a single pull-request containing the HTTP client, server & documentation or should I split it based on content?

[dabeaz]

In the big picture, I think I'd like to see Curio stay "small" as a kind of core library for handling the low-level concurrency and I/O primitives as opposed to becoming an all-encompassing framework for everything. For higher level protocols, it might make sense to spin each thing into its own separate library or project. I currently own the "python-curio" organization on Github. There's nothing there at this time, but it could be expanded to point to the different projects.

I'm open to ideas though. Maybe some form of HTTP is something that could be handled in the core library. What do people think?

[njsmith]

You might also be interested in "asks":

• http://asks.readthedocs.io/en/latest/
• https://github.com/theelous3/asks

[jkbbwr]

@dabeaz Im still working hard when I can free the time on a http framework in the flask style. I think its one of the biggest things that stopping mass curio adoption. Perhaps you should "bless" a list of libraries that do common things and have a list of awesome-curio.

[liormizr]

I think curio should have base implementations for HTTP protocols.

@dabeaz A plugin mechanism sound like a great idea. what should you be the design approach for it from your perspective?

I was thinking of having a set of basic plug-ins installed as part of the curio package so that curio code keeps on staying clean.

Like so:

from curio.plugins import http2
import curio

if __name__ == '__main__':
    curio.run(http2.server, ...)

What do you guys think of it?

@njsmith Yes. there are other HTTP implementations for curio. I created this one since there was no curio base implementation for them and all of them were incomplete implementations.

I was thinking of a library that has client and server base implementations for HTTP/1.1 and HTTP/2.

@njsmith I'd be more than happy to join forces with you on this one.

We can merge the h11 project to this plugin.

Maybe we could also offer the writers of "asks" or "curio-http" to join the project?

Coming to think of it, we need two separate plugins: curio HTTP/1.1 plugin curio HTTP/2 plugin

What do you guys think?

@dabeaz How can I kick-start this plugins project? can we create the project in "python-curio" organization?

[dabeaz]

I've been thinking about HTTP a bit. All things considered, I think I'd prefer the "curio" project itself to be focused more on the core task execution model than to be an all-inclusive framework. This would make it more like asyncio (as opposed to something more expansive like Twisted). For example, asyncio doesn't include any kind of HTTP library--that's provided by other third-party modules such as aiohttp. I see Curio in a similar model.

On plugin-systems: implementing things like that are often a tricky affair and can introduce considerable complexity to the whole module/package/packaging infrastructure of curio. I'm not sure there's much to be gained by having everything live under the top level 'curio' namespace. For example, there's no plugin system for asyncio, but I'm not sure if it's worse-off for not having it. We could always reconsider later, but for now I'd probably stay away from doing that.

Big picture: I am not personally working on an HTTP implementation for curio nor do I have plans to write one (I'd much rather use one written by someone else). However, if someone is writing one, I want to make sure that people can find it somehow. So, there should be a documentation effort to link to such projects. Maybe grouping such projects into a "python-curio" organization of some kind. At a minimum, making sure such projects make it to the Curio README file perhaps.

[jkbbwr]

Why not have blessed libraries under the curio organisation? Then you can have a kind of curio stdlib.

[adontz]

I am not here to say your code is bad or something, but is not performance kind of important and httptools way to go? Did you consider it?

[gigatexal]

@dabeaz you’re the BDFL here. Been watching the progress of curio since your presentation on it. You’ve said many times you don’t want it to grow in focus and that HTTP is out of scope. I think people should quit fighting you on it and accept that or fork the project and add HTTP if they like. Such discussions of adding things that have been decided a long time back hold the project back no?

[dabeaz]

The comments on this thread are more than a year old, but I don't really see Curio evolving into an overarching framework of some kind with plugins and such. It's basically a stand-alone library that addresses a specific problem. Things like HTTP should be supported in separate projects.

[imrn]

We already have an such an inhouse http lib. I can assure you that there is no need for inclusion of such a thing in curio or creating a fork for it. Intersection of that code and curio is an absolute none.

[imrn]

May be we can consider having a documentation section for properly defining curio's position with regard to such http inqueries. Besides, it would be highly favorable to include async http resources as pointers (even with some brief examples) since http is a very common area for asyncs. I'd suggest keeping this issue open until having such a documentation section.

[dabeaz]

I agree that it might make sense to keep this open. I still think there could be an interesting HTTP future for Curio. It's just a little unfortunate that everything got so quickly fragmented and muddled early in Curio's existence. At this point, it's not clear to me if the whole Sans-IO movement still has any traction or not. At a minimum, perhaps some kind of better interoperability with asyncio is a way forward. I still think we could have our own neat thing though.

That said, the main thing that everyone should know is that I'm not personally working on HTTP support for Curio, nor is it on my short-list of projects to think about. I say that mainly as words of encouragement for anyone who DOES decide they want to work on this. I'd want to encourage your progress (it's definitely not in competition with anything that I'm also working on or planning to work on).