Closed carllerche closed 5 years ago
So, I would ask the community there is any interest in a simplified
tokio-proto
yes :) I already asked for some clarification: https://github.com/tokio-rs/tokio-proto/issues/202
and if there are any volunteers willing to take on some of the burden in developing & maintaining the crate.
I won't be able to spend much time but I could give some feedback as a library developer.
Although I tried very hard to use tokio-proto
for tokio-imap
, it didn't work out. I now believe tokio-proto
's main value is probably in optimizing simple request/response flows, but I feel that it probably makes more sense focusing on tokio
, tokio-io
and other parts of the futures
stack for now.
@djc Thanks for the insights. This leads to my thoughts to totally deprecate tokio-proto
. It might be better to avoid it completely than to use a non-maintained unfinished crate.
[It's been suggested on Gitter that I open an issue similar to this one; I saw the one mentioned by @flosse and intended to respond there, but this is an even better place.]
I concur with @flosse that an heir to tokio-proto
should exist. I've used tokio-proto
pretty extensively, choosing to work around its limitations instead of replacing it with custom code, and IME it worked rather well. The chief pain points I've noted were, in no particular order:
It's a maze of twisty little traits, all similar (but with some baffling differences, e.g., error types in simple vs streaming variants). Small wonder, since it aims to support three independent axes: client/server, each pipelined or multiplexed, with simple and streaming flavors on top of that. That's eight scenarios, and it can be very difficult to map your problem to this space.
Some types of errors would be swallowed by the exchange machinery, leaving the user with a generic "Broken pipe", necessitating the use of tracing to find out what really happened.
Trying to accomplish things not directly provided by the trait facades, like explicitly closing the underlying I/O source, would lead to loss of genericity and, effectively, manual monomorphization, with an explosion of types, impls and boilerplate.
That may sound quite critical, but I'll reiterate that my overall experience with the crate was quite good. I especially liked the ease of slotting in a completely separate type of transport (Unix domain sockets) in a client when the need arose, as well as the fairly modest set of changes required for implementing things like StartTLS, after some type wrestling.
I think that the division of protocols into pipelined and multiplexed is sound and essential, but that streaming variants are adding more complexity than necessary, and that the crate would be majorly simplified by ditching them. Furthermore, client and server roles could be separated into their own crates, leaving the users a much clearer choice and a lower cognitive burden.
I intend to analyze tokio-proto
's reverse dependencies too see how disruptive the jettisoning of streaming would prove, fork the codebase to implement the simplifications outlined above, and try to provide some enhancements, like accomodating almost-request/response protocols, where some messages are unidirectional while the majority are not. The pace of activity is not something I can make promises about, but I hope that my work during the year can lead to simplified and revamped tokio-proto
which will be useful to existing and new Tokio users.
@inejge Great to hear.
I do think that a simplified tokio-proto has value. The primary issue is that this is pretty close to a rewrite and my time is currently pretty limited.
If you have availability to take on the work, I can provide guidance.
It's interesting: there are 46 crates that depend on tokio-proto
but there are only 4 developers who are concerned about the future of this crate. I wonder if it makes sense to invite the other authors to this discussion? What do you think?
I've used tokio-proto because it's the fastest way to get off the ground, and I trusted @carllerche to write a better and more performant server implementation than I would with the time I have. If it didn't exist, I'd probably just settle for a simpler, less performant server implementation.
I'm in the process of starting a project that is using tokio to implement a simple line based protocol over serial lines, and using tokio-proto
has really saved me a lot of time in structuring the code in a request/response model. So I'm very interested to see it continue in some form, and a simplified verion as described by @inejge sounds pretty ideal to me.
I would be happy to help with that effort, though I can't commit a ton of time to it.
Ok, I will try to put together a more organized group to attempt to take over -proto.
TL;DR: take tokio-proto split it into a non-tokio specific service crate and a crate for impl protocols and focus this on tools helping you implement on aspects each instead of providing a frameworkish interface like tokio-proto currently does.
My experience with tokio-proto was:
*this is my subjective experience when playing around with tokio and e.g. implementing smtp
I cam to the believe that instead of having tokio-proto it might be better to have two independent thinks:
@dathinab I like your thoughts :)
It would be really helpful for me to see some examples to learn from. Maybe we could just collect experiences from library authors with some descriptions how they solved their problems or how they migrated form tokio-proto
to a custom implementation.
you can take a look at new-tokio-smtp
Through it is might not be the best example here.
(It's not published as I'm still in correspondence with the tokio-smtp
crate owner
hoping to publish it as tokio-smtp).
Some aspects of it:
Connection
type)
and one for extending it i.e. defining other SMTP commands which either are
currently not provided or which alternate versions of provided ones which do
more deeply integrate with a library building on top of it. (This api view is exposed
through the Io
type)send_mail
feature there is a thin
slightly higher level abstraction layer available, and the opening of connections
is handled including integration possibilities for authentication.
The reason for this is that there are just to many ways how to handle connection
failure and retry just for the tcp/tls connections and even more so if you include thinks
like mail servers being temporary unavailable or temporary rejecting new mails,
which has to be handle for reliable mail delivery.&mut
would do, too. But borrows don't play so nice with futures, at
last until co-routines/async with self-borrows lands.NoConnection
, ConnectionInUse(impl Future)
, OpeningConnection(impl Future)
, ClosingConnection(impl Future)
and a buffer.Tools I would like to have had:
BytesMut
splitting of the beginning of the buffer without a copy normally.new-tokio-smtp
for now)new-tokio-smtp
)new-tokio-smtp
and handling thinks like connection retrying based on e.g. an configurable retry strategy (maybe with a bid of glue code pre-provided by the smtp crate for convenience).It looks like the new tokio-codec might be the base for a tokio-proto replacement?
I did start my project with tokio-proto, but since I swapped it out for new tokio without proto, life got more joyful again in Samotop.
I'm in the process of moving tarpc off of tokio-proto. In lieu of tokio-proto, I wrote a simplified, non-streaming, request-response framework. It all makes use of unstable nightly futures and async/await, so it's not really a stable replacement for tokio-proto. But if it sounds like something that interests any of you, take a look here (I probably won't merge it for a while, as it will break all current tarpc users): https://github.com/google/tarpc/pull/199/files
The future of tokio-proto is tokio–tower
, which is being championed by @jonhoo: https://github.com/tower-rs/tokio-tower
The tokio-proto crate was part of the initial release of Tokio. At that point, Tokio was focused on providing a higher level request / response based API and the I/O reactor was an implementation detail.
Today, focus has shifted and Tokio is now primarily a non-blocking I/O library and the request / response aspect is being shifted to Tower (which has not yet been released).
One problem that tokio-proto faces was that it was trying to be a one size fits all solution to binding
Service
to a socket. It tries to provide many capabilities at the cost of significant complexity. Even with all these features, libraries like h2 have opted to not use it.That said, the idea of
tokio-proto
still has value in providing an easy way to bind a socket to a service. Ideally, the next iteration oftokio-proto
would be significantly simplified to assist with getting something working fast with the assumption that, if advanced features are required, one would not use the library.Now, resources to maintain development of tokio-proto are currently limited. So, I would ask the community there is any interest in a simplified
tokio-proto
and if there are any volunteers willing to take on some of the burden in developing & maintaining the crate.