How to use runtime with a framed + codec style transformation

[dignifiedquire]

I am trying out runtime, but having a hard time figuring out how to translate an existing tokio codec, to use runtime, without having to fallback onto a Futures01 compat layer.

Before I had a TLS Stream wrapped into a framed tokio codec in my Connection struct, which was a line based codec, meaning I have a parser that transforms each line of the incoming data into a parsed struct, and vice versa when writing.

Most solutions I found so far are effectively reimplementing the Framed construction (https://github.com/tokio-rs/tokio/blob/master/tokio-io/src/framed_read.rs) which I was hoping could be avoided. Would love to hear any ideas.

[yoshuawuyts]

@dignifiedquire hey hey! -- we've been getting this question regularly, so I figured it might be useful to write some thoughts down on streams.

First draft of a blog post is here: https://github.com/yoshuawuyts/blog/pull/21, it talks about alternative approaches to codecs, and I hope it comes in useful. The post itself is not quite ready for release yet, so I'd appreciate it if it wasn't shared publicly (:

If you have a hard requirement on using the codec architecture, https://docs.rs/futures_codec/0.2.2/futures_codec/ might come in useful for std futures. But as per my post, I think there are more preferable approaches.

Hope this was helpful!

[Nemo157]

Most solutions I found so far are effectively reimplementing the Framed construction which I was hoping could be avoided.

Do you have some details on why you would like to avoid this? Once I understood how Framed and the codecs inter-related I found it to be a very straight-forward way to do the conversion between a raw byte stream and a stream of items.

[dignifiedquire]

Do you have some details on why you would like to avoid this?

If I have to reimplement it all over again then the abstraction doesn't change. I knew that @yoshuawuyts had an alternative proposal, and was trying to figure it out by myself 👍

@yoshuawuyts thanks a bunch, still experimenting, will report back once I have sth working :)

[dignifiedquire]

• futures_codec is great for now to port existing codecs into the new world
• for the new pipe approach more methods to make it easy to split a stream of bytes are needed to make it easy to work with as far as I can tell so far

[yoshuawuyts]

for the new pipe approach more methods to make it easy to split a stream of bytes are needed to make it easy to work with as far as I can tell so far

Very true! I think in particular passing Stream<Item = Vec<u8>> into AsyncWrite is rather unintuitive currently, and we would benefit from having a convenient way of doing so.

boilerplate example

// current
stream.map(io::Result::Ok).into_async_read().copy_into(writer)

// possible improvement
stream.copy_into(writer)