This PR updates the client to enable "zero copy" operation and introduces buffer pooling.
The zero copy mode of operation is based around a streaming line serialization trait in the new serialization.rs file. It leans heavily on serde_json internals, but had to duplicate some logic to provide a push based api which serde doesn't yet provide https://github.com/serde-rs/serde/issues/768 (Although the last comment on there is hopeful, and new since I hacked together this version -.-).
The buffer pooling is built on top of https://github.com/logdna/async-buf-pool-rs which is a async port of Connor's object pool crate adapted to use a channel rather than a Vec/Mutex as it's return queue.
The buffer types themselves are implemented in segmented_buffer.rs. They use the bytes crate's Bytes and BytesMut types for the segments (as Hyper expects Bytes objects for it's streaming body API) and they implement the standard Read, ReadBuf, AsyncRead, AsyncReadBuf, Write, AsyncWrite etc. They also implement bytes::Buf and hyper::HttpBody required by hyper.
TODO:
[ ] Split up big ball of mud commit
[ ] Tidy up serialize and segmented_buf files to group structs and impls
This PR updates the client to enable "zero copy" operation and introduces buffer pooling.
The zero copy mode of operation is based around a streaming line serialization trait in the new serialization.rs file. It leans heavily on serde_json internals, but had to duplicate some logic to provide a push based api which serde doesn't yet provide https://github.com/serde-rs/serde/issues/768 (Although the last comment on there is hopeful, and new since I hacked together this version -.-).
The buffer pooling is built on top of https://github.com/logdna/async-buf-pool-rs which is a async port of Connor's object pool crate adapted to use a channel rather than a Vec/Mutex as it's return queue.
The buffer types themselves are implemented in segmented_buffer.rs. They use the bytes crate's Bytes and BytesMut types for the segments (as Hyper expects Bytes objects for it's streaming body API) and they implement the standard Read, ReadBuf, AsyncRead, AsyncReadBuf, Write, AsyncWrite etc. They also implement bytes::Buf and hyper::HttpBody required by hyper.
TODO:
Out of scope for this PR, but needs done soon: