mongodb / bson-rust

Encoding and decoding support for BSON in Rust
MIT License
400 stars 132 forks source link

RUST-1025 Async versions of `Document::from_reader` and `Document::to_writer` #304

Open WindSoilder opened 3 years ago

WindSoilder commented 3 years ago

Actually, I don't if this is a right way..

Let's say that I want to design a custom client/server, which uses bson to transfer data(like mongodb oplog or my custom defined data). But Document::from_reader and Document::to_writer is not async function, I can hardly to use it directly to send/receive data (or may cause the whole event loop hang).

As alternative, I need to read data into a vector<u8>, then invoke Document::from_reader to get actual document. But using this method, I need to parse the input stream(like read header first to know how many bytes we need to read at least).

So may be there are some relative apis to make async read/write directly from/into a Document is awesome.

But actually I don't know if serialize and deserialize document is a cpu bound task. If this is a cpu bound task, it's useless to implement these methods.

I have checked mongodb rust driver implementation, it seems that it use another abstract Message to make async interactive with mongodb. (I believe these data are also in bson format): https://github.com/mongodb/mongo-rust-driver/blob/master/src/cmap/conn/wire/message.rs#L24

May I ask what do you think about this?

patrickfreed commented 3 years ago

As a workaround today, you can use spawn_blocking (if on tokio):

let socket = std::net::TcpStream::connect("127.0.0.1:8080")?;
let doc = tokio::task::spawn_blocking(|| {                   
    Document::from_reader(socket)                            
}).await??;                                                  
println!("{}", doc);   

As for a long term solution, we'd ideally be able to introduce something like

impl Document {
    async fn from_stream<R: AsyncRead>(reader: R) -> Result<Self> { todo!() }
}

One challenge with this is there isn't an AsyncRead trait in the stdlib yet (see https://github.com/rust-lang/wg-async-foundations/issues/23). There are a few ones floating around in the ecosystem already, namely in futures, tokio, and async-std, though it's unclear if any of them are stable enough for us to rely on as a post 1.0 library. At any rate, I filed RUST-1025 to discuss this with the team.

Thank you for filing this issue!