Looks like we're doing some duplicate work

vkarpov15 commented 9 years ago

Hi @zorend, I actually just stumbled across your project. We're actually just starting on a rust 1.0 driver here at MongoDB: https://github.com/mongodbinc-interns/mongo-rust-driver-prototype/tree/1.0 . We're a bit behind you right now, but we'd love to figure out a way to merge the projects and work together going forward. What do you think?

oussama commented 9 years ago

Hi @vkarpov15, yeah that would be great, actually I learned a lot about the mongo wire protocol through that project, I'm currently using https://github.com/zonyitoo/bson-rs for bson encoding and decoding, the wire protocol messages structures, their encoding and decoding are working, and I've started working on the api, currently command, insert and few other things are working. I'm still deciding on how to handle the response, my initial plan is to delegate the connection to another thread, parse the response and send it through channels or use a callback function, but I'm in favor with the channels method since it will follow the rust model in using blocking methods when reading. I'm open to the suggestion so I want to hear your plan on how to proceed.

vkarpov15 commented 9 years ago

What's the status of bson-rs, how complete / stable is it? We were planning on just using FFI around https://github.com/mongodb/libbson.

I think our general idea is to focus on a single-threaded approach first until we've properly built out something that fulfills the entire MongoDB CRUD spec and then build out a similar connection pool system to our Java driver. Admittedly I'm no expert on Rust, but our drivers for similar languages (async java driver excluded) generally feel synchronous and let the developer worry about not blocking their main thread.

oussama commented 9 years ago

I didn't do an extensive check on bson-rs but it is working perfectly till now, I'm planning to write a pure rust driver so FFI is not an option for me. Also if you work with std.net. I mean the standard library io you can't write it single threaded since read is blocking unless you want to write a driver that sends requests & wait for response one at a time something I don't think is suitable. There is mio library that can handle async network but it is not supported yet on windows and even if they did, going with the standard library is better since I'll only need 2 threads and I don't think that will be an issue. The threading model will be completely hidden from the user who will only use a style builder api.

vkarpov15 commented 9 years ago

Yeah the general idea is that most of our drivers use blocking I/O and defer threading to the user. There's a fair bit of multithreading internally with supporting connection pooling and server monitoring, but that's not exposed to the user. Obvious exception if the Node driver, and .NET driver has moved to an async model as well, but I'm not entirely sure if rust devs prefer the sync or async model. However, 2 threads won't be sufficient:

You're going to need to monitor servers
MongoDB server blocks on a per-connection basis. In other words, 1 socket open to 1 MongoDB server can only write 1 op at a time, other ops just get queued up on the mongodb server side, so you're going to have to have multiple sockets open.

vkarpov15 commented 9 years ago

Heya,

So we've been putting in a lot of work on our end over the last 6 weeks or so and are getting to the stage where we have a mostly functioning driver as long as you don't care about replica sets or auth and we'll be mostly done with those soon. We also took your path re: using the bson-rs library, because we ran into a few limitations with our FFI approach.

I wanted to ask - would you be willing to hand over the 'mongodb' crates.io to us? Our driver unfortunately hasn't found its home on crates yet and that would be the logical place for the mongodb-developed mongodb driver.

Thanks, Valeri

oussama commented 9 years ago

Hi, sure, since I've been inactive on this project I think the logical thing is that you get it. Can you send me how can I give you the access to it.

vkarpov15 commented 9 years ago

If you can add me (vkarpov15) to the crate as an admin I should be able to take care of the rest.

oussama commented 9 years ago

I did. Good Job on the driver btw. I'll check it when I have time.

vkarpov15 commented 9 years ago

Thanks, much appreciated. We'll push a v0.1.0 within the next couple weeks. The credit belongs to @saghm and @kyeah though, they did the real work :)

oussama / mongodb-rs

Looks like we're doing some duplicate work #1