Batch document creation?

[wbrickner]

Hello,

I am wondering if there is a more efficient interface to create a large number of documents at once, all of the same type.

For N documents I can simply perform N .save() calls, and wait on all the futures in parallel, but I am wondering if there is something faster (send them all in one request) / more elegant available in Wither that I did not see.

Thank you.

[thedodd]

👋 @wbrickner

As of now, the answer is no. However, there is a short-term workaround, as well as a long-term solution.

• short-term: you could do Model::collection(&db).insert_many(...). Docs for that method. You could actually pass it a vec or other iterator of model instances, as it will serialize them for you.
• long-term: we should just introduce an associated function which mirrors the same functionality, except specific to the model type.

Would you be interested in opening a PR for that long-term solution? Should be quite minimal, all things considered.

[wbrickner]

Working on a PR.

I'm a little stuck right now.

I want to add a method like

/// Inserts all model instances as documents, provided in some (rust) collection of 
/// instances which can be iterated over (e.g. `Vec`, `HashSet`, etc).
///
/// Wraps the driver's `Collection.insert_many` method.
async fn insert_many<O, D, M>(db: &Database, documents: D, options: O) -> Result<InsertManyResult>
where
    D: IntoIterator<Item = Document> + Send,
    O: Into<Option<options::InsertManyOptions>> + Send,
{
    Ok(Self::collection(db).insert_many(documents, options).await?)
}

So that any collection the user happens to end up with is suitable for batch insertion directly, so long as the collection is iterable and the iterator items can be converted into documents.

I go on to provide an path for implicit conversion:

impl<M: Model> From<M> for Document {
  fn from(model: M) -> Document {
    model.document_from_instance()
  }
}

The type system is not happy with me:

type parameter `M` must be used as the type parameter for some local type (e.g., `MyStruct<M>`)
implementing a foreign trait is only possible if at least one of the types for which it is implemented is local
only traits defined in the current crate can be implemented for a type parameter

Is this the right way to handle an unknown iterator which yields some item /which can be converted to a document/? I think it would be really elegant to be able to not worry if you're batch inserting a hashmap or a btree or some deserialization container from some crate, whatever it is.

Hoping you can give me some advice, I've never found myself in this particular situation in Rust before.

[thedodd]

Yea, so the issue is that M is not used anywhere within the new method you've defined, and Rust doesn't accept that. If you just remove the M generic param, you should be good. Rust's way of communicating this is with the line:

type parameter M must be used as the type parameter for some local type (e.g., MyStruct<M>)

We shouldn't need that impl<M: Model> From<M> for Document { either. I don't think we have anywhere in the code where such would be needed, unless there is some other part of the PR which uses it.

That should help.