SeaQL / sea-orm

🐚 An async & dynamic ORM for Rust
https://www.sea-ql.org/SeaORM/
Apache License 2.0
6.94k stars 483 forks source link

postgres: bulk insert API with UNNEST and ARRAYs #1922

Open gitmalong opened 11 months ago

gitmalong commented 11 months ago

See my comment here https://github.com/SeaQL/sea-orm/discussions/710#discussioncomment-7319840

tyt2y3 commented 11 months ago

I imagine we can have a bulk insert API that bypasses ActiveModel. Since UNNEST ARRAY is column-oriented. (It'd also be much easier than try to build on top of the existing Inserter)

Something like:

MyEntity::bulk_insert([
    (my_entity::Column::Id, &[1,2,3,4]),
    (my_entity::Column::Name, &["a","b","c","d"]),
])

That'd build:

INSERT INTO my_entity (id, name) VALUES UNNEST ARRAY($1, $2);
# where $1 is i32[1,2,3,4], $2 is text["a","b","c","d"]

It's still in brainstorming phase so suggestions are welcome.

gitmalong commented 11 months ago

From end users perspective I would still like to pass over my whole Vec<MyStruct> and let SeaORM take care of the "field to vec split" instead of setting each column on my own because it is easy to forget something etc. But I don't know about the internals and if we can first start with your suggested bulk_insert and then build something more ergonomic on top of it, it sounds like a good starting point.

tyt2y3 commented 11 months ago

Yes, that sounds great. We can may be add a fn into_bulk_insert<E: EntityTrait>(items: Vec<E::ActiveModel>) -> BulkInsert<E>