pudo / dataset

Easy-to-use data handling for SQL data stores with support for implicit table creation, bulk loading, and transactions.
https://dataset.readthedocs.org/
MIT License
4.76k stars 297 forks source link

Added ChunkedUpdate to leverage `update_many` #315

Closed mynameisfiber closed 4 years ago

mynameisfiber commented 4 years ago

Depends on #314

The ChunkedInsert object is great for speeding up inserts. This commit does the same for updates. It chunks up updates then sends them to the table's update_many in bundles that contain updates to the same fields.

In doing this the ChunkedInsert and new ChunkedUpdate were refactored to inherit from a common _Chunker object that does most of the chunking logic. This should also make it simple to add a ChunkedUpsert object.

pudo commented 4 years ago

Very cool addition! :)