storborg / manhattan

A gentleman's cocktail, for civilized analytics and testing
Other
2 stars 3 forks source link

Use bulk insert/update operations in SQL persistent store #15

Open storborg opened 12 years ago

storborg commented 12 years ago

Currently an update and possibly an insert is issued for every new record persisted in the SQL store. There's probably a performance gain to be had from doing an INSERT ... ON DUPLICATE KEY UPDATE (or for sqlite, INSERT ... ON CONFLICT REPLACE).

There are some more tips about making it sqlalchemy-idiomatic here: http://www.sqlalchemy.org/trac/ticket/960