Add bulk_ops.py, which exposes a bulk_upsert function for efficiently upserting large amounts of data into the database
Context for reviewers
Projects frequently need to read in large amounts of data into the database from external sources such as CSV files. This utility provides a flexible way of doing so efficiently.
I'm not familiar with the platform's approach to abstracting away the underlying database -- the code here is Postgres specific and uses the psycopg library. Feedback on how to adapt the code here to the platform's approach is welcome/appreciated.
Ticket
n/a
Changes
bulk_ops.py, which exposes abulk_upsertfunction for efficiently upserting large amounts of data into the databaseContext for reviewers
psycopglibrary. Feedback on how to adapt the code here to the platform's approach is welcome/appreciated.Testing
make test args="tests/src/db/test_bulk_ops.py"