INWTlab / dbrequests

python package built for easy use of raw SQL within python and pandas projects
MIT License
5 stars 2 forks source link

Send data via LOAD DATA INFILE #15

Closed wahani closed 4 years ago

wahani commented 4 years ago

In https://github.com/inwt/dbtools we send data to a db, not via the driver, but instead via the LOAD DATA INFILE SQL statement. This is useful for large datasets and would fix some performance issues we currently experience. We tested these commands on MariaDB and MySQL. For other flavors we would need different commands. We would need the same methods of insertion (insert, replace, truncate, update) as currently supported by the package. This is probably a port of the features we have in dbtools, and will behave very similar.