Bulk insert changes for DataTable. Based on some testing this could …

saulgreenberg / TimelapseDeprecatedPre2.3

Timelapse Image Analysis Tool

Other

1 stars 0 forks source link

Bulk insert changes for DataTable. Based on some testing this could … #96

Closed micahbresette closed 4 years ago

micahbresette commented 4 years ago

Bulk insert changes for DataTable. Based on some testing this could improve database insert performance anywhere between %15 - %50

Performing a single bulk insert per batch of files.
Generating statement using a StringBuilder
Changed the insert commit rate to 500 (this would eventually be faster for distributed databases, but doesn't matter as much for SQLite on the same machine).

I can send the Testing file I used for the timings if needed.

micahbresette commented 4 years ago

Attached is the Program.txt (had to change the file extension to upload it here, it is a ".cs" file though Program.txt ) file that I used to run various testing. It is a .NET 4.5 Console app.