Each individual insert uses a transaction, which causes a lot of overhead.
By batching each table insert into one big insert, we can speed up the update time, particularly for large updates. It also eliminates any risk of an out of order update borking the table locally if one insert fails, although the data ordering is supposed to be oldest to newest tstamp.
According to: https://www.sqlite.org/faq.html#q19
Each individual insert uses a transaction, which causes a lot of overhead.
By batching each table insert into one big insert, we can speed up the update time, particularly for large updates. It also eliminates any risk of an out of order update borking the table locally if one insert fails, although the data ordering is supposed to be oldest to newest tstamp.