Closed cldellow closed 8 months ago
Good spot! Usual wisdom is to create the index at the end, but because we run all the inserts in one transaction, it should be fine to do it at the start.
Yeah, I think it depends what you want to optimize for. I would expect an index created at the end to have no fragmentation, to use less CPU time to build, and to be slightly smaller on disk.
But at the end, all of our cores are idle. So even though it takes less CPU time, it will take more wall clock time vs creating it at the start, and then amortizing the cost over the run of the program while other cores are still doing useful work.
Before:
After:
This reduces the mbtiles file by 0.5% and processing time by perhaps 0.3%, although it's quite noisy.
Creating the index at the start, rather than at the end, seemed to be fastest.