CLUSTER before CREATE INDEX improves import time

omniscale / imposm3

Imposm imports OpenStreetMap data into PostGIS

http://imposm.org/docs/imposm3/latest/

Apache License 2.0

710 stars 156 forks source link

CLUSTER before CREATE INDEX improves import time #211

Closed cquest closed 4 years ago

cquest commented 4 years ago

When write+optimize are done during import, it is better to CLUSTER the tables before CREATE INDEX. This avoids:

creating the index twice
additional storage for the index during CLUSTER

A quick test shows a clustering time divided by 2 on my setup on Paris region extract, and indexing time is identical. Total extract import is done in 2'40" instead of 3'45".

cquest commented 4 years ago

Measured benefit on a full planet import:

3h59 to CLUSTER osm_bduildings with existing index
0h44 without the index (only the geohash one)

nyurik commented 4 years ago

cc: @olt hi, this seems like a huge performance improvement to imposm - would it be possible to merge if so? Thank you for this amazing project!!!