search

koordinates / kart

Distributed version-control for geospatial and tabular data
https://kartproject.org
Other
519 stars 41 forks source link

Progress output for Spatial Index creation #157

Open UberMouse opened 4 years ago

UberMouse commented 4 years ago

Creation of the Spatial Index can be quite lengthy for larger datasets. The LINZ NZ Property Titles for example took 70 seconds to create it. It would be helpful if some progress indication was output during this so better clone progress reporting could be created.

craigds commented 4 years ago

I know the ticket's about progress indication, but if we could speed it up significantly we might not need progress?

part of the slowness might be that the geometries we import don't include envelopes (since cbcb4c8bfef7f215e24769dbb9515324dc557ee2 )

so to build a spatial index we have to build OGR geometries and then ask OGR for an envelope for each feature, which is probably quite slow.

To fix that we could reverse that decision and start adding envelopes to our stored geometries. cc @rcoup

rcoup commented 4 years ago

To fix that we could reverse that decision and start adding envelopes to our stored geometries. cc @rcoup

isn't that in the v2 spec? Haven't decided on a format.

olsen232 commented 4 years ago

Just to clarify - is this the per-dataset spatial index created in the GPKG working copy?

There's (I think) another one that you can create at sno-repo/ice_seed.sno-idxd if you run sno query index but it's mostly not used