thisisaaronland
commented
3 years ago Keeping in mind that this is all very much still work-in-progress, and the documentation remains spotty, you might have better luck with go-whosonfirst-spatial SQLite packages.
This demonstrates how the SQLite databases are created and queried:
And this demonstrates how those SQLite databases are queries over an HTTP interface, like the original go-whosonfirst-pip-v2 server:
The SQLite databases take less time to index than the original in-memory RTree implementation and can be created independently of the PIP server itself.
They also have better support for "extra" properties to be included with the default responses and support indexing alternate geometries. As of this writing there is not support for filtering alternate geometries either by type or name.
The original go-whosonfirst-pip-v2 RTree implementation was always constrained on indexing time due to the volume of files being indexed, I/O limits and the sequential nature of building the RTree index itself. There is an equivalent RTree implementation of the go-whosonfirst-spatial interfaces but that hasn't been updated to reflect the most recent developments yet:
There is nothing about this package that will address any of the problems around indexing time, though. For large volumes of data something like the SQLite package is probably a better alternative.
HIRANO-Satoshi
Thanks for README. I understand it's for structural reorganization.
go-whosonfirst-pip-v2 consumes days for Indexing. Is it possible to speed up indexing? hopefully ten or hundred times.
Another concern is memory usage. I'm not sure that memory thrashing is the main cause of the slow indexing or not. Of course we could use a larger machine, but we have some restriction.