search

pelias / spatial

ALPHA: geographic data service backed by spatialite
https://spatial.demo.geocode.earth
29 stars 6 forks source link

BETA testing Point-in-polygon #47

Open missinglink opened 4 years ago

missinglink commented 4 years ago

This issue is to track beta testing issues for the point-in-polygon system intended as a replacement for pelias/pip-service

awdng commented 3 years ago

Just wondering what the current status of the Spatial project as a replace for PIP is?

missinglink commented 3 years ago

Hi @awdng, I spent some time in https://github.com/pelias/spatial/pull/65, https://github.com/pelias/spatial/pull/67 & https://github.com/pelias/spatial/pull/74 making an endpoint which is 100% compatible with the old 'pip-service' implementation, AFAIK this is complete now, so (in theory) it should 'just work' as a replacement.

The initial performance testing showed that it's slower than the in-memory version (not really surprising), but not by a huge amount (which is promising), so considering that the startup time is instant and the memory requirements massively reduced, the assumption is that horizontally scaling the service would result in significantly better performance per $.

We've started generating and publishing copies of the whosonfirst data in 'spatial format' here https://geocode.earth/data/whosonfirst/ so people don't need to build them themselves.

Of course, beyond the reduced memory requirements and instant start/restart time the major advantage of the feature is that the PIP service becomes 'source agnostic' and so Pelias could (potentially) create custom hierarchies based on WOF, OSM, a mix of the two, or custom proprietary data.

Internally (at Geocode Earth) we aren't focussing on mixed/custom hierarchies, it's not on our immediate roadmap, so for that reason the final stages of the integration (testing, rolling out, perf testing, etc) haven't been completed.

We'd actually like to 'push it over the finish line' at some point but there are still some minor issues we'd need to address such as the necessity to download WOF data twice (once in WOF sqlite and once in Spatial sqlite format) which seems odd, along with the need to import the administrative data twice (once into Elasticsearch and once into Spatial, for the PIP), which isn't really a big deal but I know will trigger a bunch of community issues because it's not super intuative.

My estimation is that it would take a significant engineering effort to complete the transition from PIP Service to Spatial.

I'm based in Prenzlberg if you would like to grab a coffee some time and chat about it some more, you can find my email address on the team page of our website.