search

soundscape-community / soundscape

An iOS application/service that aids navigation through spatialized audio
https://soundscape.services
MIT License
21 stars 22 forks source link

Overture Maps data #39

Open steinbro opened 1 year ago

steinbro commented 1 year ago

Overture Maps is a newer project that wouldn't have been available during original Soundscape's lifetime, certainly not when data sources were originally being evaluated. How do the features compare with OpenStreetMap?

https://overturemaps.org/

Instructions are provided for using DuckDB to query cloud-hosted Parquet files -- is that something that could be done directly from the app, such that we wouldn't need to run our own DB servers?

steinbro commented 1 year ago

I started a branch of overscape-server to explore using the S3-hosted Overture data files as a data source to be queried like a PostGIS database. I quickly came to the same conclusion as Simon Willison (which, incidentally, is a very helpful blog post):

I tried running a remote query that would return places within that boundary, but quickly got tired waiting for it to complete. I don't think this dataset is set up for fast spatial queries without fetching most if not all of the file.

So, while it's conceptually possible to query remote Parquet files directly via HTTP range requests, in practice this takes a few minutes per bounding box. That's not reasonable for the current request model of issuing a new query for every tile.

However, Overture still has potential as a data source that could be queried directly by the iOS app. In particular, it's fairly straightforward to use DuckDB to fetch all places within a bounding box directly into a SQL-queryable in-memory database. We could conceivably change up the data caching model to download larger regions less often, rather than individual tiles.