search

protomaps / OSMExpress

Fast database file format for OpenStreetMap
BSD 2-Clause "Simplified" License
229 stars 19 forks source link

Using an .osmx file with other languages #4

Closed bdon closed 8 months ago

bdon commented 4 years ago

This is the most important step to making this library widely useful. I envision a few ways this can happen:

  1. Other languages can call the osmx program as a subprocess, which returns JSON. This is OK for very basic use cases, but hard to extend to queries other than a single OSM entity. Also, the main reason to use this library is speed, but having to fork/exec on every query would negate that.
  2. use a library like pybind11 to provide bindings to C++ functions. I think this isn't realistic for languages other than Python, but can be convinced otherwise.
  3. Use other language's wrappers for LMDB and Cap'n Proto, and write libraries for each language for common query patterns.
    • pro: piggyback off packaging work in other languages for those wrappers
    • con: each language to be maintained separately, possibly weird conflicts around capnp versions
  4. Expose a C API - all languages should be able to wrap this in a neat way.
    • language bindings for each language are simpler
    • end user needs to get osmx library binaries somehow
bdon commented 4 years ago

Option 3 above was quite simple to implement in Python for node/ways/relations lookup. Spatial lookups are more complex and not supported yet, and would rely on the Python S2 package.

Other languages still TBD.

bdon commented 4 years ago

Immediate next steps:

brawer commented 1 year ago

For adding spatial queries to the Python client, s2sphere might be interesting? Pure Python implementation of S2 coverings (see online demo). This would allow queries by Cap and LatLngRect. No polygons though. (Edit: See https://github.com/protomaps/OSMExpress/issues/22 for Python, sorry for not seeing it before).

bdon commented 8 months ago

For now I'm going to suggest https://docs.geodesk.com/python as an alternative for polyglot analytical queries, as that design seems more suited. This project remains focused on minutely PBF out of a planet db using the CLI interface or IPC (option 1)