Feature request: Downloading OSM data using API/Overpass

[bobbyxng]

Describe the feature you'd like to see

• Currently, PyPSA-Earth downloads entire country datasets from OSM (.pbf files) using the dedicated package earth-osm. While this the most robust approach, it can take a lot of time, especially if the region of focus increases to multiple countries or e.g. Europe.
• In addition, the extracted power feature layer is only about 1/20 of the .pbf file size. OSM provides an API to directly retrieve (small) amounts of data based on a fair use policy (https://overpass-turbo.eu/).
• Right now, I am implementing this feature in PyPSA-Eur, this could be later transferred to PyPSA-Earth and/or earth-osm as an alternative to downloading .pbf files.

[mnm-matin]

I think some years ago we started the prototype with overpass-turbo (this might actually be the 2nd or 3rd commit in this repo). Unfortunately, we ran into problems with rate limits.

Perhaps things have changed, so may the rate limits be on your side!

Alternatively, one could host the intermediate files produced by earth-osm, and download those instead of the .pbf (that's why it was desgined that way in the first place, but i never got around to completing the api part of it)