OpenHistoricalMap / issues

File your issues here, regardless of repo until we get all our repos squared away; we don't want to miss anything.
Creative Commons Zero v1.0 Universal
17 stars 1 forks source link

Static tiles for Leaflet for one year #839

Closed ch-sander closed 1 week ago

ch-sander commented 2 weeks ago

Is there a way to embed OHM tiles into Leaflet for one historical year (or one point in time) without integrating the slider or any other user control or GUI? Can the date filter be passed in the tile URL directly? E.g. https://vtiles.openhistoricalmap.org/maps/osm/{z}/{x}/{y}.pbf?start_decdate =1500?end_decdate =1501 ?

1ec5 commented 2 weeks ago

Unfortunately, there isn’t a way to directly use OHM tiles inside Leaflet, because we don’t publish a raster tileset: #532. Instead, the usual approach is to install MapLibre GL JS plus MapLibre GL Leaflet as a compatibility shim (or Mapbox GL JS plus Mapbox GL Leaflet if you also need to load a Mapbox-hosted style). Then you can filter the style by date. #840 tracks publishing a non-UI library so that you don’t have to copy-paste all that code.

If this is a map you’re creating from scratch and you only need to show the map without any extra features, you could embed OHM in an iframe (documentation). This embed has a better experience than Leaflet, with better performance and smooth zooming, because it uses MapLibre GL JS directly.

jeffreyameyer commented 2 weeks ago

@ch-sander - thanks for asking about this. Making it easier to extract and display OHM data on other websites is a top priority for us right now. Minh (@1ec5) points out some workarounds, but I'm curious about the use case you have in mind - is there a focused time and geo subject you're focused on, or is this a more general interest in a world map for a particular year or something different than either of those cases?

ch-sander commented 2 weeks ago

Thanks for the instructive reply and the interest.

It's quite concrete actually. We're working with historical data for dioceses, i.e. ecclesiastical geography.

We map the results of queries related to dioceses onto modern maps, which is awkward. We'd like to do better, and also use OHM for further API queries.

We use geopandas and folium, so we rely on leaflet.

jeffreyameyer commented 2 weeks ago

Very interesting! That dataset looks like something we should have in OHM. Please keep us posted on your progress or the results of your work (we have a forum for community discussion).

The geopandas -> folium pipeline is of interest, as well. I'll leave it to Minh to provide any insight on that front!

ch-sander commented 2 weeks ago

Thank you very much, we will.

I am not a GIS expert myself, and the diocesan data is not ours, but we'd be happy to help with the migration of GeoJSON to OHM. However, I don't know how OSM or OHM can deal with incommensurable political boundaries, which is certainly out of scope for the slider GUI.

We use these Python libraries because they allow us to visualize queries in maps very easily and dynamically. Without tiles, seamless integration is probably difficult. For some things we also use Bokeh which has no leaflet or mapbox compatibility at all, but has its own engine for maps.

1ec5 commented 2 weeks ago

We map the results of queries related to dioceses onto modern maps, which is awkward. We'd like to do better, and also use OHM for further API queries.

Given your use case, you might be interested in our integrations with the Overpass API and QLever (for SPARQL queries federated with OSM and Wikidata).

We use geopandas and folium, so we rely on leaflet.

Folium seems to have some support for MVT vector tilesets. You can plug in the tile URL template in this documentation. However, Folium has its own style format, so you’d need to author your own style instead of reusing one of the official OHM styles. This might be useful to you anyways if you’re only interested in some of our data layers. You’d also have the write the date filtering yourself; I don’t know if Folium supports any kind of filters, or only style attributes.

However, I don't know how OSM or OHM can deal with incommensurable political boundaries, which is certainly out of scope for the slider GUI.

I’m unfamiliar with the term incommensurable in the context of boundaries – is that like indeterminate boundaries or something else? (Pardon my lack of awareness on this subject.) Indeterminate boundaries are an open question for us in terms of rendering, but I think we could come up with a convention for tagging the data, with your advice.

ch-sander commented 2 weeks ago

incommensurable

Apologies for the jargon. I was hinting at situations where different sources or different historical actors disagree about where some boundary is to be placed, as well as different types of boundaries, such as ecclesiastical vs. secular boundaries. The latter case might just be two different layers.

Thanks for the link to the potential support for folium. That would be very useful and I assume something appreciated by the python community.

1ec5 commented 2 weeks ago

You’d also have the write the date filtering yourself; I don’t know if Folium supports any kind of filters, or only style attributes.

To filter features in Folium, you’ll need to pass in a function that does the filtering imperatively. Our vector tile schema isn’t well-documented, but you can filter on the start_date and end_date properties, which are in YYYY, YYYY-MM, or YYYY-MM-DD format, or the start_decdate and end_decdate properties, which are in “decimal year” format. Uncertain dates can be recorded in EDTF format, but they aren’t being exposed in the tiles yet: https://github.com/OpenHistoricalMap/issues/issues/15#issuecomment-1567454126. The dates are given according to the proleptic Gregorian calendar.

Apologies for the jargon. I was hinting at situations where different sources or different historical actors disagree about where some boundary is to be placed, as well as different types of boundaries, such as ecclesiastical vs. secular boundaries. The latter case might just be two different layers.

Ah, that makes sense. We do have a convention for tagging historical disputes, but unfortunately this information isn’t being exposed in the tiles yet due to some technical obstacles we’re currently working through in #800. As far as I know, we don’t have a convention for indicating discrepancies among sources.

We already have multiple kinds of boundaries, including some ecclesiastical boundaries (mostly Czech Jewish kahal, tagged as boundary=religious_administration). However, we aren’t exposing them in the tiles yet because there aren’t enough of them to render. Personally, I’d been hoping to find some resources to map historic North American diocesan boundaries, so I’m encouraged to see that there are similar resources for other regions of the world.

jeffreyameyer commented 2 weeks ago

@ch-sander - you can see those administrative boundaries in this turbo overpass query.

Most (all?) of those edits were created by Daniel Baránek, another academic, who I believe has used his work in OHM as part of a paper or two. You may want to reach out to him to see what has worked and what hasn't (and let us know what you learn!).

ch-sander commented 2 weeks ago

Great input @jeffreyameyer @1ec5 !

I will (hopefully) go that extra mile and make OHM folium-ready!

Another question, maybe for another issue (?): Something like boundary=religious_administration, how do you ingest that in OHM? Can you simply use GeoJSON to have it as a tag and then being used in queries (even though you're not exposing it to tiles, which I understand)? Of the API could even return GeoJSON, even better.

There is so much scattered GeoJSON out there from various projects that might look for a data hub such as OHM.

1ec5 commented 1 week ago

Another question, maybe for another issue (?): Something like boundary=religious_administration, how do you ingest that in OHM? Can you simply use GeoJSON to have it as a tag and then being used in queries (even though you're not exposing it to tiles, which I understand)? Of the API could even return GeoJSON, even better.

OHM is kind of like a data hub, in that we try to make it easy to get data in and out. However, it’s different in that we don’t separate datasets or layers – all the data commingles in a single interconnected layer. This makes it easy to draw explicit connections between thematically unrelated features, such as boundaries and the waterways they follow, but it also requires us to be deliberate about conflating the data and making it all coexist reasonably well.

There are lots of tools for importing data into OHM (because it’s so similar to OpenStreetMap). The JOSM editor can load a GeoJSON file or shapefile of data to upload to OHM.

Once the data is in OHM, you can use Overpass or QLever to query for it and export the results in a few different formats. For example, this query returns all the boundary=religious_administration relations anywhere in the world. If you run this query in Overpass turbo, you can export the result as GeoJSON. There’s also an underlying Overpass API, which has clients for Python and other languages. To learn how to craft an Overpass query, check out the Overpass Tutorial or this introductory talk I gave a few years ago.