search

azavea / osmesa

OSMesa is an OpenStreetMap processing stack based on GeoTrellis and Apache Spark
Apache License 2.0
80 stars 26 forks source link

Generate faceted edit histogram tiles when 'facets is present #138

Closed mojodna closed 5 years ago

mojodna commented 5 years ago

'facets is a column containing Map[String, Int], or key -> value. FacetedEditHistogramTileCreator creates facets for different types of edits based on the combined set of tags present on nodes and the ways they're associated with. Way metadata-only changes are considered "minor versions" in this context, as the effective purpose of a node may change even if it doesn't change itself.

Node tags are merged with the tags of ways that include them. This enables a node's "purpose" (part of a road, part of a boundary, etc.) to be established. A side-effect of this is that "minor versions" of nodes are generated when related ways are modified (i.e. the name of a road changes, a node becomes part of a new way, potentially becoming an intersection).

The "visual" result of these minor versions will be that all nodes associated with a modified way will appear to be modified at the same time (because they're inadvertently changed), potentially suggesting that more drastic (geometric) changes have occurred. However, filtering out metadata-only changes should address this.

mojodna commented 5 years ago

I've managed to put together a one-shot updater using the augmented diff GeoJSON from overpass-diff-publisher (it required adding a nds property to generated way geometries in order to join them to/from nodes).

I put together a streaming version that runs, but something about the watermarking is preventing aggregations from working. Not the worst thing--the one-shot updater can run periodically to catch up.

mojodna commented 5 years ago

I spoke too soon; using current_date() as a watermark seems to be a bad idea; converting the sequence number to its corresponding timestamp appears to work properly.