azavea / osmesa

OSMesa is an OpenStreetMap processing stack based on GeoTrellis and Apache Spark
Apache License 2.0
80 stars 26 forks source link

Generation of MVTs with edit histograms #110

Closed mojodna closed 5 years ago

mojodna commented 5 years ago

This produces a tile layer suitable for displaying "recency" (as well as editing activity over time) by gridding node edits and aggregating counts by month. With the relatively low resolution (128𝗑128) of the tiles, the granularity could probably be dropped down to weekly without size problems (Detroit only takes up 1.2MB, ~10% of what it was at 512𝗑512).

Sample Output

Click through for interactive versions.

image

osm edits by month

Remaining

moradology commented 5 years ago

This is a pretty dang satisfying visualization and surprisingly small. What's more, if binning and counting works this well and pyramiding up is just a matter of doing a bit of addition it seems to me the performance should be more-or-less scale invariant.

drew-bower commented 5 years ago

@mojodna a cartographic question on the OSM edits by month viz: are the legends normalized month to month?

mojodna commented 5 years ago

@drew-bower If I understand what you're asking, yes. The color indicates the most recent time period (month, in this case) in which a cell was edited.

Counts are ignored entirely for this one, but the data does support a visualization like https://www.townsendjennings.com/sotmus-analysis/#10.02/42.3408/-83.0648 (specifically the brushable histogram to filter + show editing frequency over time within the viewport). (The limitation there is the level of granularity, which we should be able to improve, especially if we declare that we don't care about edits older than ___.)

mojodna commented 5 years ago

@drew-bower ah, I was looking at the wrong visualization. Yes, I believe "density" per month uses a static scale (it may depend slightly on Mapbox GL's heatmap implementation though).

drew-bower commented 5 years ago

So dark red represents the same volume between observations as opposed to a quantile for each dataset?

To be clear the capability is awesome, just asking for clarification on the cartography.

Nice work.

mojodna commented 5 years ago

It's close. We may want to be a bit more scientific about it, since raw values are normalized (in the same way for all months: by 0.01 - 0.05% depending on zoom) and mapped to the heatmap color scale (which is 0-1). I'm not confident that the normalized values stay in the 0-1 range though (Mapbox GL may do that, but I'm not sure).

Once I add a histogram to the view we'll be able to better visualize that (and it'll build more confidence for viewers).

Thanks!

drew-bower commented 5 years ago

Great, I know a lot of effort has gone into the internal guts of this, now comes the fun part.

rossbernet commented 5 years ago

This is great @mojodna, and I am really impressed with the responsiveness and speed of observableHQ.

mojodna commented 5 years ago

This produces invalid tile coordinates such as 0/0/-5, 0/0/1, etc.