azavea / osmesa

OSMesa is an OpenStreetMap processing stack based on GeoTrellis and Apache Spark
Apache License 2.0
79 stars 26 forks source link

Sample Analytic Vector Tile output #29

Open echeipesh opened 6 years ago

echeipesh commented 6 years ago

Connects: https://github.com/azavea/osmesa/issues/28

We should generate sample vector tile output to get early integrations testing and iteration going.

Use: s3://osm-pds-tmp/ that were generated by Seth.

Expected Output:

Some geometries may not represented in the layer and modified tag may need to be mocked. I don't believe it can be correctly generated with current vectorpipe code

mojodna commented 6 years ago

FYI, I've used the basis of #27 to produce historical geometries for the whole world. Last time took ~40 minutes on a 256 node cluster of m3.2xlarges. (updated is the equivalent of modified, I think)

Vector tiles are another can of worms; the global set of geometries contains some that result in the clipToGrid stage running indefinitely.

echeipesh commented 6 years ago

I see you already have intermediate geometries saved too. In that case we should re-define this task to generate VT from that and debug/deal with clipToGrid.