search

hotosm / hot-qa-tiles

QA Tiles Generation
2 stars 0 forks source link

Update/osmium #12

Closed dakotabenjamin closed 4 years ago

dakotabenjamin commented 4 years ago

This PR overhauls the processing tool from using minjur to osmium export. The main changes are that it creates a much larger dataset with multi-polygon support, so we also added some filters for features that are irrelevant to a qa-tiles dataset.

  1. We are using a large, memory-optimized AWS EC2 instance to run weekly processing
  2. The processing script directly downloads a weekly planet .pbf instead of maintaining our own
  3. We use filter-serial.js to remove coastlines, certain administrative boundaries (admin_level < 2), and polygons > 25sqkm. to reduce filesize
  4. We can now specify the s3 bucket for uploading the planet and country extracts