azavea / osmesa

OSMesa is an OpenStreetMap processing stack based on GeoTrellis and Apache Spark
Apache License 2.0
80 stars 26 forks source link

Add Streaming AOI Monitor one-off script #161

Closed CloudNiner closed 4 years ago

CloudNiner commented 4 years ago

Supersedes https://github.com/azavea/osmesa/pull/160.

We expect to use this branch for development of this feature. Additional changes to this work should be PRed here.

We are developing a facility to track areas of interest and provide periodic updates to subscribed users as to edits occurring in those regions. This PR provides the component which will monitor and aggregate those changes and queue up the information that will be conveyed to the users.

We intend to deploy this service via ECS scheduled tasks at preset intervals (monthly, weekly, daily), and each interval may aggregate and report its results differently. Email notifications will be handled by a different service.

CloudNiner commented 4 years ago

This effort is feature complete and will work when connected to a augmented diff stream + postgres database configured with CLI args + env variables. Further tweaks pending in future pull requests.

Merging as the individual changes have been fully reviewed, most recently in #166