Store measurements and counts as JSON columns

azavea / osmesa

OSMesa is an OpenStreetMap processing stack based on GeoTrellis and Apache Spark

Apache License 2.0

80 stars 26 forks source link

Store measurements and counts as JSON columns #148

Closed mojodna closed 5 years ago

mojodna commented 5 years ago

This also centralizes the lists of stats tracked (and how they're calculated) in addition to introducing non-streaming updaters.

ChangesetStatsCreator now writes directly to a JDBC connection (using the same implementation as the updaters) rather than a CSV, simplifying initial database population.

Hold on merging this until documentation is updated (the root README) and a corresponding implementation is complete on the osmesa-stat-server side.

mojodna commented 5 years ago

This should aggregate "total edits" (sum of all counts) per changeset row.

mojodna commented 5 years ago

It should also break out a count of "other" edits (which can be used to assist backfills of new statistics).

mojodna commented 5 years ago

This should be merged in conjunction with https://github.com/azavea/osmesa-stat-server/pull/20

mojodna commented 5 years ago

Fixes #140