michaelvl / osm-analytic-tracker

OpenStreetMap Analytic Difference Engine
GNU General Public License v2.0
38 stars 1 forks source link

Optimize API usage by using data from replication data #22

Closed michaelvl closed 4 years ago

michaelvl commented 5 years ago

Each changeset triggers a metadata requests towards the API server to test the changeset bbox against the filtering area polygon. While the amount of data per changeset is small, the number of changesets are large, hence the traffic amount is considerable.

Since the replication data contains lat/lon of points, this data can be used instead to filter against.

This change should introduce a new filtering type that allows this type of 'is changeset with filtering area' test.

michaelvl commented 5 years ago

This is partly in commit daeafa33

mmd-osm commented 5 years ago

Maybe I'm missing something obvious here, but you would also be able to get changeset metadata from here: https://planet.openstreetmap.org/replication/changesets/ without querying the API.

michaelvl commented 5 years ago

Thanks, I will investigate if this is an even option.