allow worker to skip past a failing changeset after a number of tries

AmericanRedCross / osm-stats

Track and analyze contributions to Missing Maps

http://missingmaps.org

BSD 3-Clause "New" or "Revised" License

26 stars 9 forks source link

allow worker to skip past a failing changeset after a number of tries #35

Closed kriscarle closed 7 years ago

kriscarle commented 8 years ago

If the Lambda worker errors on a changeset more than X times, make a record of what failed then remove it from Kinesis so processing can move forward. Since Lambda is stateless, this will require using Redis or Postgres.

My instance is stuck on a changeset that encounters a bug in one of the metric calculations. The problem is Lambda just keeps pulling and retrying that same changeset over and over.

matthewhanson commented 7 years ago

osm-stats-workers was updated a while ago to always return a success and log the failure.