search

AmericanRedCross / osm-stats

Track and analyze contributions to Missing Maps
http://missingmaps.org
BSD 3-Clause "New" or "Revised" License
26 stars 9 forks source link

lambda -> RDS errors #25

Closed matthewhanson closed 5 years ago

matthewhanson commented 8 years ago

The Lambda functions suffer from intermittent errors where it times out trying to connect to the database. This appears to be related to the way that lambda containers are reused and the database connection pooling that is performed through knex.

It seems that this is a known issue with lambda and RDS: https://forums.aws.amazon.com/thread.jspa?messageID=678029

so for now we are looking for a work-around, such as closing the connection after each invocation.

Note thatIf the database timesout, the lambda function is retried with the same data until it succeeds. So while this doesn't lead to any data loss, it can lead to a backlog of data.

kriscarle commented 8 years ago

Update on this, knex is switching from pool2 to generic-pool in 0.12+ in an attempt to fix all of the bad pool state issues https://github.com/tgriesser/knex/pull/1665

mojodna commented 5 years ago

This refers to the previous version of osm-stats-workers and is no longer relevant.