Retry to establish database and redis connection a few times before raising an error

alephdata / memorious

Lightweight web scraping toolkit for documents and structured data.

https://docs.alephdata.org/developers/memorious

MIT License

311 stars 59 forks source link

Retry to establish database and redis connection a few times before raising an error #181

Open sunu opened 3 years ago

sunu commented 3 years ago

Sometimes databases restart for maintenance while a long running crawler is running. Instead of immediately quitting with an in that case, we should retry to establish the connection a few times with back off to allow some time for the database to recover.