Investigate why postgres db was overloaded with moderate traffic

omgnetwork / elixir-omg

OMG-Network repository of Watcher and Watcher Info

https://omg.network

Apache License 2.0

211 stars 59 forks source link

Investigate why postgres db was overloaded with moderate traffic #261

Closed jarindr closed 5 years ago

jarindr commented 6 years ago

This incident happen in devcon4, while the blockexplorer polling the /getTransactions endpoint with 1 second interval on client, it overloaded the postgres instance and finally killed it cc. @jbunce .

From the information we have, the amount of load shouldn't be enough to break the postgres / watcher, @jbunce even upgraded the instance but still the polling managed to push the CPU up to close to ~100%

pnowosie commented 6 years ago

I'd like to know more about configuration of the machine and was there Watcher instance running on the same machine or it was dedicated to the DB only.

pnowosie commented 5 years ago

@jarindr @jbunce - can we close this prehistoric issue? A lot has changed since Ari, I guess we already have much better metrics, I can't recall other WatcherDB issues. Feel free to close without further discussion 🤓 Thx