edgi-govdata-archiving / web-monitoring-ops

Documentation and configuration files for EDGI’s deployment of Web Monitoring tools.
GNU General Public License v3.0
1 stars 1 forks source link

Describe RDS Configuration in “Manually Managed” Section #26

Closed Mr0grog closed 5 years ago

Mr0grog commented 5 years ago

Yesterday I set up a new production database in RDS, and spent a fair amount of time poking around to try and get it configured as well as I reasonably could. (A new one was easier because so much needed to change from the current production DB.) This PR adds information about how our RDS databases are configured.

The new database is named web-monitoring-db-production-a, and I’ll remove the old one once we’ve been running comfortable on this new one for a week.

It’s worth noting that the new DB instance is powerful enough to handle the low number of queries we typically experience now, but doesn’t serve our usage patterns very well: the versions table cannot fit in memory no matter what, so count(*) queries on it tend to be extremely slow (they’re still an order of magnitude faster than they used to be because of the SSD and memory optimizations, though). Ultimately, we need to stop always doing count(*) queries as part of every listing endpoint in the API, and should consider a bigger DB and/or replication if we make the system public.

Mr0grog commented 5 years ago

Going ahead and merging this.