hackoregon / civic-devops

Master collection point for issues, procedures, and code to manage the HackOregon Civic platform
MIT License
11 stars 4 forks source link

Update Amazon RDS SSL/TLS Certificates by March 5, 2020 #284

Closed jaronheard closed 4 years ago

jaronheard commented 4 years ago

TEAM NAME: CIVIC Software Foundation PRIORITY (1-5): 1.5 DO NOT INCLUDE ANY SECRETS IN THIS REQUEST. IT IS PUBLICLY ACCESSIBLE

Description of issue We need to update our Amazon RDS SSL/TLS Certificates by March 5, 2020

Error Message/Logs

This is the final reminder about your Amazon RDS certificate authority (CA) certificate updates prior to the March 5, 2020 expiry.

You are receiving this message because you have one or more Amazon RDS database instances (as of February 24, 2020) that require attention in the US-WEST-2 Region. For these instances, you still need to update your Amazon RDS CA certificate before the old certificate expires on March 5, 2020. See the 'Affected resources' tab in your Personal Health Dashboard for a list of instances.

As previously communicated, the current CA expires on March 5, 2020, requiring updates to all client applications and database instances that connect using Secure Sockets Layer (SSL)/Transport Layer Security (TLS) with certificate verification. Client applications must add new CA certificates to their trust stores, including root and intermediate certificates where necessary. RDS database instances must separately use new server certificates before this hard expiration date. If you've missed previous communications on this subject, see the Database Blog post for more information [1].

[1] https://aws.amazon.com/blogs/database/amazon-rds-customers-update-your-ssl-tls-certificates-by-february-5-2020/

Reproduction Steps N/A

Code Snippets N/A

Screenshots/GIF N/A

Priority/Impact If we don't do this by March 5th, bad things may happen?

MikeTheCanuck commented 4 years ago

According to this page, the following RDS instances require an update: https://us-west-2.console.aws.amazon.com/rds/home?region=us-west-2#ca-cert-update:

DB identifier,DB cluster identifier,Engine,Engine version,Status,Apply date censussprint2019,,postgres,11.5,Requires Update, disaster-resilience-2019-staging,,postgres,11.2,Requires Update, disaster-resilience-staging-2019-2,,postgres,11.4,Requires Update, housing-2019-staging,,postgres,11.2,Requires Update, mysample,,postgres,11.5,Requires Update, ncdbpfull,,postgres,11.5,Requires Update, openelections-production,,postgres,10.6,Requires Update, openelections-staging,,postgres,10.6,Requires Update, sandbox-2019-staging,,postgres,11.2,Requires Update, transportation-2019-staging,,postgres,11.2,Requires Update,

MikeTheCanuck commented 4 years ago

Given that I am presently very much uninformed as to development or production activity centering on these database instances, I do not feel comfortable using the Update now option that RDS makes available. So instead I've manually triggered the Update at the next maintenance window option for all ten instances.

image

There is a small risk that some RDS instances will not undergo automatic maintenance (i.e. reboot) until after the March 5th expiration, and so might refuse inbound SSL/TLS connections (or might trigger some kind of application-side refusal to complete a connection with a DB that hosts an expired cert).

image

If such an occasion occurs, it should be easy enough to reboot the affected DB instances - or to wait the 1-2 days necessary for automated maintenance (IIRC, there's at least a weekly cycle automated).

I'm unaware of whether any of the Django apps that connect to these databases are configured to require SSL/TLS connectivity, or how they are configured to handle expired certs.

jaronheard commented 4 years ago

Thanks @MikeTheCanuck! This is great!