mitodl / ol-infrastructure

Infrastructure automation code for use by MIT Open Learning
BSD 3-Clause "New" or "Revised" License
43 stars 4 forks source link

RDS certificate migrations #2194

Open Ardiea opened 4 months ago

Ardiea commented 4 months ago

We get peroidic reminders from AWS about some of our RDS instances using old certificates and they need to be updated.

Review the health dashboard and remediate as necessary.

Hello,

[AWS Health may periodically trigger reminder notifications about this communication.]

You are receiving this message because your AWS Account has one or more Amazon RDS, or Amazon Aurora database instances in the US-EAST-1 Region using a SSL/TLS Certificate that is expiring on August 22, 2024.

A list of your affected resources can be found in the 'Affected resources' tab of your AWS Health Dashboard.

This is a follow-up notification for SSL/TLS CA certification expiration. If you believe you have already finished this work and still received this email, it is likely because you created new instances using the 2019 Certificate Authority (CA). After January 25, 2024 all newly created instances that do not explicitly specify a different CA will use the ‘rds-ca-rsa2048-g1’ CA. For information on setting an account level CA override, see the modify-certificates API documentation [1].

If your applications connect to these instances using the SSL/TLS protocol, you will need to take action before August 22, 2024 to prevent connectivity failures to your existing database instances. Even if you do not currently use SSL for your connections, you could still be affected if your databases server certificate expires, so we still recommend updating your CA.

To protect your communications with your database instances, a CA generates time-bound certificates that are checked by your database client software to authenticate any database instance before exchanging information. Following industry best practices, AWS renews the CA and creates new certificates on a routine basis to ensure customer connections are properly protected for years to come. The current CA in the US-EAST-1 Region will expire on August 22, 2024. Before this date you will need to update your DB server certificate. The following is the general process to do this:

First, update your application clients with the new certificate, if your application client is using a trust store then add the new CA certificates into the trust stores of your client applications. RDS provides download links to the CA certificates in our User Guide [2]. For more detailed instructions on updating the trust stores on your client application see our documentation [3].

Second, update the certificate on all your affected database instances to one of the newly issued CAs. ‘rds-ca-rsa2048-g1’ is the default recommended CA because there is no algorithm change. The other CAs use new key algorithms so it could require more testing of your client setup to ensure compatibility. For more information on the new CAs see our documentation [4].

Third, if you want to use a different CA than the default ‘rds-ca-rsa2048-g1’, you will need to set an account level CA override so your new instances will use the CA of your choice. To do this a modify-certificates API is available that will allow you to override the default CA on newly created database instances to either the old or one of the new CAs. This override will only apply while the CA you are overriding to is valid. To use this API you will need to be running the AWS CLI version 1.17 or later. For more information see the modify-certificates API documentation [1]. There is also a describe-certificates API [5], that will indicate your current default CA override if you have one set. To set a specific CA during instance creation use the ca-certificate-identifier option on the create-db-instance API to create a DB instance with a specific CA. For more information, see the create-db-instance API documentation [6].

For more detailed instructions on how to perform these updates, please see the Amazon RDS instances [7] and Amazon Aurora instances [8] documentation.

Please be aware of the following timeline:

  1. As soon as possible - You should update your client trust store, then you can update your instances server certificate to one generated by one of the new CAs.

  2. After January 26, 2024 - any new database instances default to using the ‘rds-ca-rsa2048-g1’ certificate. If you wish for new instances to use a different certificate, you can specify which certificate to use with the AWS console or the AWS CLI. For more information, see the create-db-instance API documentation [6].

  3. After August 22, 2024 - ‘rds-ca-2019’ will expire. You will need to take action before August 22, 2024, to prevent connectivity failures to your existing database instances.

For more information see our blog [9], and if you have questions or concerns, please contact AWS Support [10].

[1] https://docs.aws.amazon.com/cli/latest/reference/rds/modify-certificates.html [2] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL.html#UsingWithRDS.SSL.CertificatesAllRegions [3] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL-certificate-rotation.html#UsingWithRDS.SSL-certificate-rotation-updating [4] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL.html#UsingWithRDS.SSL.RegionCertificateAuthorities [5] https://docs.aws.amazon.com/cli/latest/reference/rds/describe-certificates.html [6] https://docs.aws.amazon.com/cli/latest/reference/rds/create-db-instance.html [7] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL-certificate-rotation.html [8] https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/UsingWithRDS.SSL-certificate-rotation.html [9] https://aws.amazon.com/blogs/aws/rotate-your-ssl-tls-certificates-now-amazon-rds-and-amazon-aurora-expire-in-2024/ [10] https://console.aws.amazon.com/support/home

Sincerely, Amazon Web Services

Amazon Web Services, Inc. is a subsidiary of Amazon.com, Inc. Amazon.com is a registered trademark of Amazon.com, Inc. This message was produced and distributed by Amazon Web Services Inc., 410 Terry Ave. North, Seattle, WA 98109-5210

shaidar commented 3 months ago

I updated some CI/QA DBs to use the new default rds-ca-rsa204-g1 and will give it a bit of time to test/verify before we go ahead and schedule prod ones to happen during the maintenance window.

shaidar commented 2 months ago

Down to 43 from 67 I believe. Scheduled a few more to happen during the scheduled maintenance window and should drop the number to the 30s.

Ardiea commented 1 week ago

This is down to 7 with 3 pending maint window updates. The remaining four are reddit and CAS which no one wants to touch. Probably let them expire? There is already a production CAS database with an expired cert from 2015, so it is probably not a huge issue.