search

cds-snc / notification-planning-core

Project planning for GC Notify Core Team
0 stars 0 forks source link

Have the ability to move RDS database snapshots across accounts #351

Open ben851 opened 1 month ago

ben851 commented 1 month ago

Description

As a developer/operator of GC Notify I would like to be able to move RDS snapshots across accounts so that I have a reliable way of sharing the latest production snapshots. This is useful for BCP scenarios such as moving production to a new account.

WHY are we building?

Currently for BCP scenarios requiring a new AWS account, we must manually do a PG_DUMP of the database and manually restore it. This is a lengthy process and prone to connection errors/dropouts.

WHAT are we building?

We have to migrate our database off of the AWS provided KMS and onto a self-managed KMS as snapshot migration with the default KMS is not supported.

VALUE created by our solution

Reduced MTR for BCP scenarios Increased reliability

Acceptance Criteria

QA Steps

ben851 commented 1 month ago

Support case opened to get advice on best practices: https://support.console.aws.amazon.com/support/home?region=ca-central-1#/case/?displayId=171578212900463&language=en

sastels commented 1 month ago

Reply from AWS

Dear Ben,

Thank you for contacting AWS. This is Tushar from AWS Support Engineering , I have taken ownership of your case and would be happy to assist you today.

From the case description I understand that you wanted to understand how to change the encryption from AWS default to customer manager key and if it is possible to do it via blue/green deployment.

Please correct me if my understanding is not in line with your query.

I would like to mention that unfortunately blue green deployment could not be used to change the KMS key for your existing aurora postgresql cluster , As you can not change the KMS key while deploying a blue green deployment.

The following are limitations for changes in a blue/green deployment:

https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/blue-green-deployments-overview.html#blue-green-deployments-limitations

Further , You can follow steps mentioned in below article to change the encryption key of the existing RDS instance but requires downtime and cutover.

https://repost.aws/knowledge-center/update-encryption-key-rds

For lower downtime and cutover , you can consider using DMS (AWS Database migration service) for creation of new instance with custom KMS Key and replicate data across old and new instance for sync and then cutover.

Overall steps :

1) Create replication slot 2) start snapshot backup of current instance. 3) restore snapshot with new key to create new instance. 4) setup Replication instance , endpoints and task 5) Setup cdc task with the help of replication slot and wait for the sync to complete . 6) Switchover to new database . later if required , you can change DB identifier same as old database .

Above mentioned steps are all high level steps , and it requires in-depth planning for the seamless migration along with task setup and configuration.

You can also refer to below Documents for further details:

Reference Document

https://aws.amazon.com/dms/ https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html https://docs.aws.amazon.com/dms/latest/userguide/CHAP_BestPractices.html
https://aws.amazon.com/blogs/database/using-logical-replication-to-replicate-managed-amazon-rds-for-postgresql-and-amazon-aurora-to-self-managed-postgresql/

I hope you find the above information helpful , please feel free to reach out to us again if you have any further questions or concerns.

Thanks and have a AWSome day ahead.

We value your feedback. Please share your experience by rating this and other correspondences in the AWS Support Center. You can rate a correspondence by selecting the stars in the top right corner of the correspondence.

Best regards, Tushar R. Amazon Web Services

P0NDER0SA commented 1 month ago

beginning to look at this one -- starting with simplistic scenarios such as backing up and deploying a snapshot on the same account.

ben851 commented 3 weeks ago

Ben/Pat/Pond spoke about this and it may be possible to just create a stand-alone KMS and use it for snapshots.