search

aws-controllers-k8s / community

AWS Controllers for Kubernetes (ACK) is a project enabling you to manage AWS services from Kubernetes
https://aws-controllers-k8s.github.io/community/
Apache License 2.0
2.44k stars 258 forks source link

Support RDS replica promotion #2172

Open mmanes opened 2 months ago

mmanes commented 2 months ago

Is your feature request related to a problem? We currently use ACK to provision PostgreSQL RDS instances, and sometimes need to create a cross-region replica for disaster recovery purposes. This can be accomplished by setting sourceDBInstanceIdentifier on the DBInstance resource. However, there is no way to promote it to a primary instance if the primary region fails.

Describe the solution you'd like One possibility would be to detect the removal of the sourceDBInstanceIdentifier field, and then issue a PromoteReadReplicaRequest to complete the failover process.

a-hilaly commented 2 months ago

Hi @mmanes thank you for raising this issue. So if understand well we're looking into adding a new spec field that allows you to promote a db instance to primary? My worry here is, what if 3 instances have this field set to true? we'll have 3 instances trying to compete to become primary and there will never reach final state where only one of them is primary. I'm trying to think if there is a better way to do this.

mmanes commented 2 months ago

Hi @a-hilaly. I was suggesting to use the the existing sourceDBInstanceIdentifier field. When it is set, the instance is created as a replica, so removing that field value could cause it to be promoted to a primary. I'm sure there are other implementation options as well.

I see how having multiple replicas complicates this. When you promote a replica, it leaves the original cluster and becomes a new, non-clustered primary instance. It does not become the primary of the original cluster.

Using this solution, if you cleared out the field of 3 replicas, they would likely all eventually reconcile to be promoted to their own primaries.