DBClusterAlreadyExistsFault when replacing cluster resource & not using randomly generated identifiers

[jfchevrette]

Description

When making a change to a cluster parameter which forces a resource re-creation, and also not using randomly generated cluster_instance identifier (or prefix), AWS returns a DBClusterAlreadyExistsFault error, since the aws_rds_cluster resource is being created first before the existing one is destroyed

• [x] ✋ I have searched the open/closed issues and my issue is not listed.

Versions

• Module version [Required]: 8.3

• Terraform version: 1.4.6

• Provider version(s):

  • provider registry.terraform.io/hashicorp/aws v4.67.0

Reproduction Code [Required]

resource "aws_rds_global_cluster" "this" {
  provider                  = aws.primary
  global_cluster_identifier = var.name
  engine                    = var.engine
  engine_version            = var.engine_version
  database_name             = var.database_name
  storage_encrypted         = true
}

module "primary" {
  source  = "terraform-aws-modules/rds-aurora/aws"
  version = "~> 8.3"

  providers = { aws = aws.primary }

  name = var.name
  database_name             = aws_rds_global_cluster.this.database_name

  global_cluster_identifier = aws_rds_global_cluster.this.id
  engine                    = aws_rds_global_cluster.this.engine
  engine_mode               = var.engine_mode
  engine_version            = aws_rds_global_cluster.this.engine_version
  instance_class            = var.instance_class
  instances                 = { for i in range(var.instances_per_cluster) : i => {} }

  vpc_id                    = data.aws_vpc.primary.id
  db_subnet_group_name      = aws_db_subnet_group.primary.name
  security_group_rules = {
    vpc_ingress = {
      cidr_blocks = ["0.0.0.0/0"]
    }
  }

  kms_key_id = aws_kms_key.primary.arn

  master_username             = var.master_username
  master_password             = random_password.master.result
  manage_master_user_password = false

  skip_final_snapshot       = false
  final_snapshot_identifier = "${var.name}-primary-final-snapshot"

  create_db_cluster_parameter_group = true
  db_cluster_parameter_group_name   = "${var.name}-parameter-group"
  db_cluster_parameter_group_family      = "aurora-postgresql14"
  db_cluster_parameter_group_description = "Aurora RDS param group for ${var.name}"
  db_cluster_parameter_group_parameters = [
    {
      name  = "rds.logical_replication"
      value = "1"
      apply_method = "pending-reboot"
    },
    {
      name  = "wal_sender_timeout"
      value = "0"
      apply_method = "pending-reboot"
    }
  ]
}

Steps to reproduce the behavior:

• Change var.master_username

Expected behavior

Because master_username on aws_rds_cluster does not support in-place update, the resource must be re-created. Because of this, terraform wants to replace both the aws_rds_cluster and aws_rds_cluster_instance(s)

Terraform should destroy the DB instances and cluster first, and then create them

Actual behavior

Terraform starts with trying to create a new aws_rds_cluster first, before destroying the previous one, but with the same name (because I am not using cluster_identifier_prefix?)

Terminal Output Screenshot(s)

module.primary.aws_rds_cluster.this[0]: Creating...
╷
│ Error: creating RDS Cluster (foo): DBClusterAlreadyExistsFault: DB Cluster already exists
│   status code: 400, request id: REDACTED
│
│   with module.primary.aws_rds_cluster.this[0],
│   on .terraform/modules/primary/main.tf line 39, in resource "aws_rds_cluster" "this":
│   39: resource "aws_rds_cluster" "this" {
│

Additional context

I think if I would be using cluster_identifier_prefix this would not be a problem because terraform would attempt to create a new cluster and new instances first, with a generated name based on the prefix. But because I am not using this and instead defining a fixed cluster_identifier the order of operations must be different.

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 10 days

[github-actions[bot]]

This issue was automatically closed because of stale in 10 days

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.