InvalidParameterCombination: Cannot upgrade aurora-postgresql from 12.11 to 13.6

gowthamakanthan commented 2 years ago

Description

Not able to upgrade the postgresql from 12.11 to 13.6 due to InvalidParameterCombination error.

[x ] ✋ I have searched the open/closed issues and my issue is not listed.

Versions

Module version [Required]: 6.2.0
Terraform version: v0.13.6
Provider version(s): aws - 3.74.1 postgresql - 1.11.2

Reproduction Code [Required]

resource "aws_rds_cluster_parameter_group" "enable_replication" {
  name        = "aurora-pg13-cluster-repl"
  family      = "aurora-postgresql13"
  description = "RDS cluster parameter group replication"

  parameter {
    name         = "rds.logical_replication"
    value        = "1"
    apply_method = "pending-reboot"
  }

  parameter {
    name         = "max_worker_processes"
    value        = "50"
    apply_method = "pending-reboot"
  }

  parameter {
    name         = "max_replication_slots"
    value        = "50"
    apply_method = "pending-reboot"
  }
  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_db_parameter_group" "enable_pglogical" {
  name   = "aurora-pg13-pglogical"
  family = "aurora-postgresql13"

  parameter {
    name         = "shared_preload_libraries"
    value        = "pglogical,pg_stat_statements"
    apply_method = "pending-reboot"
  }

  parameter {
    name         = "log_connections"
    value        = "1"
    apply_method = "immediate"
  }
  lifecycle {
    create_before_destroy = true
  }
}

module "psqldb" {
  source  = "terraform-aws-modules/rds-aurora/aws"
  version = "6.2.0"

  name = "psqldb"

  engine         = "aurora-postgresql"
  engine_version = "13.6"

  vpc_id         = data.terraform_remote_state.aws_core_metadata.outputs.vpc_id
  subnets        = data.terraform_remote_state.aws_core_metadata.outputs.private_subnets
  instance_class = var.rds_cluster_instance_class
  instances = {
    1 = {}
    2 = {}
  }

  allowed_security_groups = [
    data.terraform_remote_state.aws_core_metadata.outputs.eks_worker_security_group_id,
    data.terraform_remote_state.aws_core_metadata.outputs.bastion_security_group_id
  ]

  allowed_cidr_blocks = local.rds_cidrs
  security_group_egress_rules = {
    to_cidrs = {
      cidr_blocks = local.rds_cidrs
      description = "From allowed_cidr_blocks"
      from_port   = "5432"
      to_port     = "5432"
    }
  }
  deletion_protection         = true
  security_group_description  = "Managed by Terraform"
  storage_encrypted           = true
  apply_immediately           = true
  monitoring_interval         = 10
  publicly_accessible         = false
  skip_final_snapshot         = local.rds_skip_final_snapshot
  allow_major_version_upgrade = true

  db_parameter_group_name         = aws_db_parameter_group.enable_pglogical.name
  db_cluster_parameter_group_name = aws_rds_cluster_parameter_group.enable_replication.name

  enabled_cloudwatch_logs_exports = ["postgresql"]

  iam_database_authentication_enabled = true

  tags = local.tags

  autoscaling_enabled      = true
  autoscaling_min_capacity = 2
  autoscaling_max_capacity = 3
}

Steps to reproduce the behaviour:

Upgrading cluster from 12.11 to 13.6.

terraform apply ## Expected behavior RDS cluster,instances and parameter groups should upgraded. ## Actual behavior ``` # aws_db_parameter_group.enable_pglogical must be replaced +/- resource "aws_db_parameter_group" "enable_pglogical" { ~ arn = "arn:aws:rds:us-east-1:681496624581:pg:aurora-pg12-pglogical" -> (known after apply) description = "Managed by Terraform" ~ family = "aurora-postgresql12" -> "aurora-postgresql13" # forces replacement ~ id = "aurora-pg12-pglogical" -> (known after apply) ~ name = "aurora-pg12-pglogical" -> "aurora-pg13-pglogical" # forces replacement + name_prefix = (known after apply) - tags = {} -> null ~ tags_all = {} -> (known after apply) parameter { apply_method = "immediate" name = "log_connections" value = "1" } parameter { apply_method = "pending-reboot" name = "shared_preload_libraries" value = "pglogical,pg_stat_statements" } } # aws_rds_cluster_parameter_group.enable_replication must be replaced +/- resource "aws_rds_cluster_parameter_group" "enable_replication" { ~ arn = "arn:aws:rds:us-east-1:681496624581:cluster-pg:aurora-pg12-cluster-repl" -> (known after apply) description = "RDS cluster parameter group replication" ~ family = "aurora-postgresql12" -> "aurora-postgresql13" # forces replacement ~ id = "aurora-pg12-cluster-repl" -> (known after apply) ~ name = "aurora-pg12-cluster-repl" -> "aurora-pg13-cluster-repl" # forces replacement + name_prefix = (known after apply) - tags = {} -> null ~ tags_all = {} -> (known after apply) parameter { apply_method = "pending-reboot" name = "max_replication_slots" value = "50" } parameter { apply_method = "pending-reboot" name = "max_worker_processes" value = "50" } parameter { apply_method = "pending-reboot" name = "rds.logical_replication" value = "1" } } # module.psqldb.aws_rds_cluster.this[0] will be updated in-place ~ resource "aws_rds_cluster" "this" { allow_major_version_upgrade = true apply_immediately = true arn = "arn:aws:rds:us-east-1:681496624581:cluster:psqldb" availability_zones = [ "us-east-1a", "us-east-1b", "us-east-1d", ] backtrack_window = 0 backup_retention_period = 7 cluster_identifier = "psqldb" cluster_members = [ "psqldb-1", "psqldb-2", ] cluster_resource_id = "cluster-AGN3JQUSROTLPAX6L6NL3FSYKM" copy_tags_to_snapshot = false ~ db_cluster_parameter_group_name = "aurora-pg12-cluster-repl" -> "aurora-pg13-cluster-repl" db_subnet_group_name = "psqldb" deletion_protection = true enable_global_write_forwarding = false enable_http_endpoint = false enabled_cloudwatch_logs_exports = [ "postgresql", ] endpoint = "psqldb.cluster-cmt6phsh6krf.us-east-1.rds.amazonaws.com" engine = "aurora-postgresql" engine_mode = "provisioned" ~ engine_version = "12.11" -> "13.6" engine_version_actual = "12.11" final_snapshot_identifier = "final-psqldb-7bf0383d" hosted_zone_id = "Z2R2ITUGPM61AM" iam_database_authentication_enabled = true iam_roles = [] id = "psqldb" kms_key_id = "arn:aws:kms:us-east-1:681496624581:key/f581b6ba-0295-408c-a3cf-ae1a916f5b9e" master_password = (sensitive value) master_username = "root" port = 5432 preferred_backup_window = "02:00-03:00" preferred_maintenance_window = "sun:05:00-sun:06:00" reader_endpoint = "psqldb.cluster-ro-cmt6phsh6krf.us-east-1.rds.amazonaws.com" skip_final_snapshot = true storage_encrypted = true tags = { "Account" = "dev" "ApplicationName" = "psql-base" "CiscoMailAlias" = "dcloud-devops@cisco.com" "DataClassification" = "Cisco Confidential" "DataTaxonomy" = "Cisco Operations Data" "Environment" = "dev" "ResourceOwner" = "dCloud" "Terraform" = "true" } tags_all = { "Account" = "dev" "ApplicationName" = "psql-base" "CiscoMailAlias" = "dcloud-devops@cisco.com" "DataClassification" = "Cisco Confidential" "DataTaxonomy" = "Cisco Operations Data" "Environment" = "dev" "ResourceOwner" = "dCloud" "Terraform" = "true" } vpc_security_group_ids = [ "sg-054df949da5864e56", ] timeouts {} } # module.psqldb.aws_rds_cluster_instance.this["1"] will be updated in-place ~ resource "aws_rds_cluster_instance" "this" { apply_immediately = true arn = "arn:aws:rds:us-east-1:681496624581:db:psqldb-1" auto_minor_version_upgrade = true availability_zone = "us-east-1b" ca_cert_identifier = "rds-ca-2019" cluster_identifier = "psqldb" copy_tags_to_snapshot = false ~ db_parameter_group_name = "aurora-pg12-pglogical" -> "aurora-pg13-pglogical" db_subnet_group_name = "psqldb" dbi_resource_id = "db-U5CPJMUW6VRXIVHFLHISWWM4VQ" endpoint = "psqldb-1.cmt6phsh6krf.us-east-1.rds.amazonaws.com" engine = "aurora-postgresql" ~ engine_version = "12.11" -> "13.6" engine_version_actual = "12.11" id = "psqldb-1" identifier = "psqldb-1" instance_class = "db.t3.medium" kms_key_id = "arn:aws:kms:us-east-1:681496624581:key/f581b6ba-0295-408c-a3cf-ae1a916f5b9e" monitoring_interval = 10 monitoring_role_arn = "arn:aws:iam::681496624581:role/rds-enhanced-monitoring-psqldb" performance_insights_enabled = false performance_insights_retention_period = 0 port = 5432 preferred_backup_window = "02:00-03:00" preferred_maintenance_window = "sun:05:00-sun:06:00" promotion_tier = 0 publicly_accessible = false storage_encrypted = true tags = { "Account" = "dev" "ApplicationName" = "psql-base" "CiscoMailAlias" = "dcloud-devops@cisco.com" "DataClassification" = "Cisco Confidential" "DataTaxonomy" = "Cisco Operations Data" "Environment" = "dev" "ResourceOwner" = "dCloud" "Terraform" = "true" } tags_all = { "Account" = "dev" "ApplicationName" = "psql-base" "CiscoMailAlias" = "dcloud-devops@cisco.com" "DataClassification" = "Cisco Confidential" "DataTaxonomy" = "Cisco Operations Data" "Environment" = "dev" "ResourceOwner" = "dCloud" "Terraform" = "true" } writer = true timeouts {} } # module.psqldb.aws_rds_cluster_instance.this["2"] will be updated in-place ~ resource "aws_rds_cluster_instance" "this" { apply_immediately = true arn = "arn:aws:rds:us-east-1:681496624581:db:psqldb-2" auto_minor_version_upgrade = true availability_zone = "us-east-1b" ca_cert_identifier = "rds-ca-2019" cluster_identifier = "psqldb" copy_tags_to_snapshot = false ~ db_parameter_group_name = "aurora-pg12-pglogical" -> "aurora-pg13-pglogical" db_subnet_group_name = "psqldb" dbi_resource_id = "db-5X7GWHG67Q6YPPVTGE76LXMV54" endpoint = "psqldb-2.cmt6phsh6krf.us-east-1.rds.amazonaws.com" engine = "aurora-postgresql" ~ engine_version = "12.11" -> "13.6" engine_version_actual = "12.11" id = "psqldb-2" identifier = "psqldb-2" instance_class = "db.t3.medium" kms_key_id = "arn:aws:kms:us-east-1:681496624581:key/f581b6ba-0295-408c-a3cf-ae1a916f5b9e" monitoring_interval = 10 monitoring_role_arn = "arn:aws:iam::681496624581:role/rds-enhanced-monitoring-psqldb" performance_insights_enabled = false performance_insights_retention_period = 0 port = 5432 preferred_backup_window = "02:00-03:00" preferred_maintenance_window = "sun:05:00-sun:06:00" promotion_tier = 0 publicly_accessible = false storage_encrypted = true tags = { "Account" = "dev" "ApplicationName" = "psql-base" "CiscoMailAlias" = "dcloud-devops@cisco.com" "DataClassification" = "Cisco Confidential" "DataTaxonomy" = "Cisco Operations Data" "Environment" = "dev" "ResourceOwner" = "dCloud" "Terraform" = "true" } tags_all = { "Account" = "dev" "ApplicationName" = "psql-base" "CiscoMailAlias" = "dcloud-devops@cisco.com" "DataClassification" = "Cisco Confidential" "DataTaxonomy" = "Cisco Operations Data" "Environment" = "dev" "ResourceOwner" = "dCloud" "Terraform" = "true" } writer = false timeouts {} } Plan: 2 to add, 3 to change, 2 to destroy. Do you want to perform these actions in workspace "dev"? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes aws_rds_cluster_parameter_group.enable_replication: Creating... aws_db_parameter_group.enable_pglogical: Creating... aws_rds_cluster_parameter_group.enable_replication: Still creating... [10s elapsed] aws_db_parameter_group.enable_pglogical: Still creating... [10s elapsed] aws_rds_cluster_parameter_group.enable_replication: Creation complete after 12s [id=aurora-pg13-cluster-repl] module.psqldb.aws_rds_cluster.this[0]: Modifying... [id=psqldb] aws_db_parameter_group.enable_pglogical: Still creating... [20s elapsed] aws_db_parameter_group.enable_pglogical: Creation complete after 23s [id=aurora-pg13-pglogical] Error: Failed to modify RDS Cluster (psqldb): InvalidParameterCombination: Cannot upgrade aurora-postgresql from 12.11 to 13.6 status code: 400, request id: c9ee2dd1-e8de-44d9-9757-671ecb969c89 on .terraform/modules/psqldb/main.tf line 47, in resource "aws_rds_cluster" "this": 47: resource "aws_rds_cluster" "this" { ```

gowthamakanthan commented 2 years ago

Have checked the valid upgrade target from 12.11, which resulted 13.7 as the next valid version to upgrade. Tried the same and got the following error. Any thoughts?

 #aws rds describe-db-engine-versions --engine postgres  --engine-version 12.11 --query "DBEngineVersions[*].ValidUpgradeTarget[*].{EngineVersion:EngineVersion}" --output text
13.7
14.3

terraform apply output:


  # aws_db_parameter_group.enable_pglogical must be replaced
+/- resource "aws_db_parameter_group" "enable_pglogical" {
      ~ arn         = "arn:aws:rds:us-east-1:681496624581:pg:aurora-pg12-pglogical" -> (known after apply)
        description = "Managed by Terraform"
      ~ family      = "aurora-postgresql12" -> "aurora-postgresql13" # forces replacement
      ~ id          = "aurora-pg12-pglogical" -> (known after apply)
      ~ name        = "aurora-pg12-pglogical" -> "aurora-pg13-pglogical" # forces replacement
      + name_prefix = (known after apply)
      - tags        = {} -> null
      ~ tags_all    = {} -> (known after apply)

        parameter {
            apply_method = "immediate"
            name         = "log_connections"
            value        = "1"
        }
        parameter {
            apply_method = "pending-reboot"
            name         = "shared_preload_libraries"
            value        = "pglogical,pg_stat_statements"
        }
    }

  # aws_rds_cluster_parameter_group.enable_replication must be replaced
+/- resource "aws_rds_cluster_parameter_group" "enable_replication" {
      ~ arn         = "arn:aws:rds:us-east-1:681496624581:cluster-pg:aurora-pg12-cluster-repl" -> (known after apply)
        description = "RDS cluster parameter group replication"
      ~ family      = "aurora-postgresql12" -> "aurora-postgresql13" # forces replacement
      ~ id          = "aurora-pg12-cluster-repl" -> (known after apply)
      ~ name        = "aurora-pg12-cluster-repl" -> "aurora-pg13-cluster-repl" # forces replacement
      + name_prefix = (known after apply)
      - tags        = {} -> null
      ~ tags_all    = {} -> (known after apply)

        parameter {
            apply_method = "pending-reboot"
            name         = "max_replication_slots"
            value        = "50"
        }
        parameter {
            apply_method = "pending-reboot"
            name         = "max_worker_processes"
            value        = "50"
        }
        parameter {
            apply_method = "pending-reboot"
            name         = "rds.logical_replication"
            value        = "1"
        }
    }

  # module.psqldb.aws_rds_cluster.this[0] will be updated in-place
  ~ resource "aws_rds_cluster" "this" {
        allow_major_version_upgrade         = true
        apply_immediately                   = true
        arn                                 = "arn:aws:rds:us-east-1:681496624581:cluster:psqldb"
        availability_zones                  = [
            "us-east-1a",
            "us-east-1b",
            "us-east-1d",
        ]
        backtrack_window                    = 0
        backup_retention_period             = 7
        cluster_identifier                  = "psqldb"
        cluster_members                     = [
            "psqldb-1",
            "psqldb-2",
        ]
        cluster_resource_id                 = "cluster-AGN3JQUSROTLPAX6L6NL3FSYKM"
        copy_tags_to_snapshot               = false
      ~ db_cluster_parameter_group_name     = "aurora-pg12-cluster-repl" -> "aurora-pg13-cluster-repl"
        db_subnet_group_name                = "psqldb"
        deletion_protection                 = true
        enable_global_write_forwarding      = false
        enable_http_endpoint                = false
        enabled_cloudwatch_logs_exports     = [
            "postgresql",
        ]
        endpoint                            = "psqldb.cluster-cmt6phsh6krf.us-east-1.rds.amazonaws.com"
        engine                              = "aurora-postgresql"
        engine_mode                         = "provisioned"
      ~ engine_version                      = "12.11" -> "13.7"
        engine_version_actual               = "12.11"
        final_snapshot_identifier           = "final-psqldb-7bf0383d"
        hosted_zone_id                      = "Z2R2ITUGPM61AM"
        iam_database_authentication_enabled = true
        iam_roles                           = []
        id                                  = "psqldb"
        kms_key_id                          = "arn:aws:kms:us-east-1:681496624581:key/f581b6ba-0295-408c-a3cf-ae1a916f5b9e"
        master_password                     = (sensitive value)
        master_username                     = "root"
        port                                = 5432
        preferred_backup_window             = "02:00-03:00"
        preferred_maintenance_window        = "sun:05:00-sun:06:00"
        reader_endpoint                     = "psqldb.cluster-ro-cmt6phsh6krf.us-east-1.rds.amazonaws.com"
        skip_final_snapshot                 = true
        storage_encrypted                   = true
        tags                                = {
            "Account"            = "dev"
            "ApplicationName"    = "psql-base"
            "CiscoMailAlias"     = "dcloud-devops@cisco.com"
            "DataClassification" = "Cisco Confidential"
            "DataTaxonomy"       = "Cisco Operations Data"
            "Environment"        = "dev"
            "ResourceOwner"      = "dCloud"
            "Terraform"          = "true"
        }
        tags_all                            = {
            "Account"            = "dev"
            "ApplicationName"    = "psql-base"
            "CiscoMailAlias"     = "dcloud-devops@cisco.com"
            "DataClassification" = "Cisco Confidential"
            "DataTaxonomy"       = "Cisco Operations Data"
            "Environment"        = "dev"
            "ResourceOwner"      = "dCloud"
            "Terraform"          = "true"
        }
        vpc_security_group_ids              = [
            "sg-054df949da5864e56",
        ]

        timeouts {}
    }

  # module.psqldb.aws_rds_cluster_instance.this["1"] will be updated in-place
  ~ resource "aws_rds_cluster_instance" "this" {
        apply_immediately                     = true
        arn                                   = "arn:aws:rds:us-east-1:681496624581:db:psqldb-1"
        auto_minor_version_upgrade            = true
        availability_zone                     = "us-east-1b"
        ca_cert_identifier                    = "rds-ca-2019"
        cluster_identifier                    = "psqldb"
        copy_tags_to_snapshot                 = false
      ~ db_parameter_group_name               = "aurora-pg12-pglogical" -> "aurora-pg13-pglogical"
        db_subnet_group_name                  = "psqldb"
        dbi_resource_id                       = "db-U5CPJMUW6VRXIVHFLHISWWM4VQ"
        endpoint                              = "psqldb-1.cmt6phsh6krf.us-east-1.rds.amazonaws.com"
        engine                                = "aurora-postgresql"
      ~ engine_version                        = "12.11" -> "13.7"
        engine_version_actual                 = "12.11"
        id                                    = "psqldb-1"
        identifier                            = "psqldb-1"
        instance_class                        = "db.t3.medium"
        kms_key_id                            = "arn:aws:kms:us-east-1:681496624581:key/f581b6ba-0295-408c-a3cf-ae1a916f5b9e"
        monitoring_interval                   = 10
        monitoring_role_arn                   = "arn:aws:iam::681496624581:role/rds-enhanced-monitoring-psqldb"
        performance_insights_enabled          = false
        performance_insights_retention_period = 0
        port                                  = 5432
        preferred_backup_window               = "02:00-03:00"
        preferred_maintenance_window          = "sun:05:00-sun:06:00"
        promotion_tier                        = 0
        publicly_accessible                   = false
        storage_encrypted                     = true
        tags                                  = {
            "Account"            = "dev"
            "ApplicationName"    = "psql-base"
            "CiscoMailAlias"     = "dcloud-devops@cisco.com"
            "DataClassification" = "Cisco Confidential"
            "DataTaxonomy"       = "Cisco Operations Data"
            "Environment"        = "dev"
            "ResourceOwner"      = "dCloud"
            "Terraform"          = "true"
        }
        tags_all                              = {
            "Account"            = "dev"
            "ApplicationName"    = "psql-base"
            "CiscoMailAlias"     = "dcloud-devops@cisco.com"
            "DataClassification" = "Cisco Confidential"
            "DataTaxonomy"       = "Cisco Operations Data"
            "Environment"        = "dev"
            "ResourceOwner"      = "dCloud"
            "Terraform"          = "true"
        }
        writer                                = true

        timeouts {}
    }

  # module.psqldb.aws_rds_cluster_instance.this["2"] will be updated in-place
  ~ resource "aws_rds_cluster_instance" "this" {
        apply_immediately                     = true
        arn                                   = "arn:aws:rds:us-east-1:681496624581:db:psqldb-2"
        auto_minor_version_upgrade            = true
        availability_zone                     = "us-east-1a"
        ca_cert_identifier                    = "rds-ca-2019"
        cluster_identifier                    = "psqldb"
        copy_tags_to_snapshot                 = false
      ~ db_parameter_group_name               = "aurora-pg12-pglogical" -> "aurora-pg13-pglogical"
        db_subnet_group_name                  = "psqldb"
        dbi_resource_id                       = "db-RMB6LTBPSZVI4LLPLVIOLRIWCU"
        endpoint                              = "psqldb-2.cmt6phsh6krf.us-east-1.rds.amazonaws.com"
        engine                                = "aurora-postgresql"
      ~ engine_version                        = "12.11" -> "13.7"
        engine_version_actual                 = "12.11"
        id                                    = "psqldb-2"
        identifier                            = "psqldb-2"
        instance_class                        = "db.t3.medium"
        kms_key_id                            = "arn:aws:kms:us-east-1:681496624581:key/f581b6ba-0295-408c-a3cf-ae1a916f5b9e"
        monitoring_interval                   = 10
        monitoring_role_arn                   = "arn:aws:iam::681496624581:role/rds-enhanced-monitoring-psqldb"
        performance_insights_enabled          = false
        performance_insights_retention_period = 0
        port                                  = 5432
        preferred_backup_window               = "02:00-03:00"
        preferred_maintenance_window          = "sun:05:00-sun:06:00"
        promotion_tier                        = 0
        publicly_accessible                   = false
        storage_encrypted                     = true
        tags                                  = {
            "Account"            = "dev"
            "ApplicationName"    = "psql-base"
            "CiscoMailAlias"     = "dcloud-devops@cisco.com"
            "DataClassification" = "Cisco Confidential"
            "DataTaxonomy"       = "Cisco Operations Data"
            "Environment"        = "dev"
            "ResourceOwner"      = "dCloud"
            "Terraform"          = "true"
        }
        tags_all                              = {
            "Account"            = "dev"
            "ApplicationName"    = "psql-base"
            "CiscoMailAlias"     = "dcloud-devops@cisco.com"
            "DataClassification" = "Cisco Confidential"
            "DataTaxonomy"       = "Cisco Operations Data"
            "Environment"        = "dev"
            "ResourceOwner"      = "dCloud"
            "Terraform"          = "true"
        }
        writer                                = false

        timeouts {}
    }

Plan: 2 to add, 3 to change, 2 to destroy.

Do you want to perform these actions in workspace "dev"?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_db_parameter_group.enable_pglogical: Creating...
aws_rds_cluster_parameter_group.enable_replication: Creating...
aws_rds_cluster_parameter_group.enable_replication: Still creating... [11s elapsed]
aws_db_parameter_group.enable_pglogical: Still creating... [11s elapsed]
aws_rds_cluster_parameter_group.enable_replication: Creation complete after 13s [id=aurora-pg13-cluster-repl]
module.psqldb.aws_rds_cluster.this[0]: Modifying... [id=psqldb]
aws_db_parameter_group.enable_pglogical: Creation complete after 19s [id=aurora-pg13-pglogical]

Error: Failed to modify RDS Cluster (psqldb): InvalidDBInstanceState: Cannot modify engine version because instance psqldb-2 is running on an old configuration. Apply any pending maintenance actions on the instance before proceeding with the upgrade
    status code: 400, request id: 11018cd1-2f9b-4f71-8f2a-e327c2528a5f

  on .terraform/modules/psqldb/main.tf line 47, in resource "aws_rds_cluster" "this":
  47: resource "aws_rds_cluster" "this" {

Releasing state lock. This may take a few moments...
➜  psql git:(rds_aurora/egress) ✗

bryantbiggs commented 2 years ago

Did you read the error message?

gowthamakanthan commented 2 years ago

yep, it's stating that one of the db instances is on old config and applies any pending maintenance, but nothing is listed for the maintenance. Verified the same in console as well.

 ~ aws rds describe-pending-maintenance-actions --resource-identifier arn:aws:rds:us-east-1:681496624581:cluster:psqldb
{
    "PendingMaintenanceActions": []
}

bryantbiggs commented 2 years ago

It says InvalidDBInstanceState so you need to check the instances themselves, not the cluster

Either way, this is not a module issue

pfragoso commented 2 years ago

Tried this upgrade myself and bumped into similar issues.

First issue is that changing the family forces a parameter group replacement which will fail to delete as it's attached to the running RDS instance. Adding a lifecyle create_before_destroy solved this.

Second, i'm also unable to upgrade due to InvalidParameterCombination.

# module.app.aws_rds_cluster.this[0] will be updated in-place
  ~ resource "aws_rds_cluster" "this" {
      ~ db_cluster_parameter_group_name     = "cluster-20220824111932012900000002" -> "cluster-20220824113436472400000002"
      ~ engine_version                      = "13.7" -> "14.3"
        id                                  = "app"
    }

  # module.app.aws_rds_cluster_instance.this["1"] will be updated in-place
  ~ resource "aws_rds_cluster_instance" "this" {
      ~ db_parameter_group_name               = "instance-20220824111932012800000001" -> "instance-20220824113436472300000001"
      ~ engine_version                        = "13.7" -> "14.3"
        id                                    = "app-1"
    }

module.app.aws_rds_cluster.this[0]: Modifying... [id=app]
╷
│ Error: Failed to modify RDS Cluster (app): InvalidParameterCombination: The current DB instance parameter group instance-20220824111932012800000001 is custom. You must explicitly specify a new DB instance parameter group, either default or custom, for the engine version upgrade.
│       status code: 400, request id: dd9a9e9a-26c2-4197-b6c8-d71fefc3ebf6
│ 
│   with module.app.aws_rds_cluster.this[0],
│   on .terraform/modules/app/main.tf line 47, in resource "aws_rds_cluster" "this":
│   47: resource "aws_rds_cluster" "this" {

Using hashicorp/aws v4.27.0 and rds 7.3.0

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 10 days

gowthamakanthan commented 1 year ago

RDS upgrade has been done. by following the below Steps

Step 1: Applying the recommended patches.

Step 2:

Update OS patching on the existing instance.

Step 3:

Taken a snapshot

Step 4:

Ran terraform apply from the branch rds_aurora/egress which has failed with following error.

Error: Failed to modify RDS Cluster (kapuadb): InvalidParameterCombination: Cannot upgrade aurora-postgresql from 12.11 to 13.6
    status code: 400, request id: 699892b5-72e0-4b26-b0dd-a12781507609

Which sorted out after changing the parameter group from aurora-pg12-pglogical to default.aurora-postgresql12 for the db instances .

Step 5:

Rerun the terraform apply. This time, it's just upgraded.

thomasplarsson commented 1 year ago

@pfragoso Did you solve the issue you encountered? I'm having the same issue.

pfragoso commented 1 year ago

@thomasplarsson Hi. Not really, ended up doing the upgrade by hand.

thomasplarsson commented 1 year ago

@pfragoso Ok. I can report that this worked almost out of the box now for me. I only needed to add the "create_before_destroy" to the parameter groups. I had only made a manual error before.

github-actions[bot] commented 1 year ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

terraform-aws-modules / terraform-aws-rds-aurora