Auto minor version upgrade & ignore changes

el-capitan-insait commented 4 months ago

Is your request related to a new offering from AWS?

Is this functionality available in the AWS provider for Terraform? See CHANGELOG.md, too.

No 🛑: please wait to file a request until the functionality is avaialble in the AWS provider
Yes ✅: please list the AWS provider version which introduced this functionality

Yes - ignore_changes

Is your request related to a problem? Please describe.

I'm setting [auto_minor_version_upgrade] and the DB had been upgrade . now the next time I'm the TF it notice the change and want to rollback ...

Describe the solution you'd like.

if [auto_minor_version_upgrade] the version should be not trigger an update

lifecycle {
    ignore_changes = [
      engine_version,
    ]
  }

Describe alternatives you've considered.

Right now I'm updating the module var to reflect the new version

Additional context

github-actions[bot] commented 3 months ago

This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 10 days

gayashankgithub commented 3 months ago

I have a similar issue. with auto_minor_version_upgrade set to true TF tries to revert from PostgreSQL (for example) 13.14 to 13.12. (DB is created using 13.12 originally and minor version upgrade has gone through)

I've read couple of updates related to this issue. Seems like the engine_version is not a mandatory argument anymore. Anyway when using PostgreSQL, (suggested)[https://github.com/terraform-aws-modules/terraform-aws-rds-aurora/issues/286#issuecomment-1456380420] methods by @bryantbiggs doesn't seems to work.

I'm using below versions.

    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.55.0"
    }

 ----
 module "postgres_aurora_serverless" {
  source  = "terraform-aws-modules/rds-aurora/aws"
  version = "7.6.2"

Also noticed that when using a module there's a limitation to use lifecycle

  lifecycle {
    ignore_changes = [
      engine_version,
    ]
  }

bryantbiggs commented 3 months ago

issues without reproductions will be ignored - we simply are unable to help troubleshoot without seeing what values are being set and being able to interact with a repro

gayashankgithub commented 3 months ago

Thank you for the update. Adding more details with reproduced issue and steps.

Description

I created an RDS setup for Aurora-PostgreSQL using the terraform-aws-modules/rds-aurora/aws module and set auto_minor_version_upgrade to true. When AWS upgrades the minor versions, Terraform will recognize the new Aurora PostgreSQL version and attempt to roll back when it runs next time.

I noticed that the engine_version is optional, but in my use case, I need to specify a certain major version.

example - my code is here.

I have created PostgreSQL using engine_version = 13.12. AWS has upgraded the minor version to engine_version = 13.13.

When running Terraform I could see the below changes.

engine_version                        = "13.13" -> "13.12"

module.postgres_aurora_serverless.aws_rds_cluster.this[0]: Modifying... [id=test-rds-versioning-test]

Error: updating RDS Cluster (test-rds-versioning-test): operation error RDS: ModifyDBCluster, https response error StatusCode: 400, RequestID: 8ba677ff-6a19-44a2-ac72-4fcb0593e953, api error InvalidParameterCombination: Cannot upgrade aurora-postgresql from 13.13 to 13.12

  with module.postgres_aurora_serverless.aws_rds_cluster.this[0],
  on .terraform/modules/postgres_aurora_serverless/main.tf line 61, in resource "aws_rds_cluster" "this":
  61: resource "aws_rds_cluster" "this" {

Releasing state lock. This may take a few moments...
make: *** [apply-qa] Error 1

Versions

Module version [Required]: 7.6.2
Terraform version:
```
Terraform v1.5.7
on darwin_amd64
```

Provider version(s):

terraform {
required_providers {
aws = {
  source  = "hashicorp/aws"
  version = ">= 4.55.0"
}
random = {
  source  = "hashicorp/random"
  version = ">= 3.5.1"
}
postgresql = {
  source  = "cyrilgdn/postgresql"
  version = "1.21.0"
}
}
}

Reproduction Code [Required]

Steps to reproduce the behaviour:

Are you using workspaces? No Have you cleared the local cache (see Notice section above)? Yes List steps in order that led up to the issue you encountered

Please take note of the following steps:

Set up a PostgreSQL database using the terraform-aws-modules/rds-aurora/aws module. You can refer to the example code for assistance.
Access the DB and select the "Modify" option.
Choose version 13.13 for manual updating of the minor version, which is handled by auto_minor_version_upgrade.
Execute the Terraform commands by running terraform init and terraform plan.
When you make changes to the engine_version, the output will indicate that the database is rolling back to version 13.12 as specified in the configuration file.

Terraform will perform the following actions:

  # module.postgres_aurora_serverless.aws_rds_cluster.this[0] will be updated in-place
  ~ resource "aws_rds_cluster" "this" {
      ~ engine_version                        = "13.13" -> "13.12"
        id                                    = "test-rds-versioning-test"
        tags                                  = {}
        # (43 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

  # module.postgres_aurora_serverless.aws_rds_cluster_instance.this["one"] will be updated in-place
  ~ resource "aws_rds_cluster_instance" "this" {
      ~ engine_version                        = "13.13" -> "13.12"
        id                                    = "test-rds-versioning-test-one"
        tags                                  = {}
        # (27 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.postgres_aurora_serverless.aws_rds_cluster_instance.this["two"] will be updated in-place
  ~ resource "aws_rds_cluster_instance" "this" {
      ~ engine_version                        = "13.13" -> "13.12"
        id                                    = "test-rds-versioning-test-two"
        tags                                  = {}
        # (27 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

Plan: 0 to add, 3 to change, 0 to destroy.

I attempted the following two steps to verify if the suggestions for the previous issues were helpful.

Run terraform refresh to update the state. engine_version = "13.12" -> "13.13" Running terraform plan --refresh-only as mentioned on the issue

Change engine_version (

locals {
engine    = "aurora-postgresql"
version   = "13" 
}

Expected behaviour

After a minor version upgrade, Terraform should ignore the change and should not attempt to roll back to the version specified in the configuration. I have tried mentioning only the major version as engine_version can use a relaxed version constraints

Actual behaviour

Terraform tries to roll back to the version mentioned on the config if it has a minor version. Or throws the below error when the version is specified as 13.

Terraform will perform the following actions:

  # module.postgres_aurora_serverless.aws_rds_cluster.this[0] will be updated in-place
  ~ resource "aws_rds_cluster" "this" {
      ~ engine_version                        = "13.13" -> "13"
        id                                    = "test-rds-versioning-test"
        tags                                  = {}
        # (43 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

  # module.postgres_aurora_serverless.aws_rds_cluster_instance.this["one"] will be updated in-place
  ~ resource "aws_rds_cluster_instance" "this" {
      ~ engine_version                        = "13.13" -> "13"
        id                                    = "test-rds-versioning-test-one"
        tags                                  = {}
        # (27 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.postgres_aurora_serverless.aws_rds_cluster_instance.this["two"] will be updated in-place
  ~ resource "aws_rds_cluster_instance" "this" {
      ~ engine_version                        = "13.13" -> "13"
        id                                    = "test-rds-versioning-test-two"
        tags                                  = {}
        # (27 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

Plan: 0 to add, 3 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.postgres_aurora_serverless.aws_rds_cluster.this[0]: Modifying... [id=test-rds-versioning-test]

Error: updating RDS Cluster (test-rds-versioning-test): operation error RDS: ModifyDBCluster, https response error StatusCode: 400, RequestID: bb9bb19c-51d7-4bab-9a2c-89328e8bd884, api error InvalidParameterCombination: Cannot upgrade aurora-postgresql from 13.13 to 13.12

  with module.postgres_aurora_serverless.aws_rds_cluster.this[0],
  on .terraform/modules/postgres_aurora_serverless/main.tf line 61, in resource "aws_rds_cluster" "this":
  61: resource "aws_rds_cluster" "this" {

Releasing state lock. This may take a few moments...

bryantbiggs commented 3 months ago

you link to where I said to not fully specify the version since it will conflict with the auto minor upgrade, yet you are doing exactly that https://github.com/gayashankgithub/rds-version-testing/blob/bc0ca76b826833bf910c359afc8c771b0fb54a78/infrastructure/main.tf#L3

Do not do that and the issue should be resolved

gayashankgithub commented 3 months ago

Thank you for the update.

Correct me if I'm wrong in the example provided the version is not fully specified when creating the resources allowing version upgrades to happen and not conflict in future.

But In my use case (which I believe is the same as the reported issue), I already have a PostgreSQL database created with version 13.12 (created a couple of months ago), which has been auto-updated. After some time when running Terraform again it cannot figure out what to do with the updated version.

Also below are the steps I tried to get the state updated to the new version and run it with 13.

Get the state update with terraform refresh terraform apply --refresh-only -no-color -var-file="qa/qa.tfvars"

Check updated version in the state. Check the engine_version and engine_version_actual


terraform state  show 'module.postgres_aurora_serverless.aws_rds_cluster_instance.this["one"]'
# module.postgres_aurora_serverless.aws_rds_cluster_instance.this["one"]:
resource "aws_rds_cluster_instance" "this" {
arn                                   = "arn:aws:rds::db:test-rds-versioning-test-one"
auto_minor_version_upgrade            = true
availability_zone                     = "us-east-1b"

engine                                = "aurora-postgresql"
engine_version                        = "13.13"
engine_version_actual                 = "13.13"

}


3. Change engine_version on the configuration file

https://github.com/gayashankgithub/rds-version-testing/blob/new-branch-1/infrastructure/main.tf#L3

```hcl
locals {
  engine    = "aurora-postgresql"
  version   = "13"
}

Apply terraform.

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.postgres_aurora_serverless.aws_rds_cluster.this[0] will be updated in-place
  ~ resource "aws_rds_cluster" "this" {
      ~ engine_version                        = "13.13" -> "13"
        id                                    = "test-rds-versioning-test"
        tags                                  = {}
        # (43 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

  # module.postgres_aurora_serverless.aws_rds_cluster_instance.this["one"] will be updated in-place
  ~ resource "aws_rds_cluster_instance" "this" {
      ~ engine_version                        = "13.13" -> "13"
        id                                    = "test-rds-versioning-test-one"
        tags                                  = {}
        # (27 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.postgres_aurora_serverless.aws_rds_cluster_instance.this["two"] will be updated in-place
  ~ resource "aws_rds_cluster_instance" "this" {
      ~ engine_version                        = "13.13" -> "13"
        id                                    = "test-rds-versioning-test-two"
        tags                                  = {}
        # (27 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

Plan: 0 to add, 3 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.postgres_aurora_serverless.aws_rds_cluster.this[0]: Modifying... [id=test-rds-versioning-test]

Error: updating RDS Cluster (test-rds-versioning-test): operation error RDS: ModifyDBCluster, https response error StatusCode: 400, RequestID: 0d470073-267b-4c1f-bd6c-b4dbd2ff1e66, api error InvalidParameterCombination: Cannot upgrade aurora-postgresql from 13.13 to 13.12

  with module.postgres_aurora_serverless.aws_rds_cluster.this[0],
  on .terraform/modules/postgres_aurora_serverless/main.tf line 61, in resource "aws_rds_cluster" "this":
  61: resource "aws_rds_cluster" "this" {

Releasing state lock. This may take a few moments...

bryantbiggs commented 3 months ago

if you are telling RDS to auto upgrade the minor version, then you should not specify the minor version for the engine - you only specify the major version and from then on, RDS will automatically upgrade it for you

But the fact still remains that we are not going to ignore the engine version in this module - there are users that want to set the full version and have strict control over the version through Terraform. It is up to you to get your code and infra aligned so that they work together, we cannot do that here fore you

gayashankgithub commented 3 months ago

Yes. Same for me, I need to specify the major version. The workaround would be removing the minor version and running Terraform a couple of times until the state gets updated or refresh terraform.

github-actions[bot] commented 2 months ago

This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 10 days

github-actions[bot] commented 2 months ago

This issue was automatically closed because of stale in 10 days

github-actions[bot] commented 1 month ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

terraform-aws-modules / terraform-aws-rds-aurora