bug in aws_rds_cluster_parameter_group with two parameters - ssl and rds.force_ssl

midestefanis commented 2 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v1.1.6

Affected Resource(s)

aws_rds_cluster_parameter_group

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "aws_rds_cluster_parameter_group" "param" {
  name        = "${var.name}-${var.family}-cluster-parameter-group"
  family      = var.family
  description = "${var.name}-${var.family}-cluster-parameter-group"

  dynamic "parameter" {
    for_each = var.cluster_parameters
    content {
      name         = parameter.value.name
      value        = parameter.value.value
      apply_method = parameter.value.apply_method
    }
  }

  tags = local.tags
}

input:

cluster_parameters = [
    {
      name         = "idle_in_transaction_session_timeout",
      value        = "0",
      apply_method = "pending-reboot"
    },
    {
      name         = "statement_timeout",
      value        = "0",
      apply_method = "pending-reboot"
    },
    {
      name         = "orafce.timezone",
      value        = "America/Argentina/Buenos_Aires",
      apply_method = "pending-reboot"
    },
    {
      name         = "timezone",
      value        = "America/Argentina/Buenos_Aires",
      apply_method = "pending-reboot"
    },
    {
      name         = "cron.database_name",
      value        = "db_name",
      apply_method = "pending-reboot"
    },
    {
      name         = "rds.force_ssl",
      value        = "0",
      apply_method = "pending-reboot"
    },
    {
      name         = "ssl",
      value        = "1",
      apply_method = "pending-reboot"
    },

  ]

Debug Output

Output of a 'terraform plan':


Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the
last "terraform apply":

  # aws_rds_cluster_parameter_group.param has changed
  ~ resource "aws_rds_cluster_parameter_group" "param" {
        id          = "****"
        name        = "****"
        tags        = {
            "Teco"        = "PePa"
            "created-by"  = "terraform"
            "environment" = "dev"
            "owners"      = "team-devops"
        }
        # (4 unchanged attributes hidden)

      - parameter {
          - apply_method = "pending-reboot" -> null
          - name         = "rds.force_ssl" -> null
          - value        = "0" -> null
        }
      - parameter {
          - apply_method = "pending-reboot" -> null
          - name         = "ssl" -> null
          - value        = "1" -> null
        }
        # (5 unchanged blocks hidden)
    }

Unless you have made equivalent changes to your configuration, or ignored the
relevant attributes using ignore_changes, the following plan may include
actions to undo or respond to these changes.

─────────────────────────────────────────────────────────────────────────────

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_rds_cluster_parameter_group.param will be updated in-place
  ~ resource "aws_rds_cluster_parameter_group" "param" {
        id          = "****"
        name        = "****"
        tags        = {
        }
        # (4 unchanged attributes hidden)

      + parameter {
          + apply_method = "pending-reboot"
          + name         = "rds.force_ssl"
          + value        = "0"
        }
      + parameter {
          + apply_method = "pending-reboot"
          + name         = "ssl"
          + value        = "1"
        }
        # (5 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.

Panic Output

Expected Behavior

The plan should not show me that output because those parameters in the console look well applied. It only happens with these two.

Actual Behavior

Terraform detects as null only those two parameters (ssl and rds.force_ssl) and tries to correct it.

evanstachowiak commented 2 years ago

I also have this issue with the log_output parameter

seanamos commented 1 year ago

I have this as well with track_activity_query_size and track_io_timing. From the console, it looks like they are applied but terraform always detects them as changes.

EDIT: Strangely, I do not have this problem with ssl and rds.force_ssl, those work fine.

seanamos commented 1 year ago

I dug through the source code for this to figure out what was going on.

aws_rds_cluster_parameter_group filters out DB parameters returned by the AWS API to those with a Source of user. This makes sense as an optimization so Terraform doesn't have to do a bunch of unnecessary paging API calls to AWS, we only care about the parameters we've modified. However......

A DB parameter will have its Source set to user if it is modified from the default. If you set a DB parameter to the same value as its default, the Source is never modified. Source is set by AWS.

In my case for my params, and @midestefanis case, we are setting the same value as the default for the parameter group, so it never sets the Source to user on those parameters. The DB parameters are filtered out by aws_rds_cluster_parameter_group's paging API calls and they are constantly detected as "new" parameters.

@midestefanis The defaults in the cluster parameter group for postgres 14 are: rds.force_ssl = 0 ssl = 1

So they are always picked up by Terraform as "new".

Without changing aws_rds_cluster_parameter_group to make a bunch of additional API requests, the only workaround for now is to not specify parameters in aws_rds_cluster_parameter_group that are the same as the defaults.

simi commented 1 year ago

same here, I see change of track_io_timing everytime, for example following

      + parameter {
          + apply_method = "immediate"
          + name         = "track_io_timing"
          + value        = "1"
        }
      - parameter {
          - apply_method = "pending-reboot" -> null
          - name         = "track_io_timing" -> null
          - value        = "1" -> null
        }

sahulyousuf commented 5 months ago

We also have been experiencing same issue with aws_db_parameter_group for the last few days, the terraform version is 1.5.

  - parameter {
      - apply_method = "pending-reboot" -> null
      - name         = "rds.force_ssl" -> null
      - value        = "1" -> null
    }

  + parameter {
      + apply_method = "immediate"
      + name         = "rds.force_ssl"
      + value        = "1"
    }
  - parameter {
      - apply_method = "pending-reboot" -> null
      - name         = "autovacuum_vacuum_scale_factor" -> null
      - value        = "0.05" -> null
    }
  - parameter {
      - apply_method = "pending-reboot" -> null
      - name         = "wal_sender_timeout" -> null
      - value        = "0" -> null
    }
  + parameter {
      + apply_method = "immediate"
      + name         = "autovacuum_vacuum_scale_factor"
      + value        = "0.05"
    }
  + parameter {
      + apply_method = "immediate"
      + name         = "wal_sender_timeout"
      + value        = "0"
    }

philomory commented 2 months ago

It'd be worse than just adding a bunch of extra API calls, if they didn't filter down to only the user-sourced parameters, because then when the provider passed to Terraform core the list of hundreds of parameter values that it received from the API, but which you didn't specify in your manifest, Terraform core would plan to delete them, every run. It'd be nice if Amazon just had a sane API and allowed setting source to user when a user explicitly specifies a value for a parameter, even if that value happens to be the default value. But given that the API doesn't work that way, I don't think this is actually possible to fix (at least, not using the Terraform Plugin SDKv2 that the aws provider is built on; possibly one of the newer provider frameworks have features that would help here, like "plan modifiers").

EDIT: I take it back, someone actually came up with a fix for this back in 2018, but they only applied it to the aws_rds_db_parameter_group resource, not the aws_rds_cluster_parameter_group resource.

hashicorp / terraform-provider-aws