[Bug]: Updating aws_db_instance to set manage_master_user_password to true and accessing master_user_secret[0] attributes fails because the attribute is not deferred until after apply

[gchristidis]

Terraform Core Version

1.5.3

AWS Provider Version

5.22

Affected Resource(s)

• aws_db_instance

Expected Behavior

Accessing master_user_secret[0] attributes should be deferred until after apply when changing manage_master_user_password=true on an aws_db_instance

Actual Behavior

Plan fails Accessing master_user_secret[0] as there is no elements in the list on the current aws_db_instance

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

provider "aws" {
}

terraform {
  required_version = "> 1"
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "~> 5.0" 
    }
  }
}

variable "known_password" {
  type = string
  default = null
}

resource "aws_db_instance" "db" {
  identifier_prefix                     = "test-master-passwd"
  engine                                = "postgres"
  engine_version                        = "15"
  instance_class                        = "db.t4g.micro"
  allocated_storage                     = 10
  db_name                               = "test"

  username                              = "dbmgr"
  password                              = var.known_password
  manage_master_user_password           = var.known_password == null ? true : null

  skip_final_snapshot                   = true
  apply_immediately                     = true
}

# create secret & version with known password if using
resource "aws_secretsmanager_secret" "known_password" {
  count                   = var.known_password != null ? 1 : 0
  name                    = "/test-master-passwd/known-master"
  description             = "Known Master credentials"
  recovery_window_in_days = 0
}

resource "aws_secretsmanager_secret_version" "known_password" {
  count         = var.known_password != null ? 1 : 0
  secret_id     = aws_secretsmanager_secret.known_password[0].id
  secret_string = jsonencode({
    username = aws_db_instance.db.username,
    password = aws_db_instance.db.password,
  })
}

# publish master ARN to SSM parameter for single point to get master creds in application
resource "aws_ssm_parameter" "master_arn" {
  name        = "/test-master-passwd/master_arn"
  description = "The ARN of the Secret containing the master credentials for RDS"
  type        = "String"
  value       = var.known_password == null ? aws_db_instance.db.master_user_secret[0].secret_arn : aws_secretsmanager_secret_version.known_password[0].arn
}

Steps to Reproduce

Create instance with a specified password terraform apply -var known_password=known_pass123 and it will created the RDS instance with the known password, create a secret to hold the known password credentials and create the SSM parameter with the ARN of the known password secret.

Now update the instance to use managed master password by using the default value for known_password terraform apply and the plan fails with the error

│ Error: Invalid index
│ 
│   on main.tf line 71, in resource "aws_ssm_parameter" "master_arn":
│   71:   value       = var.known_password == null ? aws_db_instance.db.master_user_secret[0].secret_arn : aws_secretsmanager_secret_version.known_password[0].arn
│     ├────────────────
│     │ aws_db_instance.db.master_user_secret is empty list of object
│ 
│ The given key does not identify an element in this collection value: the collection has no elements.

The other direction works fine, you can create an RDS instance with managed master password terraform apply and then update it to using a known_password terraform apply -var known_password=known_pass123 and it updates as expected.

Debug Output

No response

Panic Output

No response

Important Factoids

This test snipped is a simplified example of what we have but shows the issue. Our RDS instances have their passwords set by a Terraform variable and we want to move them to RDS managed, in either case the Master Secret ARN needs to be in an SSM parameter for and so this makes this impossible.

I have worked around the issue by using an aws_db_instance data source to read aws_db_instance.db.identifier, meaning the data source read is deferred until after apply and using that in the SSM parameter value. Using the data source to reread the instance after modification allows the change to be applied both ways as the SSM parameter calculation is deferred.

provider "aws" {
}

terraform {
  required_version = "> 0.13"
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "~> 5.0" 
    }
  }
}

variable "known_password" {
  type = string
  default = null
}

resource "aws_db_instance" "db" {
  # identifier cant have '.' so replace them with '-' in module_name
  identifier_prefix                     = "test-master-passwd"
  engine                                = "postgres"
  engine_version                        = "15"
  instance_class                        = "db.t4g.micro"
  allocated_storage                     = 10
  db_name                               = "test"

  username                              = "dbmgr"
  password                              = var.known_password
  manage_master_user_password           = var.known_password == null ? true : null

  skip_final_snapshot                   = true
  apply_immediately                     = true
}

data "aws_db_instance" "db" {
  db_instance_identifier = aws_db_instance.db.identifier
}

# create secret & version with known password if using
resource "aws_secretsmanager_secret" "known_password" {
  count                   = var.known_password != null ? 1 : 0
  name                    = "/test-master-passwd/known-master"
  description             = "Known Master credentials"
  recovery_window_in_days = 0
}

resource "aws_secretsmanager_secret_version" "known_password" {
  count         = var.known_password != null ? 1 : 0
  secret_id     = aws_secretsmanager_secret.known_password[0].id
  secret_string = jsonencode({
    username = aws_db_instance.db.username,
    password = aws_db_instance.db.password,
  })
}

# publish master ARN to SSM parameter for single point to get master creds in application
resource "aws_ssm_parameter" "master_arn" {
  name        = "/test-master-passwd/master_arn"
  description = "The ARN of the Secret containing the master credentials for RDS"
  type        = "String"
  value       = var.known_password == null ? data.aws_db_instance.db.master_user_secret[0].secret_arn : aws_secretsmanager_secret_version.known_password[0].arn
}

References

No response

Would you like to implement a fix?

None

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[bowczarek-converge]

I'm experiencing same issue, it fails on plan and is not deferred to apply phase.

[matanlb]

Having the same issue as well. Imported a DB that was created outside of Terraform. Plain fails with the error aforementioned, but succeeds if the we introduce a change that calls for the DB to be replaced. seems like the TF doesn't plan to create the managed master password for an existing DB

[AthenaFlora]

same issue

[JGSweets]

A workaround:

join("", aws_db_instance.db.master_user_secret.*.secret_arn)

[taraspos]

I'm facing this problem as well, but also get following error

╷
│ Error: Provider produced inconsistent final plan
│ 
│ When expanding the plan for
│ <redacted>.aws_iam_policy.<redacted>
│ to include new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .policy: was
│ cty.StringVal("{\"Statement\":[{\"Action\":[\"secretsmanager:GetSecretValue\"],\"Effect\":\"Allow\",\"Resource\":[\"arn:aws:secretsmanager:*:<redacted>:secret:/\"],\"Sid\":\"GetSecretValue\"}],\"Version\":\"2012-10-17\"}"),
│ but now
│ cty.StringVal("{\"Statement\":[{\"Action\":[\"secretsmanager:GetSecretValue\"],\"Effect\":\"Allow\",\"Resource\":[\"arn:aws:secretsmanager:<redacted>:<redacted>:secret:rds!cluster-<redacted>\"],\"Sid\":\"GetSecretValue\"}],\"Version\":\"2012-10-17\"}").
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.