[Bug]: RDS change from id to identifier makes restricting access to a DB instance impossible

Terraform Core Version

1.5

AWS Provider Version

5.36

Affected Resource(s)

aws_db_instance Going through upgrading the AWS provider in a module that provisions an RDS instance from 4.65 to 5.36 and were presented with the error:

Error: reading RDS DB Instance (xxx-rt-xxx): AccessDenied: User: arn:aws:sts::111111111111:assumed-role/TerraformDB/terraform-db is not authorized to perform: rds:DescribeDBInstances on resource: arn:aws:rds:us-east-1:111111111111:db:db-<randomID> because no identity-based policy allows the rds:DescribeDBInstances action

We use dedicated IAM roles to provision resources within a given module, which has not changed during the upgrade of provider. The policy does have the rds:DescribeDBInstances permission, which is restricting access to DBs that this module creates:

    {
        "Action": [
            "rds:RemoveTagsFromResource",
            "rds:ModifyDBSubnetGroup",
            "rds:ModifyDBParameterGroup",
            "rds:ModifyDBInstance",
            "rds:ListTagsForResource",
            "rds:DescribeDBSubnetGroups",
            "rds:DescribeDBParameters",
            "rds:DescribeDBParameterGroups",
            "rds:DescribeDBInstances",
            "rds:DeleteDBSubnetGroup",
            "rds:DeleteDBParameterGroup",
            "rds:DeleteDBInstance",
            "rds:CreateDBSubnetGroup",
            "rds:CreateDBParameterGroup",
            "rds:CreateDBInstance",
            "rds:AddTagsToResource"
        ],
        "Effect": "Allow",
        "Resource": [
            "arn:aws:rds:*:111111111111:subgrp:*-rt-*",
            "arn:aws:rds:*:111111111111:pg:*-rt-*",
            "arn:aws:rds:*:111111111111:db:*-rt-*"
        ]
    },

This appears to be related to the changes brought in as part of the 5.0.0 release (#31232). The only way to mitigate this is to wildcard the resource in the policy, which somewhat seems to defeat the purpose of being able to limit access to given resources. This may be related to the issue described in #32732 (although they talk of it being down to the IAM sniffing adding in a later version).

I've verified that it's the 5.0.0 release by testing 4.67.0 (no problem) and then going up to 5.0.0 (problem occurs). The confusing bit for me is that the instance ARN still shows the instance name not the random ID, i.e. ARN = arn:aws:rds:us-east-1:111111111111:db:foo-rt-bar, which would match the resource definition in the IAM policy.

Expected Behavior

We should be able to restrict access to certain DB instances using IAM policies

Actual Behavior

We've had to wildcard the IAM policy to allow access to effectively any DB instance in the same account/region

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

IAM role:

data "aws_iam_policy_document" "assume_role_policy" {
  statement {
    effect  = "Allow"
    actions = ["sts:AssumeRole"]

    principals {
      identifiers = ["arn:aws:iam::111111111111:root"]
      type        = "AWS"
    }
  }
}

locals {
    rds_actions = [
    "rds:AddTagsToResource",
    "rds:CreateDBInstance",
    "rds:CreateDBParameterGroup",
    "rds:CreateDBSubnetGroup",
    "rds:DeleteDBInstance",
    "rds:DeleteDBParameterGroup",
    "rds:DeleteDBSubnetGroup",
    "rds:DescribeDBInstances",
    "rds:DescribeDBParameterGroups",
    "rds:DescribeDBParameters",
    "rds:DescribeDBSubnetGroups",
    "rds:ListTagsForResource",
    "rds:ModifyDBInstance",
    "rds:ModifyDBParameterGroup",
    "rds:ModifyDBSubnetGroup",
    "rds:RemoveTagsFromResource",
  ]

  rds_resources = [
    "arn:aws:rds:*:111111111111:db:*",
    "arn:aws:rds:*:111111111111:pg:*-rt-*",
    "arn:aws:rds:*:111111111111:subgrp:*-rt-*",
  ]
}

data "aws_iam_policy_document" "policy" {
  statement {
    actions   = local.rds_actions
    resources = local.rds_resources
  }
}

resource "aws_iam_role" "role" {
  name               = TerraformDB
  description        = "Role to be assumed by Terraform when deploying the module"
  assume_role_policy = data.aws_iam_policy_document.assume_role_policy.json
}

main:

provider "aws" {
  profile = terraform
  region  = "us-east-1"

  assume_role {
    role_arn     = "arn:aws:iam::111111111111:role/TerraformDB"
    session_name = "${var.name}-${terraform.workspace}"
  }
}

terraform {
  required_version = "~> 1.5"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

resource "random_password" "db_initial_master_password" {
  length  = 20
  special = false
}

resource "aws_db_instance" "db" {
  identifier                      = "blah-rt-db
  engine                          = "mariadb"
  engine_version                  = "10.6"
  allocated_storage               = 20
  instance_class                  = "db.t3.small"
  storage_type                    = "gp3"
  multi_az                        = true
  port                            = 3306
  username                        = "root"
  password                        = random_password.db_initial_master_password.result
  parameter_group_name            = aws_db_parameter_group.db_maria.name
  vpc_security_group_ids          = [aws_security_group.db.id]
  storage_encrypted               = true

  tags = {
    Name = "blah-rt-db"
  }
}

resource "aws_db_parameter_group" "core_db_maria" {
  name_prefix = "foo-db-mariadb"
  family      = "mariadb10.6"
  description = "Parameter group for MariaDB instance"

  parameter {
    name  = "binlog_format"
    value = "ROW"
  }
}

Steps to Reproduce

Create an RDS DB instance using the AWS provider v4.67.0 with an IAM policy that specifies the instance name being created in the resources section
Update the provider to 5.0.0
Re-apply and the plan/apply will fail with the above error

Debug Output

In case it's useful, the sate on 4.67.0 contains:

terraform state show aws_db_instance.db
# aws_db_instance.db:
resource "aws_db_instance" "db" {
...
    arn                                   = "arn:aws:rds:us-east-2:111111111111:db:foo-rt-bar"
    id                                    = "foo-rt-bar"
    identifier                            = "foo-rt-bar"
    identifier_prefix                     = ""

After the upgrade to 5.0.0, it shows:

terraform state show aws_db_instance.db
# aws_db_instance.db:
resource "aws_db_instance" "db" {
...
    arn                                   = "arn:aws:rds:us-east-2:111111111111:db:foo-rt-bar"
    id                                    = "db-<randomID>"
    identifier                            = "foo-rt-bar"

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

hashicorp / terraform-provider-aws