[BUG]: rds instance "new" ID-sniffing approach doesn't work very well with restricted IAM policies

Description

Having just upgraded from provider 4.x to 5.x (5.9.0 to be precise, with terraform 1.5.2), we noticed we're now unable to terraform destroy our aws_db_instances:

Error: reading RDS DB Instance (abc-foo): AccessDenied: User: arn:aws:sts::999999999999:assumed-role/foo-lite/i-0aaaaaaaaaaaaaaaa is not authorized to perform: rds:DescribeDBInstances on resource: arn:aws:rds:eu-west-1:999999999999:db:db-aaaaaaaaaaaaaaaaaaaaaaaaa because no identity-based policy allows the rds:DescribeDBInstances action

It looks like this is a result of the "NEW ID configuration" in #31232, and in particular the function(s) used to "sniff" the correct identifier for the db instance:

https://github.com/hashicorp/terraform-provider-aws/blob/db5ffa9e789f63d56a1f3e1d3b4fb08dde0981b3/internal/service/rds/instance.go#L2370

The issue being that we're running terraform under a role that has the (IMHO reasonable) restriction only allowing:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "rds:*"
      ],
      "Resource": [
        "arn:aws:rds:*:*:*:*-foo",
        "arn:aws:rds:*:*:*:*-foo*"
      ]
    }
  ]
}

So it is prevented from doing the first lookup in findDBInstanceByID*, because despite the dbi-resource-id filter, IAM considers this a rds.DescribeDBInstances on arn:aws:rds:eu-west-1:999999999999:db:*. It then falls back to look it up trying to find out if the ID is a DBInstanceIdentifier, resulting in the 403 that gets presented to the user.

In the short term I suspect we're going to have to widen our role policies to allow this to work, but it doesn't seem ideal. Don't know if you'll consider this a bug, but it's certainly a heads up for anyone in a similar situation.

Perhaps the bug is that the first request's failure isn't detected as a 403, causing it to continue the sniffing process and ultimately resulting in an error message that's more confusing than it should be ("wtf why is it trying to use the resource_id in the arn??").

References

No response

Would you like to implement a fix?

None

hashicorp / terraform-provider-aws