search

hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.75k stars 9.1k forks source link

[Provider]: Implement exclusive relationship management resources #39376

Open jar-b opened 2 days ago

jar-b commented 2 days ago

Description

This meta issue will track the progress of the effort to implement "exclusive relationship management" resources, as described in this proposal (see the rendered version in the contributor guide).

IAM inline policies:

IAM customer managed policies:

VPC security groups:

References

Relates #39203

Would you like to implement a fix?

None

github-actions[bot] commented 2 days ago

Community Note

Voting for Prioritization

Volunteering to Work on This Issue

lorengordon commented 21 hours ago

I love this sooooo much! Here's another related feature request:

lorengordon commented 21 hours ago

Routes in a route table is another use case for exclusive management.

lorengordon commented 21 hours ago

Commenting here for visibility since the proposal is already approved and merged, but what is the expected config migration path for practitioners? For example, if I am currently using aws_iam_role with the inline_policy argument exactly for the exclusive management feature, how do I move to the new pattern without impacting existing resources?

yermulnik commented 20 hours ago

Commenting here for visibility since the proposal is already approved and merged, but what is the expected config migration path for practitioners? For example, if I am currently using aws_iam_role with the inline_policy argument exactly for the exclusive management feature, how do I move to the new pattern without impacting existing resources?

+1 It's been for years that we've been enforcing inline_policy blocks inside aws_iam_role resource to ensure exclusive inline policies management by TF and now we're to migrate (back?) to aws_iam_role_policy with auxiliary aws_iam_role_policies_exclusive which is to manage at least two standalone resources outside aws_iam_role including policies being decoupled from the IAM Role resource to decrease visibility 😢

yermulnik commented 20 hours ago

I believe there's a good reasoning behind deprecating inline_policy in favour of when one looks into aws_iam_role resource and sees no inline policies attached because they are managed by at least two other standalone resources and each may be a part of separate .tf file 🤷🏻

lorengordon commented 4 hours ago

Commenting here for visibility since the proposal is already approved and merged, but what is the expected config migration path for practitioners? For example, if I am currently using aws_iam_role with the inline_policy argument exactly for the exclusive management feature, how do I move to the new pattern without impacting existing resources?

I went ahead and demoed a quick change to our iam-principals module to see what the migration path looks like currently. I think it needs some work. It's going to be a big lift, without any way to move a resource created by an inline_policy block to the aws_iam_role_policy resource.

PR here: https://github.com/plus3it/terraform-aws-tardigrade-iam-principals/pull/205

To see what I mean:

  1. checkout the default branch for that project
  2. run init/apply in tests/create_roles/prereq
  3. run init/apply in tests/create_roles
  4. checkout the branch feat/new-exclusive-pattern
  5. run plan in tests/create_roles

You'll see that it wants to create the inline role policies again.

Of course, a user could write import blocks to import all the inline policies, but that's a little nuts. It would be amazing if I could write a moved block for the module, where the from is actually the inline_policy block on the aws_iam_role resource, and the to is the new aws_iam_role_policy resource.

Without such a moved feature, I'd have to consider this a breaking change for the module. 😢

@jar-b Thoughts?

lorengordon commented 4 hours ago

Also, managed prefix lists and their entries are another resource that could benefit from this new exclusive relationship pattern...

jar-b commented 1 hour ago

To migrate an existing inline policy to the standalone resource, an import block (or alternatively a manual import via the Terraform CLI) is required. Unfortunately a moved block cannot be used, as the shape of the two resources (aws_iam_role and aws_iam_role_policy) do not match.

However, because the inline_policy argument is Optional/Computed (meaning removal of this argument will not trigger deletion of inline policy, just change it to track as computed only), this step can be accomplished in a single apply.

Here's a small example (the configuration is hidden by default to improve readability):

Show/Hide Configuration The uncommented configuration below represents the existing state where inline policies are defined directly on the `aws_iam_role` resource. ```terraform terraform { required_providers { aws = { source = "hashicorp/aws" version = "~> 5.0" } } } # Configure the AWS Provider provider "aws" {} data "aws_iam_policy_document" "trust" { statement { actions = ["sts:AssumeRole"] principals { type = "Service" identifiers = ["ec2.amazonaws.com"] } } } data "aws_iam_policy_document" "inline" { statement { actions = ["s3:ListBucket"] resources = [ "arn:aws:s3:::some-bucket-a", "arn:aws:s3:::some-bucket-b", ] } } resource "aws_iam_role" "test" { name = "jb-test-inline-policy-migration" assume_role_policy = data.aws_iam_policy_document.trust.json # Remove this when the new standalone resource is added inline_policy { name = "test-inline" policy = data.aws_iam_policy_document.inline.json } } ### Start of new resources ### # # This can be removed after the apply in which the resource is imported # import { # to = aws_iam_role_policy.test # id = "jb-test-inline-policy-migration:test-inline" # } # # resource "aws_iam_role_policy" "test" { # name = "test-inline" # role = aws_iam_role.test.name # policy = data.aws_iam_policy_document.inline.json # } # # # resource "aws_iam_role_policies_exclusive" "test" { # role_name = aws_iam_role.test.name # policy_names = [ # aws_iam_role_policy.test.name, # ] # } ### End of new resources ### ```

To covert this to use the standalone aws_iam_role_policy resource, do the following:

  1. Remove the inline_policy block.
  2. Uncomment the ### Start of new resources ### section.
  3. terraform apply. There should be 1 resource to import and 1 to add.
% terraform apply -auto-approve
data.aws_iam_policy_document.inline: Reading...
data.aws_iam_policy_document.trust: Reading...
data.aws_iam_policy_document.inline: Read complete after 0s [id=748695472]
data.aws_iam_policy_document.trust: Read complete after 0s [id=2851119427]
aws_iam_role.test: Refreshing state... [id=jb-test-inline-policy-migration]
aws_iam_role_policy.test: Preparing import... [id=jb-test-inline-policy-migration:test-inline]
aws_iam_role_policy.test: Refreshing state... [id=jb-test-inline-policy-migration:test-inline]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_iam_role_policies_exclusive.test will be created
  + resource "aws_iam_role_policies_exclusive" "test" {
      + policy_names = [
          + "test-inline",
        ]
      + role_name    = "jb-test-inline-policy-migration"
    }

  # aws_iam_role_policy.test will be imported
    resource "aws_iam_role_policy" "test" {
        id          = "jb-test-inline-policy-migration:test-inline"
        name        = "test-inline"
        name_prefix = null
        policy      = jsonencode(
            {
                Statement = [
                    {
                        Action   = "s3:ListBucket"
                        Effect   = "Allow"
                        Resource = [
                            "arn:aws:s3:::some-bucket-b",
                            "arn:aws:s3:::some-bucket-a",
                        ]
                    },
                ]
                Version   = "2012-10-17"
            }
        )
        role        = "jb-test-inline-policy-migration"
    }

Plan: 1 to import, 1 to add, 0 to change, 0 to destroy.
aws_iam_role_policy.test: Importing... [id=jb-test-inline-policy-migration:test-inline]
aws_iam_role_policy.test: Import complete [id=jb-test-inline-policy-migration:test-inline]
aws_iam_role_policies_exclusive.test: Creating...
aws_iam_role_policies_exclusive.test: Creation complete after 1s

Apply complete! Resources: 1 imported, 1 added, 0 changed, 0 destroyed.
  1. After a successful apply, the import block can be removed, and there are no planned changes.
% terraform plan
data.aws_iam_policy_document.trust: Reading...
data.aws_iam_policy_document.inline: Reading...
data.aws_iam_policy_document.trust: Read complete after 0s [id=2851119427]
data.aws_iam_policy_document.inline: Read complete after 0s [id=748695472]
aws_iam_role.test: Refreshing state... [id=jb-test-inline-policy-migration]
aws_iam_role_policy.test: Refreshing state... [id=jb-test-inline-policy-migration:test-inline]
aws_iam_role_policies_exclusive.test: Refreshing state...

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.
jar-b commented 1 hour ago

In theory if all previous versions of the module guarantee the inline policies will already exist, you could publish a version containing the import directive in a non-breaking way. However, a breaking major version change may be preferable to avoid cases where users need to sequence minor versions in such a way to guarantee the import conditions are satisfied.

The terraform-aws-s3-bucket module may provide some precedent for this in how it handled breaking arguments which were previously defined inline on the aws_s3_bucket resource into their own standalone resources between v2 and v3. https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/blob/v4.1.2/UPGRADE-3.0.md