terraform shows forever delta with (known after apply) for iam-role-for-service-accounts-eks module calls

mikalinnanoja-rovio commented 2 weeks ago

Description

edit: updated used module to iam-role-for-service-accounts-eks, no difference to outcome however.

Hi, I have a module that sets up infra prerequisites for a kubernetes application that includes module call for the submodule iam-assumable-role-with-oidc from this repo to use in AWS EKS with IRSA (thx for the module :bowing_man:).

The module, tested up to latest version 5.40.0 as of writing this, works nicely and produces nice IAM roles our applications can assume inside kubernetes cluster, but terraform is showing this in plan every time for each of our services, making the plan painful to go through:

  # module.services["dummy"].module.iam_eks_role.data.aws_caller_identity.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_caller_identity" "current" {
      + account_id = (known after apply)
      + arn        = (known after apply)
      + id         = (known after apply)
      + user_id    = (known after apply)
    }

  # module.services["dummy"].module.iam_eks_role.data.aws_iam_policy_document.this[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_iam_policy_document" "this" {
      + id   = (known after apply)
      + json = (known after apply)

      + statement {
          + actions = [
              + "sts:AssumeRoleWithWebIdentity",
            ]
          + effect  = "Allow"

          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "sts.amazonaws.com",
                ]
              + variable = "oidc.eks.us-east-1.amazonaws.com/id/DUMMY_ID:aud"
            }
          + condition {
              + test     = "StringEquals"
              + values   = [
                  + "system:serviceaccount:dummy:dummy",
                ]
              + variable = "oidc.eks.us-east-1.amazonaws.com/id/DUMMY_ID:sub"
            }

          + principals {
              + identifiers = [
                  + "arn:aws:iam::ACC_ID:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/DUMMY_ID",
                ]
              + type        = "Federated"
            }
        }
    }

  # module.services["dummy"].module.iam_eks_role.data.aws_partition.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_partition" "current" {
      + dns_suffix         = (known after apply)
      + id                 = (known after apply)
      + partition          = (known after apply)
      + reverse_dns_prefix = (known after apply)
    }

  # module.services["dummy"].module.iam_eks_role.data.aws_region.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_region" "current" {
      + description = (known after apply)
      + endpoint    = (known after apply)
      + id          = (known after apply)
      + name        = (known after apply)
    }

  # module.services["dummy"].module.iam_eks_role.aws_iam_role.this[0] will be updated in-place
  ~ resource "aws_iam_role" "this" {
      ~ assume_role_policy    = jsonencode(
            {
              - Statement = [
                  - {
                      - Action    = "sts:AssumeRoleWithWebIdentity"
                      - Condition = {
                          - StringEquals = {
                              - "oidc.eks.us-east-1.amazonaws.com/id/DUMMY_ID:aud" = "sts.amazonaws.com"
                              - "oidc.eks.us-east-1.amazonaws.com/id/DUMMY_ID:sub" = "system:serviceaccount:dummy:dummy"
                            }
                        }
                      - Effect    = "Allow"
                      - Principal = {
                          - Federated = "arn:aws:iam::ACC_ID:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/DUMMY_ID"
                        }
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> (known after apply)
        id                    = "dummy-20240104150621270300000002"
        name                  = "dummy-20240104150621270300000002"
        tags                  = {}
        # (9 unchanged attributes hidden)
    }

In above plan output I replaced all real IDs with dummy strings.

When applied, nothing actually changes. But this is quite verbose hence having this kind of thing in plan x 100 makes the plan basically useless. How can we avoid this sort of forever-delta? I've tried passing fixed string provider_url (oidc url from amazon) instead of getting it from EKS module call outputs, and that at least doesn't solve this. None of the dependant resources have any counts or for_eaches.

Our module call looks like this:

module "iam_eks_role" {
  source           = "terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks"
  version          = "5.40.0"
  role_name_prefix = "${var.service_name}-"
  role_policy_arns = {
    policy = aws_iam_policy.service.arn
  }
  oidc_providers = {
    sa = {
      provider_arn               = var.eks_oidc_provider_arn
      namespace_service_accounts = ["${var.service_namespace}:${var.service_name}"]
    }
  }
}

I found one issue from 2022 that had similar findings, but it got closed. Thanks!

[X] ✋ I have searched the open/closed issues and my issue is not listed.

Versions

Module version [Required]: 5.40.0
Terraform version: Terraform v1.5.7 on linux_amd64
Provider version(s):
provider registry.terraform.io/hashicorp/aws v5.42.0
provider registry.terraform.io/hashicorp/cloudinit v2.3.4
provider registry.terraform.io/hashicorp/external v2.3.3
provider registry.terraform.io/hashicorp/helm v2.12.1
provider registry.terraform.io/hashicorp/kubernetes v2.27.0
provider registry.terraform.io/hashicorp/local v2.5.1
provider registry.terraform.io/hashicorp/null v3.2.2
provider registry.terraform.io/hashicorp/random v3.6.2
provider registry.terraform.io/hashicorp/time v0.11.2
provider registry.terraform.io/hashicorp/tls v4.0.5

Reproduction Code [Required]

Steps to reproduce the behavior: Module call with given kind of input. Everything applied and working.

Not using workspaces.

Issue reproducible in CI system and local computers for multiple persons, not caching related.

Expected behavior

No delta i.e "No infrastructure changes" in module created role IAM trust policy if nothing changes.

Actual behavior

Given delta always present in plan.

bryantbiggs commented 2 weeks ago

I think you are looking for the https://github.com/terraform-aws-modules/terraform-aws-iam/tree/master/modules/iam-role-for-service-accounts-eks module - I would switch to that module to see if it resolves your issue

mikalinnanoja-rovio commented 2 weeks ago

I think you are looking for the https://github.com/terraform-aws-modules/terraform-aws-iam/tree/master/modules/iam-role-for-service-accounts-eks module - I would switch to that module to see if it resolves your issue

Hmm interesting, looks like this newer module would be a good fit. Will give it a whirl :) Thanks a bunch.

bryantbiggs commented 2 weeks ago

and if you wan to use EKS Pod Identity instead of IRSA (Pod Identity is the next evolution after IRSA) https://github.com/terraform-aws-modules/terraform-aws-eks-pod-identity

mikalinnanoja-rovio commented 2 weeks ago

Sadly changing module didn't help for this issue. I'll update the plan and module call references but otherwise issue remains. I think it was good to migrate the module anyway :)

terraform-aws-modules / terraform-aws-iam