feat(pc): add providerconfig example for aws eks pod identity feature

[haarchri]

Description of your changes

add providerconfig example for aws eks pod identity feature (https://aws.amazon.com/blogs/containers/amazon-eks-pod-identity-a-new-way-for-applications-on-eks-to-obtain-iam-credentials/) and additional hints for IAM Role and OpenIDConnectProvider realted to this convo: https://github.com/crossplane-contrib/provider-upjet-aws/issues/1252

Fixes #1249 #1252

I have:

• [ ] Read and followed Crossplane's contribution process.
• [ ] Run make reviewable to ensure this PR is ready for review.
• [ ] Added backport release-x.y labels to auto-backport this PR if necessary.

How has this code been tested

[wesleyorama2]

This helped me greatly with getting this up and running using pod identities. Please be sure to also include the findings in: https://github.com/crossplane-contrib/provider-upjet-aws/issues/1252 in any documentation as I did have to reference it for the trust policy changes that are needed for this.

[aiell0]

Hey there! So I recently dug into Pod Identity and Crossplane and was able to get the following configuration to work:

apiVersion: pkg.crossplane.io/v1beta1
kind: DeploymentRuntimeConfig
metadata:
  name: patch-service-account
spec:
  deploymentTemplate:
    spec:
      selector: {}
      template:
        spec:
          serviceAccountName: crossplane
          containers: []
---
apiVersion: aws.upbound.io/v1beta1
kind: ProviderConfig
metadata:
  name: prod
spec:
  credentials:
    source: IRSA
---
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-aws-s3
spec:
  package: xpkg.upbound.io/upbound/provider-aws-s3:v1.1.0
  runtimeConfigRef:
    name: patch-service-account
---
apiVersion: s3.aws.upbound.io/v1beta1
kind: Bucket
metadata:
  generateName: crossplane-bucket-
spec:
  forProvider:
    region: us-east-1
  providerConfigRef:
    name: prod

The crossplane service account is linked to a pod identity with a role that has Admin privileges (I was just trying to get this to work) and the following trust policy:

apiVersion: s3.aws.upbound.io/v1beta1
kind: Bucket
metadata:
  generateName: crossplane-bucket-
spec:
  forProvider:
    region: us-east-1
  providerConfigRef:
    name: prod

I only bring this up because it is a lot different from the example provided here in that I did not have to use a web identity at all. Is there something I am missing?

[danielloader]

@aiell0 interestingly your example works for me with aws upbound provider v1.1.0, but if I bump the provider to v1.4.0 it falls apart.

The error is: cannot initialize the Terraform plugin SDK async external client: cannot get terraform setup: cache manager failure: cannot calculate the hash for the credentials file: token file name cannot be empty

I think the issue is likely covered in the v1.3.0 release: https://github.com/crossplane-contrib/provider-upjet-aws/releases/tag/v1.3.0

This release also introduces a credential cache for IRSA authentication, which greatly reduces the number of AWS STS calls the provider makes. This cache is currently only employed for IRSA configurations. Please refer to the description https://github.com/crossplane-contrib/provider-upjet-aws/pull/1235#issue-2202869791 for the results of some experiments and the observed improvements in those experiments.

Can confirm 1.1.0, 1.2.1 works and 1.3.0, 1.4.0 don't.

Been banging my head on wall trying to understand why your example applied to the cluster worked a treat, and my own in argocd didn't work at all.

Given this, either the change is a regression, or the IRSA mode working with Pod Identity was a fluke (seems like it was) and the cache breaks it when you use the Pod Identity token, which is in a different format to the IRSA one.

We might need a dedicated source for PodIdentity.

Edit: Noticed this was raised above by wesleyorama2 but it doesn't change the fact it's a more explicit configuration of what in 1.2.1 just works implicitly - that is, binding a SA/Namespace to a Pod Identity, just implicitly seems to work with no further configuration - and I'd really like to maintain this decoupling of the IAC that provisions an EKS cluster (terraform in my case) and the AWS Upbound provider running in the cluster, if possible.

Longer term I'd like to deprecate IRSA auth completely in my EKS environments, and potentially drop OIDC at the same time if possible and just move everything into this new paradigm.

[haarchri]

let's double check the Issues with 1.3 and 1.4 and include in the fix an working example without any workaround for hardcoded filesystem files

[danielloader]

Thanks for looking at this for us.

[truongnht]

@haarchri I am curious if you have update on this issue 🤔

[quanlk2511]

Hi @haarchri , guys, do we have progress on this?