IRSA on CAPA clusters

[alex-dabija]

Story

-As a cluster admin, I want to have IAM Roles for Service Accounts (IRSA) running on CAPA clusters in order to simplify authentication and authorization to AWS services by applications.

Background

Some applications need to access AWS services in order to work properly. IRSA simplifies the authentication and authorization of such applications to AWS services by allowing them to just assume roles (of course it's more complicated then that).

One such application is the aws-network-topology-operator.

Changes

• [x] Deploy IRSA to CAPA clusters;
• [x] Make IRSA a default application on CAPA clusters;
• [x] Ensure IRSA is running also on management clusters;

Resources

• Giant Swarm IRSA documentation;
• AWS IRSA documentation.

[alex-dabija]

Nick, Vaclav, Nikola and I had a call today to discuss what needs to be done to get IRSA running on CAPA clusters.

Main conclusions from the call:

• both irsa-operator and CAPI controllers already generate a service acccount. It's probably easier if irsa-operatoris able to reuse a service account that already exists. This needs to be investigated and changes in irsa-operator are required;
• kubeadm is able to wait until all required secrets and configmaps are available. We'll use this behavior as the signal that the CAPI controllers can continue with cluster creation;
• the configmap format needs to be evaluated to see if the key can easily be extracted.

[njuettner]

Here's the configmap how it looks like for our Cloudfront information

k get cm -n org-namespace clusterid-irsa-cloudfront -o yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: clusterid-irsa-cloudfront
  namespace: org-namespace
data:
  arn: REDACTED
  distributionId: REDACTED
  domain: REDACTED
  originAccessIdentityId: REDACTED

And the secret looks like this

k get secret -n org-namespace clusterid-service-account-v2 -o yaml

apiVersion: v1
data:
  key: REDACTED
  pub: REDACTED
kind: Secret
metadata:
 ...

[njuettner]

The secret can be skipped because we use the generated service account private key and service account pub key from CAPA cluster, but the domain field from the configmap needs to be attached in API flags

Here's the integration of IRSA Cloudfront in aws-operator, https://github.com/giantswarm/aws-operator/pull/3392.

Cloudfront won't work in China that's we we use the public S3 bucket, because you need to use your own certificate.

[njuettner]

@calvix could you point me how kubeadm is waiting for configmaps? where is the service account key located when creating a CAPA cluster?

[njuettner]

irsa-operator is deployed in golem but it's not reconciling anything. I build a toggle which can be used to turn on/off the reconcilation, see https://github.com/giantswarm/config/blob/main/installations/golem/apps/irsa-operator/configmap-values.yaml.patch#L1.

Once this PR https://github.com/giantswarm/cluster-aws/pull/131 is merged we can set the capa toggle to true and it will create IRSA for every CAPA cluster.