examples/external-secrets does not sync external secrets

MohammadAlavi1986 commented 1 year ago

Description

The two ClusterSecretStore and SecretStore custom resources use IRSA to access AWS secret manager and SSM parameter store, but service accounts specified in theses two resources are not being created.

> kubectl get ClusterSecretStore
NAME                     AGE     STATUS                  CAPABILITIES   READY
cluster-secretstore-sm   8m47s   InvalidProviderConfig   ReadWrite      False

> kubectl get SecretStore -A
NAMESPACE          NAME             AGE     STATUS                  CAPABILITIES   READY
external-secrets   secretstore-ps   8m53s   InvalidProviderConfig   ReadWrite      False

> kubectl get sa -n external-secrets
NAME                               SECRETS   AGE
default                            0         2m23s
external-secrets-cert-controller   0         2m22s
external-secrets-sa                0         2m22s
external-secrets-webhook           0         2m22s

> kubectl get secret -A
NAMESPACE          NAME                                     TYPE                 DATA   AGE
external-secrets   external-secrets-webhook                 Opaque               4      18m
external-secrets   sh.helm.release.v1.external-secrets.v1   helm.sh/release.v1   1      18m

Unlike the local ../../modules/irsa module, the terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks module does not create Kubernetes service account resources. Since no service accounts are being created, ClusterSecretStore and SecretStore resources will be created with an InvalidProviderConfig status.

Versions

Module version [Required]: ~> 1.0
Terraform version: v1.2.2
Provider version(s):
provider registry.terraform.io/gavinbunney/kubectl v1.14.0
provider registry.terraform.io/hashicorp/aws v5.4.0
provider registry.terraform.io/hashicorp/cloudinit v2.3.2
provider registry.terraform.io/hashicorp/helm v2.10.1
provider registry.terraform.io/hashicorp/kubernetes v2.21.1
provider registry.terraform.io/hashicorp/time v0.9.1
provider registry.terraform.io/hashicorp/tls v4.0.4

Module version(s):

"aws-ia/eks-blueprints-addons/aws": "~> 1.0"
"terraform-aws-modules/eks/aws": "~> 19.13"
"terraform-aws-modules/vpc/aws": "~> 5.0"
"terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks": "~> 5.20"

Reproduction Code [Required]

Steps to reproduce the behavior: Run terraform apply in examples/external-secrets directory.

Expected behaviour

> kubectl get externalsecret -A
NAMESPACE          NAME                  STORE                    REFRESH INTERVAL   STATUS         READY
external-secrets   external-secrets-ps   secretstore-ps           1h                 SecretSynced   True
external-secrets   external-secrets-sm   cluster-secretstore-sm   1h                 SecretSynced   True

> kubectl get secret -A
NAMESPACE          NAME                                     TYPE                 DATA   AGE
external-secrets   external-secrets-ps                      Opaque               2      50s
external-secrets   external-secrets-sm                      Opaque               2      50s
external-secrets   external-secrets-webhook                 Opaque               4      85s
external-secrets   sh.helm.release.v1.external-secrets.v1   helm.sh/release.v1   1      85s

Actual behaviour

> kubectl get ClusterSecretStore
NAME                     AGE     STATUS                  CAPABILITIES   READY
cluster-secretstore-sm   8m47s   InvalidProviderConfig   ReadWrite      False

> kubectl get SecretStore -A
NAMESPACE          NAME             AGE     STATUS                  CAPABILITIES   READY
external-secrets   secretstore-ps   8m53s   InvalidProviderConfig   ReadWrite      False

Terminal Output Screenshot(s)

Additional context

bryantbiggs commented 1 year ago

The helm chart creates the service account by default https://github.com/external-secrets/external-secrets/blob/a4508a809161a291f658ae4fa9ad84a10f66a06a/deploy/charts/external-secrets/values.yaml#L71

I think one issue is that we are not passing the name of the service account down to the chart correctly https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/c5f51610e469e741eceb1c4cb185d317cbe6ddd2/main.tf#LL1944C3-L1944C35

MohammadAlavi1986 commented 1 year ago

Sorry, I created the issue in the wrong repo. K8s service account for the external secret controller is created by the helm chart. However ClusterSecretStore and SecretStore resources are using two separate service accounts (cluster-secretstore-sa and secretstore-sa), which were used to be created by the local irsa module, but after migrating to the iam-role-for-service-accounts-eks module these two service accounts are NOT being created.

bryantbiggs commented 1 year ago

yes, I moved the issue here since this is where the implementation lives for the addon. We will look at fixing this here and making the appropriate changes in the respective examples

aws-ia / terraform-aws-eks-blueprints-addons