[Bug]: modules/eks-monitoring/add-ons/external-secrets does not work with Fargate

[mpalumbo7]

Welcome to the AWS Observability Accelerator

• [X] Yes, I've searched similar issues on GitHub and didn't find any.

AWS Observability Accelerator Release version

v2.9.2

What is your environment, configuration and the example used?

❯ terraform --version          
Terraform v1.5.7
on darwin_arm64
+ provider registry.terraform.io/gavinbunney/kubectl v1.14.0
+ provider registry.terraform.io/hashicorp/aws v5.22.0
+ provider registry.terraform.io/hashicorp/awscc v0.63.0
+ provider registry.terraform.io/hashicorp/cloudinit v2.3.2
+ provider registry.terraform.io/hashicorp/helm v2.11.0
+ provider registry.terraform.io/hashicorp/kubernetes v2.23.0
+ provider registry.terraform.io/hashicorp/null v3.2.1
+ provider registry.terraform.io/hashicorp/time v0.9.1
+ provider registry.terraform.io/hashicorp/tls v4.0.4

Used to deploy:

• EKS v1.27
  • Deployed via terraform-aws-modules/eks/aws version 19.15.3 and computeType = "Fargate"
• terraform-aws-obervability-accelerator v2.9.2
• eks-monitoring v2.9.2

What did you do and What did you see instead?

While deploying eks-monitoring, I received the following:

❯ terraform apply .tf-out                                                                                                                                      
Acquiring state lock. This may take a few moments...
module.eks_observability.module.eks_monitoring.module.external_secrets[0].kubectl_manifest.cluster_secretstore: Creating...
module.eks_observability.module.eks_monitoring.module.external_secrets[0].kubectl_manifest.secret: Creating...
module.eks_observability.module.eks_monitoring.module.operator[0].module.cert_manager[0].module.helm_addon.helm_release.addon[0]: Creating...
module.eks_observability.module.eks_monitoring.module.external_secrets[0].kubectl_manifest.cluster_secretstore: Still creating... [10s elapsed]
module.eks_observability.module.eks_monitoring.module.external_secrets[0].kubectl_manifest.secret: Still creating... [10s elapsed]
╷
│ Error: cannot re-use a name that is still in use
│ 
│   with module.eks_observability.module.eks_monitoring.module.operator[0].module.cert_manager[0].module.helm_addon.helm_release.addon[0],
│   on .terraform/modules/eks_observability.eks_monitoring.operator.cert_manager/modules/kubernetes-addons/helm-addon/main.tf line 1, in resource "helm_release" "addon":
│    1: resource "helm_release" "addon" {
│ 
╵
╷
│ Error: cluster-secretstore-sm failed to run apply: error when creating "/var/folders/7b/tdztr7dj46z531m5m0pxzymc0000gp/T/389883418kubectl_manifest.yaml": Internal error occurred: failed calling webhook "validate.clustersecretstore.external-secrets.io": failed to call webhook: Post "https://external-secrets-webhook.external-secrets.svc:443/validate-external-secrets-io-v1beta1-clustersecretstore?timeout=5s": tls: failed to verify certificate: x509: certificate is valid for ip-XX-XX-XX-XX.us-west-2.compute.internal, not external-secrets-webhook.external-secrets.svc
│ 
│   with module.eks_observability.module.eks_monitoring.module.external_secrets[0].kubectl_manifest.cluster_secretstore,
│   on .terraform/modules/eks_observability.eks_monitoring/modules/eks-monitoring/add-ons/external-secrets/main.tf line 59, in resource "kubectl_manifest" "cluster_secretstore":
│   59: resource "kubectl_manifest" "cluster_secretstore" {
│ 
╵
╷
│ Error: grafana-operator/external-secrets-sm failed to run apply: error when creating "/var/folders/7b/tdztr7dj46z531m5m0pxzymc0000gp/T/525858993kubectl_manifest.yaml": Internal error occurred: failed calling webhook "validate.externalsecret.external-secrets.io": failed to call webhook: Post "https://external-secrets-webhook.external-secrets.svc:443/validate-external-secrets-io-v1beta1-externalsecret?timeout=5s": tls: failed to verify certificate: x509: certificate is valid for ip-XX-XX-XX-XX.us-west-2.compute.internal, not external-secrets-webhook.external-secrets.svc
│ 
│   with module.eks_observability.module.eks_monitoring.module.external_secrets[0].kubectl_manifest.secret,
│   on .terraform/modules/eks_observability.eks_monitoring/modules/eks-monitoring/add-ons/external-secrets/main.tf line 89, in resource "kubectl_manifest" "secret":
│   89: resource "kubectl_manifest" "secret" {
│ 
╵
Releasing state lock. This may take a few moments...

Some research leads me to believe the issue is the same as this:

https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/issues/55

However, the workaround of setting the external-secrets webhook port to 9443 is not possible with the observability accelerator. The helm_config variable of external-secrets is not exposed at the top level module variables.

Do I understand the problem correctly, or is there something else going on?

Additional Information

No response

[bpgould]

I have the same issue