tetratelabs / terraform-eksblueprints-tetrate-istio-addon

7 stars 5 forks source link

[Bug] When installing via ArgoCD (GitOps) the `istio-system` namespace is not created #8

Open spkane opened 2 years ago

spkane commented 2 years ago
module "k8s-addons" {
  source  = "github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons?ref=093574f79a50413b961376d9ec0310bda6b514b9"

  eks_cluster_id               = local.eks_cluster_id
  eks_worker_security_group_id = local.eks_worker_security_group_id

  enable_amazon_eks_coredns    = true
  amazon_eks_coredns_config    = {
    addon_version              = "v1.8.3-eksbuild.1"
  }

  enable_amazon_eks_kube_proxy = true
  amazon_eks_kube_proxy_config = {
    addon_version              = "v1.20.7-eksbuild.1"
  }

  enable_amazon_eks_vpc_cni    = true

  enable_argocd                       = true
  argocd_helm_config = {
    version          = "4.5.4"
    values           = [templatefile("${path.module}/helm_values/argocd-values.yaml.tftpl", {nodeSelector = local.primaryNodeSelector})]
  }

  argocd_manage_add_ons               = true
  argocd_applications     = {
    addons = {
      path                = "chart"
      repo_url            = "https://github.com/aws-samples/eks-blueprints-add-ons.git"
      project             = "default"
      add_on_application  = true // This indicates the root add-on application.
    }
  }

  // See: https://github.com/tetratelabs/terraform-eksblueprints-tetrate-istio-addon
  enable_tetrate_istio           = true
  tetrate_istio_version          = "1.13.3"
  tetrate_istio_base_helm_config = {
    values           = [templatefile("${path.module}/helm_values/tetrate-istio-base-values.yaml.tftpl", {nodeSelector = local.primaryNodeSelector})]
  }
  tetrate_istio_cni_helm_config = {
    values           = [templatefile("${path.module}/helm_values/tetrate-istio-cni-values.yaml.tftpl", {nodeSelector = local.primaryNodeSelector})]
  }
  tetrate_istio_istiod_helm_config = {
    values           = [templatefile("${path.module}/helm_values/tetrate-istio-istiod-values.yaml.tftpl", {nodeSelector = local.primaryNodeSelector})]
  }
  tetrate_istio_gateway_helm_config = {
    values           = [templatefile("${path.module}/helm_values/tetrate-istio-gateway-values.yaml.tftpl", {nodeSelector = local.primaryNodeSelector})]
  }

CleanShot 2022-04-25 at 16 57 22

Creating the namespace by hand gets past that error, but an initial look seemed to suggest that the namespace creation should be handled, and creating it is the default.

https://github.com/tetratelabs/terraform-eksblueprints-tetrate-istio-addon/blob/main/locals.tf#L13

That being said, it is not clear to me that the create_namespace value is actually being used anywhere.

ZeroDeth commented 2 years ago

After deploy/destroy several times, only way to disable argocd_manage_add_ons = false

ArgoCD installed in the management shared cluster

Example of working staging cluster

module "eks_addons_0" {
  source             = "github.com/aws-ia/terraform-aws-eks-blueprints.git//modules/kubernetes-addons?ref=v4.0.3"
  eks_cluster_id     = module.eks_0.eks_cluster_id
  eks_cluster_domain = var.hosted_name

  #---------------------------------------------------------------
  # Amazon EKS Managed ADD-ON
  #---------------------------------------------------------------

  ## VPC-CNI
  enable_amazon_eks_vpc_cni = true
  amazon_eks_vpc_cni_config = {
    addon_version            = "v1.10.1-eksbuild.1"
  }

  ## CoreDNS
  enable_amazon_eks_coredns = true
  amazon_eks_coredns_config = {
    addon_version     = "v1.8.4-eksbuild.1"
  }

  ## Kube-Proxy
  enable_amazon_eks_kube_proxy = true
  amazon_eks_kube_proxy_config = {
    addon_version     = "v1.21.2-eksbuild.2"
  }

  ## EBS-CSI Driver
  enable_amazon_eks_aws_ebs_csi_driver = false
  amazon_eks_aws_ebs_csi_driver_config = {
    addon_version            = "v1.5.2-eksbuild.1"
  }

  #---------------------------------------------------------------
  # K8s ADD-ONS
  #---------------------------------------------------------------

  enable_external_dns       = true
  enable_cluster_autoscaler = true
  enable_prometheus         = true
  enable_aws_for_fluentbit  = true

  enable_metrics_server = true
  metrics_server_helm_config = {
    name       = "metrics-server"
    chart      = "metrics-server"
    version    = "3.8.1"
    repository = "https://kubernetes-sigs.github.io/metrics-server/"
    namespace  = "kube-system"
    timeout    = "1200"
    lint       = "true"
    values = [templatefile("${path.module}/helm_values/metrics-server-values.yaml", {
      operating_system = "linux"
    })]
  }

  enable_cert_manager = false
  cert_manager_helm_config = {
    name       = "cert-manager"
    chart      = "cert-manager"
    repository = "https://charts.jetstack.io"
    version    = "v1.7.1"
    namespace  = "cert-manager"
  }

  enable_keda = true
  keda_helm_config = {
    name       = "keda"
    chart      = "keda"
    repository = "https://kedacore.github.io/charts"
    version    = "2.6.2"
    namespace  = "keda"
    values     = [templatefile("${path.module}/helm_values/keda-values.yaml", {})]
  }

  enable_aws_load_balancer_controller = true
  aws_load_balancer_controller_helm_config = {
    name       = "aws-load-balancer-controller"
    chart      = "aws-load-balancer-controller"
    repository = "https://aws.github.io/eks-charts"
    version    = "1.3.1"
    namespace  = "kube-system"
  }

  enable_vpa = true
  vpa_helm_config = {
    name       = "vpa"
    chart      = "vpa"
    repository = "https://charts.fairwinds.com/stable"
    version    = "1.0.0"
    namespace  = "vpa"
    values     = [templatefile("${path.module}/helm_values/vpa-values.yaml", {})]
  }

  enable_ingress_nginx = false
  ingress_nginx_helm_config = {
    name       = "ingress-nginx"
    chart      = "ingress-nginx"
    repository = "https://kubernetes.github.io/ingress-nginx"
    version    = "4.0.17"
    values = [templatefile("${path.module}/helm_values/nginx-values.yaml", {
      hostname     = var.hosted_name
      ssl_cert_arn = module.stg_acm.acm_certificate_arn
    })]
  }

  enable_tetrate_istio = true
  tetrate_istio_base_helm_config = {
  }

  depends_on = [
    module.eks_0.managed_node_groups,
    module.vpc_0
  ]
}

Istio deployed and was successful on that example, however failed to make cert-manager & ingress-nginx work with tag v4.03/02

@spkane How did you manage sg in example above? eks_worker_security_group_id = local.eks_worker_security_group_id

spkane commented 2 years ago

@spkane How did you manage sg in example above? eks_worker_security_group_id = local.eks_worker_security_group_id

@ZeroDeth I am just reading it from the Terraform state that created the node group and then creating a local.

 eks_worker_security_group_id = data.terraform_remote_state.eks.outputs.eks_worker_security_group_id