todd-dsm / dev-infras

Terraform 101 Material
Apache License 2.0
3 stars 3 forks source link

pass KUBECONFIG between modules or to providers #9

Closed todd-dsm closed 11 months ago

todd-dsm commented 11 months ago

Attempting to enable addons, oidc auth, et al with this module.

The Kubernetes/Helm providers in the new eks_blueprints_addons module (irsa branch) requires Kubernetes credentials (KUBECONFIG/active context) to finish their work. If not, things fail:

% make apply                                                                                                  <aws:vivsoft-gov> <region:us-gov-east-1>
terraform apply --auto-approve -no-color -parallelism=1 \
    -input=false /tmp/tf-gitops-demo.plan \
    2>&1 | tee /tmp/tf-gitops-demo-apply.out
module.eks_blueprints_addons.module.metrics_server.helm_release.this[0]: Creating...
module.eks_blueprints_addons.module.cluster_proportional_autoscaler.helm_release.this[0]: Creating...
module.eks_blueprints_addons.module.aws_load_balancer_controller.helm_release.this[0]: Creating...
module.eks_blueprints_addons.module.external_dns.helm_release.this[0]: Creating...

Error: Kubernetes cluster unreachable: invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable

  with module.eks_blueprints_addons.module.aws_load_balancer_controller.helm_release.this[0],
  on .terraform/modules/eks_blueprints_addons.aws_load_balancer_controller/main.tf line 9, in resource "helm_release" "this":
   9: resource "helm_release" "this" {

Error: Kubernetes cluster unreachable: invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable

  with module.eks_blueprints_addons.module.cluster_proportional_autoscaler.helm_release.this[0],
  on .terraform/modules/eks_blueprints_addons.cluster_proportional_autoscaler/main.tf line 9, in resource "helm_release" "this":
   9: resource "helm_release" "this" {

Error: Kubernetes cluster unreachable: invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable

  with module.eks_blueprints_addons.module.external_dns.helm_release.this[0],
  on .terraform/modules/eks_blueprints_addons.external_dns/main.tf line 9, in resource "helm_release" "this":
   9: resource "helm_release" "this" {

Error: Kubernetes cluster unreachable: invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable

  with module.eks_blueprints_addons.module.metrics_server.helm_release.this[0],
  on .terraform/modules/eks_blueprints_addons.metrics_server/main.tf line 9, in resource "helm_release" "this":
   9: resource "helm_release" "this" {

Check these issues

todd-dsm commented 11 months ago

Fixed references to the above resources

provider "helm" {
  kubernetes {
    host                   = aws_eks_cluster.apps.endpoint
    cluster_ca_certificate = base64decode(aws_eks_cluster.apps.certificate_authority[0].data)

    exec {
      api_version = "client.authentication.k8s.io/v1beta1"
      command     = "aws"
      args        = ["eks", "get-token", "--cluster-name", aws_eks_cluster.apps.id]
    }
  }
}