Unable to deploy Gitlab Agent chart using the Helm provider on OpenTofu, but fine using Helm directly

[toby-griffiths]

Terraform, Provider, Kubernetes and Helm Versions

Terraform version: v1.5.7
Provider version: v2.15.0
Kubernetes version: Kubernetes 1.31.1-do.1

Affected Resource(s)

• helm_release (The template suggested helm_repository but I can find now reference to this in the docs on the Terraform website)?

Terraform Configuration Files

terraform {
  required_providers {
    digitalocean = {
      source  = "digitalocean/digitalocean"
      version = ">= 2.42"
    }
    helm = {
      source  = "hashicorp/helm"
      version = ">= 2.15.0"
    }
  }
  backend "http" {}
}

provider "digitalocean" {
  token = var.do_token
}

provider "helm" {
    kubernetes {
        host = digitalocean_kubernetes_cluster.default.endpoint
        client_certificate = base64decode(digitalocean_kubernetes_cluster.default.kube_config.0.client_certificate)
        client_key = base64decode(digitalocean_kubernetes_cluster.default.kube_config.0.client_key)
        cluster_ca_certificate = base64decode(digitalocean_kubernetes_cluster.default.kube_config.0.cluster_ca_certificate)
    }
}

variable "do_token" {
  default = ""
}

variable "do_region" {
  type    = string
  default = "lon1"
}

variable "do_k8s_version" {
  type = string
  # Grab the latest version slug from `doctl kubernetes options versions`
  default = "1.31.1-do.1"
}

variable "do_k8s_default_node_pool_node_size" {
  type = string
  # Grab the latest version slug from `doctl kubernetes options versions`
  default = "s-2vcpu-2gb"
}

variable "do_k8s_default_node_pool_count" {
  type = number
  # Grab the latest version slug from `doctl kubernetes options versions`
  default = 3
}

variable "gitlab_k8s_agent_token" {
    description = "Token used to authenticate Gitlab k8s agent with Gitlab.com"
    type = string
}

resource "digitalocean_project" "rapid_software" {
    name        = "Rapid Software"
    environment = "Production"
    resources = [
        digitalocean_kubernetes_cluster.default.urn
    ]
}

resource "digitalocean_vpc" "default" {
  name   = "ra.pid.software"
  region = var.do_region
}

resource "digitalocean_kubernetes_cluster" "default" {
  name     = "rapid-software"
  region   = var.do_region
  version  = var.do_k8s_version
  vpc_uuid = digitalocean_vpc.default.id

  node_pool {
    name       = "default-pool"
    size       = var.do_k8s_default_node_pool_node_size
    node_count = var.do_k8s_default_node_pool_count
    auto_scale = false
  }
}

resource "helm_release" "gitlab_k8s_agent" {
    name       = "rapid-software-production"
    repository = "https://charts.gitlab.io"
    chart      = "gitlab-agent"

    namespace = "gitlab-agent-rapid-software-core"
    create_namespace = true
    atomic = true
    cleanup_on_fail = true
    replace = true
    set {
        name  = "config.token"
        value = var.gitlab_k8s_agent_token
    }
    set {
        name  = "kasAddress"
        value = "ss://kas.gitlab.com"
    }
}

Debug Output

See gist: https://gist.github.com/toby-griffiths/cd7a1abaffeeb6f7ed67f8fa928dddec

Panic Output

Not sure how to find if I haev this?

Steps to Reproduce

1. terraform apply -var="do_token=$DIGITALOCEAN_CM_FULL_ACCESS_TOKEN" -var="gitlab_k8s_agent_token=[REDACTE]" -auto-approve

Expected Behavior

Helm chart should deploy

Actual Behavior

I see error…

Error: could not get apiVersions from Kubernetes: could not get apiVersions from Kubernetes: unknown
│
│   with helm_release.gitlab_k8s_agent,
│   on helm.tf line 1, in resource "helm_release" "gitlab_k8s_agent":
│    1: resource "helm_release" "gitlab_k8s_agent" {
│

Important Factoids

If I attempt to run this install using Helm directly, it works fine…

helm repo add gitlab https://charts.gitlab.io
helm repo update
helm upgrade --install rapid-software-default-cluster gitlab/gitlab-agent \
    --namespace gitlab-agent-rapid-software-default-cluster \
    --create-namespace \
    --set config.token=[REDACTED] \
    --set config.kasAddress=wss://kas.gitlab.com

This is using the Kubeconfig saved by calling doctl kubernetes cluster kubeconfig save rapid-software --expiry-seconds=3600, so there's a change there's dicrepancy between the Kubeconfigs, perhaps?

References

n/a

Community Note

n/a

Thanks for any help you're able to offer.