Failing to install helm chart using Helm Provider for Terraform

[timur-khadimullin]

I am trying to deploy k8s and install Octopus Agent on it using Terraform.

1. I logged onto our instance of Octopus Cloud and went through the process of adding new deployment target.
2. I write the following Terraform code (which I reverse engineered from the command lines that Octopus UI gave me):

#provider "helm" {
#  kubernetes {
#    // this part is probably irrelevant; I include it to show I don't have any credentials configured here
#  }
#}

resource "helm_release" "csi_driver_nfs" {
  name       = "csi-driver-nfs"
  repository = "https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/charts"
  chart      = "csi-driver-nfs"
  version    = "v4.6.0"
  atomic     = true
  namespace = "kube-system"
  dependency_update = true  
  depends_on = [ azurerm_kubernetes_cluster.example ] // assume this condition has been satisfied and deployment went ahead
}

resource "helm_release" "octopus_tentacle" {
  name       = "test-sandbox-k8s-agent"
  repository = "oci://registry-1.docker.io/octopusdeploy"
  chart      = "kubernetes-agent"

  verify = false
  dependency_update = true  
  version    = "1.*.*"
  atomic     = true
  namespace = "octopus-agent-test-sandbox-k8s-agent"
  create_namespace = true

  depends_on = [ helm_release.csi_driver_nfs ]

  set {
    name  = "agent.acceptEula"
    value = "Y"
  }

  set {
    name  = "agent.serverUrl"
    value = "https://xxx.octopus.app/"
  }

  set {
    name  = "agent.serverCommsAddresses"
    value = "{https://polling.xxx.octopus.app/}"
  }

  set {
    name  = "agent.space"
    value = "Default"
  }

  set {
    name  = "agent.targetName"
    value = "test-sandbox-k8s-agent"
  }

  set {
    name  = "agent.targetEnvironments"
    value = "{my-environment}"
  }

  set {
    name  = "agent.targetRoles"
    value = "{my-deployment-tag}"
  }

  set_sensitive {
    name  = "agent.bearerToken"
    value = "xxx"
  }
}

When I run terraform apply on this code, helm_release.csi_driver_nfs gets deployed into my k8s cluster but when deployment gets to helm_release.octopus_tentacle, I get the following error:

│ Error: could not download chart: GET "https://registry-1.docker.io/v2/octopusdeploy/kubernetes-agent/tags/list": unable to retrieve credentials
│ 
│   with helm_release.octopus_tentacle,
│   on octopus.tf line 12, in resource "helm_release" "octopus_tentacle":
│   12: resource "helm_release" "octopus_tentacle" {
│ 

I am not sure if using OCI repositories is an issue with helm provider, which is that not an issue with Octopus Agent, but I was wondering if you would consider supporting fetching charts from either github or .tar.gz releases to enable this scenario?

[timur-khadimullin]

A quick update: I downloaded the latest release to my machine and changed chart location to local disk:

wget https://github.com/OctopusDeploy/helm-charts/archive/refs/tags/kubernetes-agent/1.10.0.tar.gz
tar -zxf 1.10.0.tar.gz

resource "helm_release" "octopus_tentacle" {
  // unchanged code
  chart      = "./helm-charts-kubernetes-agent-1.10.0/charts/kubernetes-agent"
  // unchanged code  

this worked. So I don't believe it's an issue with the chart itself. It seems to be specific to how it's being delivered.

[timur-khadimullin]

another update: I found this page going through almost my exact scenario: https://octopus.com/docs/infrastructure/deployment-targets/kubernetes/kubernetes-agent/automated-installation this however fails for me with the same error:

│ Error: could not download chart: GET "https://registry-1.docker.io/v2/octopusdeploy/kubernetes-agent/tags/list": unable to retrieve credentials
│ 
│   with helm_release.octopus_tentacle[0],
│   on octopus-agent.tf line 40, in resource "helm_release" "octopus_tentacle":
│   40: resource "helm_release" "octopus_tentacle" {
│ 

[APErebus]

Hi @timur-khadimullin,

What version of the helm terraform plugin are you using? It looks like that endpoint requires authentication. Can you try after authenticating via docker?

[timur-khadimullin]

What version of the helm terraform plugin are you using?

versions are as follows:

terraform {
  required_version = ">= 1.6.0"

  required_providers {
    helm = {
      source  = "hashicorp/helm"
      version = ">= 2.14.0"
    }
    octopusdeploy = {
      source  = "OctopusDeployLabs/octopusdeploy"
      version = ">= 0.22.0"
    }
  }
}

It looks like that endpoint requires authentication.

This thought has crossed my mind. But I was under impression public Docker registry does not require authentication.

Can you try after authenticating via docker?

Let me see if I can try. Having said that, I don't think it would resolve our issue as the documentation makes no note of authentication being required. So, we expect it to work either way

[APErebus]

But I was under impression public Docker registry does not require authentication.

So was I, however that endpoint seems to want to require authentication for some reason.

I had a look in the helm source code and it seems to throw an error when it can't retrieve credentials from the docker registry client https://github.com/helm/helm/blob/b48f9c9b1dc01f49fdec45624c16a924182fa035/pkg/registry/client.go#L135

[timur-khadimullin]

I haven't had a chance to try it yet. But this probably will be a case of updating documentation.

For all the people who are unable to use docker registry - I wonder if you could also provide an example of how one would go about pulling the charts off GitHub instead of Docker Hub?

[APErebus]

Good question.

We'd need to start pushing the .tar.gz files to the git repo... I'll have to discuss it in the team 🤔

[timur-khadimullin]

@APErebus I got around to adding creds to my config as follows:

provider "helm" {
  // some k8s auth code here

  registry {
    url = "oci://registry-1.docker.io"
    username = "xxxxxxx"
    password = "xxxxxxx" 
  }
}

resource "helm_release" "octopus_tentacle" {
  // unchanged code
  #chart            = "./helm-charts-kubernetes-agent-1.10.0/charts/kubernetes-agent" # commented out
  repository       = "oci://registry-1.docker.io" # added repo
  chart            = "octopusdeploy/kubernetes-agent" # added chart
  // unchanged code

yet still getting the error:

│ Error: could not login to OCI registry "registry-1.docker.io": error storing credentials - err: exit status 127, out: ``
│ 
│   with provider["registry.terraform.io/hashicorp/helm"],
│   on providers.tf line 14, in provider "helm":
│   14: provider "helm" {