Set container registry proxy in terraform helm chart deployment

[markbangert]

I am trying to set a registry proxy within a terraform helm chart deployment of an nginx ingress controller comparable to the approach described in https://github.com/Azure/container-service-for-azure-china/blob/master/aks/README.md#22-container-registry-proxy. Unfortunately I am unable get things going and I do not find any other directions online so I would like to kindly ask for help here.

My latest try was

resource "helm_release" "nginx_ingress" {
  name      = "ingress-nginx"
  chart     = "ingress-nginx"
  repository = "https://kubernetes.github.io/ingress-nginx"
  namespace = kubernetes_namespace.nginx_ingress.metadata[0].name

  values = [ "defaultBackend.image.repository: gcr.azk8s.cn/google_containers/defaultbackend" ]

  set {
    name  = "controller.service.annotations.service\\.beta\\.kubernetes\\.io/azure-load-balancer-resource-group"
    value = azurerm_public_ip.nginx_ingress_pip.resource_group_name
  }

  set {
    name  = "controller.service.loadBalancerIP"
    value = azurerm_public_ip.nginx_ingress_pip.ip_address
  }
}

Any hints are greatly appreciated. Thanks / Mark

[andyzhangx]

what's the error msg? gcr.azk8s.cn is only allowed for Azure China VMs

[markbangert]

Thank you for jumping in so quickly!

The error is

Failed to pull image "k8s.gcr.io/ingress-nginx/controller:v0.44.0@sha256:3dd0fac48073beaca2d67a78c746c7593f9c575168a17139a9955a82c63c4b9a": rpc error: code = Unknown desc = Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

so the process still tries to retrieve the images from the original repo, not from the mirror at k8sgcr.azk8s.cn.

[andyzhangx]

not familiar with terraform, I think you could file an issue in terraform repo.

[markbangert]

Alright will do so but keep this thread open for the time being if somebody else has something to add...

Thank you!

[markbangert]

Turns out I had some problems figuring out how to modify the helm chart in the correct way. This works:

resource "helm_release" "nginx_ingress" {
  name      = "ingress-nginx"
  chart     = "ingress-nginx"
  repository = "https://kubernetes.github.io/ingress-nginx"
  namespace = kubernetes_namespace.nginx_ingress.metadata[0].name

  set {
    name  = "controller.service.annotations.service\\.beta\\.kubernetes\\.io/azure-load-balancer-resource-group"
    value = azurerm_public_ip.nginx_ingress_pip.resource_group_name
  }

  set {
    name  = "controller.service.loadBalancerIP"
    value = azurerm_public_ip.nginx_ingress_pip.ip_address
  }

  set {
    name = "controller.image.repository"
    value = "k8sgcr.azk8s.cn/ingress-nginx/controller"
  }
}

Thank you anyways for your input!