googleforgames / agones

Dedicated Game Server Hosting and Scaling for Multiplayer Games on Kubernetes
https://agones.dev
Apache License 2.0
6k stars 791 forks source link

AKS labels are not supported in the Terraform provider, wrong Controller placement #1383

Closed aLekSer closed 4 years ago

aLekSer commented 4 years ago

Agones Controller pod get scheduled to the wrong node pool.

Due to a know bug in terraform provider, we can only assign taints to the Node pool (azurerm_kubernetes_cluster_node_pool resource)

https://github.com/terraform-providers/terraform-provider-azurerm/issues/4968 https://github.com/terraform-providers/terraform-provider-azurerm/pull/5531

This can be generalised to AKS Portal, which seems to not support labels as well.

What happened: Pod got assigned to a wrong node pool, default, where GameServers resides.

What you expected to happen: I have added a special output depends_on in order to wait for all azurerm_kubernetes_cluster_node_pool resource before running Helm Terraform module. But taints does not help, cause we are using nodeAffinity with Labels: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/

    spec:
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - preference:
              matchExpressions:
              - key: agones.dev/agones-system
                operator: Exists
            weight: 1

How to reproduce it (as minimally and precisely as possible):

Follow this guide https://agones.dev/site/docs/installation/terraform/aks/

kubectl get pods --namespace agones-system
kubectl describe pod agones-controller-.. --namespace agones-system

Anything else we need to know?: This ticket is blocked until AKS will update its provider. By the way it seems there is now

AKS Node Pools NoLabels

Environment:

aLekSer commented 4 years ago

Now that https://github.com/terraform-providers/terraform-provider-azurerm/issues/4968 is resolved I can start fixing this issue.