Azure / AKS

Azure Kubernetes Service
https://azure.github.io/AKS/
1.97k stars 310 forks source link

ServiceCidr not accepting non-default values in arm templates #1970

Open Tyler-Angell opened 4 years ago

Tyler-Angell commented 4 years ago

What happened: Attempted to deploy an aks cluster via an arm template and received a ServiceCidrOverlapExistingSubnetsCidr error code with the following message: The specified service CIDR 10.0.0.0/16 is conflicted with an existing subnet CIDR 10.0.0.0/20. However, as shown below my template passes in 10.4.0.0/16. What am I missing? Why doesn't the 'specified' value match the value I passed my template?

Parameters:

{
  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
  "contentVersion": "1.0.0.0",
  "paths": [],
  "parameters": {
    "clusterName": {
      "value": "k8s-dt-su-ta-scus-01"
    },
    "agentPools":{
      "value": [
        //
          {
            "name": "nodepool01",
            "count": 2,
            "sku": "Standard_DS2_v2",
            "vnet": "vnet-dt-su-ta-scus-01",
            "subnet": "Kubernetes",
          },
        //
      ]
    },
    "networkProfile":{
      "value": {
        "loadBalancerSku": "Standard",
        "networkPlugin": "Azure",
        "networkPolicy": "Azure",
        "serviceCidr": "10.4.0.0/16",
        "dnsServiceIP": "10.4.0.10",
        "dockerBridgeCidr": "172.17.0.1/16"
      }
    },
    "enablePrivateCluster":{
      "value": "True"
    },
    "containerRegistry":{
      "value": "tyangell"
    },
    "containerRegistryResourceGroup":{
      "value": "k8s-learning"
    }
  }
}

Template:

{
  "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "clusterName": {
      "type": "string"
    },
    "agentPools": {
      "type": "array"
    },
    "networkProfile": {
      "type": "object"
    },
    "enablePrivateCluster": {
      "type": "string"
    },
    "containerRegistry": {
      "type": "string"
    },
    "containerRegistryResourceGroup": {
      "type": "string"
    },
  },
  "variables": {},
  "resources": [
    {
      "apiVersion": "2020-09-01",
      "dependsOn": [],
      "type": "Microsoft.ContainerService/managedClusters",
      "location": "[resourceGroup().location]",
      "name": "[parameters('clusterName')]",
      "properties": {
        "kubernetesVersion": "1.18.10",
        "enableRBAC": true,
        "dnsPrefix": "[parameters('clusterName')]",
        "networkProfile": "[parameters('networkProfile')]",
        "apiServerAccessProfile": {
          "enablePrivateCluster": "[bool(parameters('enablePrivateCluster'))]"
        },
        "addonProfiles": {
          "httpApplicationRouting": {
            "enabled": false
          },
          "azurePolicy": {
            "enabled": true
          }
        },
        "copy": [
          {
            "name": "agentPoolProfiles",
            "count": "[length(parameters('agentPools'))]",
            "input": {
              "name": "[parameters('agentPools')[copyIndex('agentPoolProfiles')].name]",
              "count": "[parameters('agentPools')[copyIndex('agentPoolProfiles')].count]",
              "vmSize": "[parameters('agentPools')[copyIndex('agentPoolProfiles')].sku]",
              "osType": "Linux",
              "osDiskType": "Managed",
              "type": "VirtualMachineScaleSets",
              "mode": "System",
              "vnetSubnetID": "[resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('agentPools')[copyIndex('agentPoolProfiles')].vnet, parameters('agentPools')[copyIndex('agentPoolProfiles')].subnet)]"
            }
          }
        ]
      },
      "tags": {},
      "identity": {
        "type": "SystemAssigned"
      }
    }
  ],
  "outputs": {
    "controlPlaneFQDN": {
      "type": "string",
      "value": "[reference(concat('Microsoft.ContainerService/managedClusters/', parameters('clusterName'))).privateFQDN]"
    }
  }
}

Deployment CorrelationId: 2566c123-bb0e-4176-903a-60d8bbfb122e

ghost commented 4 years ago

Hi Tyler-Angell, AKS bot here :wave: Thank you for posting on the AKS Repo, I'll do my best to get a kind human from the AKS team to assist you.

I might be just a bot, but I'm told my suggestions are normally quite good, as such: 1) If this case is urgent, please open a Support Request so that our 24/7 support team may help you faster. 2) Please abide by the AKS repo Guidelines and Code of Conduct. 3) If you're having an issue, could it be described on the AKS Troubleshooting guides or AKS Diagnostics? 4) Make sure your subscribed to the AKS Release Notes to keep up to date with all that's new on AKS. 5) Make sure there isn't a duplicate of this issue already reported. If there is, feel free to close this one and '+1' the existing issue. 6) If you have a question, do take a look at our AKS FAQ. We place the most common ones there!

Tyler-Angell commented 4 years ago

It turns out that the accepted values for the below are case sensitive. Fixing the casing allowed the template to successfully deploy. I am unblocked, but this still feels like a bug (both casing requirement and error message discoverability) so I'm leaving this opened to be triaged appropriately.

Before

"loadBalancerSku": "Standard",
"networkPlugin": "Azure",
"networkPolicy": "Azure",

After

"loadBalancerSku": "standard",
"networkPlugin": "azure",
"networkPolicy": "azure",
yangl900 commented 4 years ago

thanks for reporting @Tyler-Angell . It does look like an bug we have. The message was very confusing, sorry about that.