Unable to access Kubernetes 1.12.6 dashboard when enabling "networkPlugin": "kubenet"

[vannovv]

Is this a request for help?: Yes

Is this an ISSUE or FEATURE REQUEST? (choose one): Issue

What version of aks-engine?: Version: v0.32.0

Kubernetes version: 1.12.6 What happened: Have been deploying via a set of scripts Kubernetes 1.8.6 clusters which from my understanding use the default kubenet networking. We are wanting to upgrade to 1.12.16 which by default uses the Azure network plugin. We are willing to consider moving to this but would like to currently continue to use the kubenet networking for now.

What you expected to happen: To be able to continue to assess the dashboard

How to reproduce it (as minimally and precisely as possible): Deploy a 1.12.6 cluster withe "networkPlugin": "kubenet" line added to the Kubernetes.json file.

Anything else we need to know: Have tested removing this one line and deploying a new cluster and the dashboard comes right up.

Here is my Kubernetes.json:

{ "apiVersion": "vlabs", "properties": { "orchestratorProfile": { "orchestratorType": "Kubernetes", "orchestratorVersion": "1.12.6", "kubernetesConfig": { "networkPolicy": "calico", "gcHighThreshold": 75, "gcLowThreshold": 70, "controllerManagerConfig": { "--terminated-pod-gc-threshold": "100" } } }, "aadProfile": { "serverAppID": "xxxxxxxxxxxx", "clientAppID": "xxxxxxxxxxxx", "tenantID": "xxxxxxxxxxxx" }, "masterProfile": { "count": $MASTERNODECOUNT$, "dnsPrefix": "$TIER$$REGION$$APP$k8s$VERSION$", "vnetSubnetId": "/subscriptions/$SUBSCRIPTIONID$/resourceGroups/$TIER$$REGION$$APP$net$VERSION$rg/providers/Microsoft.Network/virtualNetworks/$TIER$$REGION$$APP$k8svn$VERSION$/subnets/$TIER$$REGION$$APP$mst$VERSION$sn", "vmSize": "Standard_D2_v2", "OSDiskSizeGB": 100, "storageProfile" : "ManagedDisks", "availabilityProfile": "AvailabilitySet", "vnetCidr" : "$CIDRBLOCK$", "firstConsecutiveStaticIP": "10.$SECONDOCTET$$THIRDOCTET$.150" }, "agentPoolProfiles": [ { "name": "fe", "count": $FRONTENDNODECOUNT$, "vmSize": "Standard_A4m_v2", "OSDiskSizeGB": 100, "storageProfile" : "ManagedDisks", "availabilityProfile": "AvailabilitySet", "vnetSubnetId": "/subscriptions/$SUBSCRIPTIONID$/resourceGroups/$TIER$$REGION$$APP$net$VERSION$rg/providers/Microsoft.Network/virtualNetworks/$TIER$$REGION$$APP$k8svn$VERSION$/subnets/$TIER$$REGION$$APP$fea$VERSION$sn", "availabilityProfile": "AvailabilitySet" }, { "name": "be", "count": $BACKENDNODECOUNT$, "vmSize": "Standard_A4m_v2", "OSDiskSizeGB": 100, "storageProfile" : "ManagedDisks", "availabilityProfile": "AvailabilitySet", "vnetSubnetId": "/subscriptions/$SUBSCRIPTIONID$/resourceGroups/$TIER$$REGION$$APP$net$VERSION$rg/providers/Microsoft.Network/virtualNetworks/$TIER$$REGION$$APP$k8svn$VERSION$/subnets/$TIER$$REGION$$APP$bea$VERSION$sn", "availabilityProfile": "AvailabilitySet" } ], "linuxProfile": { "adminUsername": "azureuser", "ssh": { "publicKeys": [ { "keyData": "$ID_RSA_PUB$" } ] } }, "servicePrincipalProfile": { "ClientID": "$SP_CLIENTID$", "Secret": "$SP_SECRET$"} } }

[mpalumbo7]

I think I misunderstood your problem. In the how to reproduce section it says "Deploy a 1.12.6 cluster withe "networkPlugin": "kubenet" line added to the Kubernetes.json file." So I assumed your problem is with the kubenet plugin...but after re-reading it looks like your problem is with the azurecni plugin.

Just checking....did you attach the route table to the VNET, per the post-deployment instructions?

https://github.com/Azure/aks-engine/blob/master/docs/tutorials/custom-vnet.md#post-deployment-attach-cluster-route-table-to-vnet

[vannovv]

It was my understanding that this was no longer needed. I have had that removed from my scripts for some time (v1.8.6) and it has been working. It appears I can get to everything from the CLI I just can't get to the dashboard. And no I think the problem is with the kubenet plugin since I am not using the azurecni plugin. In the above example of my config I had removed it to test. But it should show the kubenet line. Sorry for the confusion on that.

[vannovv]

I am back at my computer now. It appears that the route tables are not being created by AKS-Engine. Based on the link above I found this statement "For Kubernetes, the cluster should be deployed in the same resource group as the virtual network and the service principal you use for the cluster needs permissions on the VNET resource's group too". I have the network and the cluster in separate RGs, so I am assuming this is the issue. I will adjust my scripts to dump everything in the K8S RG and see what happens. This is a definite change in the behavior at some point between 1.86. to 1.12.6.

[mpalumbo7]

What URL are you using to access the dashboard?

It should be: http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/

[vannovv]

Yes that's it. I added a command to create a route table and added the commands back in from a previous version of the script to assign the route table to the subnets. Everything seems to be working now. I see in the release notes that for AKS-Engine v0.32.0 that they removed the creation of the route table for clusters using Flannel. Wondering if that change is related to what I am seeing. I'm not using Flannel but I am using Calico and I did not start seeing this until I upgraded to this version. https://github.com/Azure/aks-engine/pull/486

[vannovv]

Correction. I was going through and verifying everything and realized I had not added the "networkPlugin": "kubenet" line back into my Kubernetes.json from when I was testing with the Azure networking plugin. I have deleted the cluster and am redeploying now with that added in and it appears that the route table is being created automatically now. Which is confusing because I had someone visually looking over my shoulder and confirming that it was not there. I will refer I can get to the dashboard one the build is complete.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.