kubectl command does not work to manage K8S cluster deployed in Azure China

[benlam62]

Is this a request for help?:

Yes

Is this an ISSUE or FEATURE REQUEST? (choose one):

Issue

What version of acs-engine?:

0.9.4

Orchestrator and version (e.g. Kubernetes, DC/OS, Swarm) Kubernetes

What happened: On Windows 10 command prompt, I cannot run the kubectl on K8s cluster deployed via acs-engine on Azure China with error "Unable to connect to the server: dial tcp 42.159.202.193:443: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond."

The environment variable KUBECONFIG has been set to reference kubeconfig.chinaeast.json

The k8s cluster has 1 master and 1 agent server in D2V2.

What you expected to happen: Expect the kubectl can work.

How to reproduce it (as minimally and precisely as possible): Run either "kubectl get nodes" or "kubectl clusterinfo dump"

Anything else we need to know:

[andyzhangx]

Did you try run "kubectl get nodes" on k8s master? Never tried kubectl on Windows, while I don't think it's related to azure china.

[benlam62]

I find that the master VM extension cse0 provisioning failed. Any hint?

[benlam62]

The provisioning works fine in Azure International.

[benlam62]

When I ssh to the k8s master VM hosted in Azure China and run "sudo journalctl -u kubelet", the following messages are listed: Nov 28 05:50:35 k8s-master-62207847-0 systemd[1]: Started Kubelet. Nov 28 05:50:35 k8s-master-62207847-0 docker[3188]: Unable to find image 'gcrio.azureedge.net/google_containers/hyperkub Nov 28 05:50:51 k8s-master-62207847-0 docker[3188]: /usr/bin/docker: Error response from daemon: Get https://gcrio.azure Nov 28 05:50:51 k8s-master-62207847-0 docker[3188]: See '/usr/bin/docker run --help'. Nov 28 05:50:51 k8s-master-62207847-0 systemd[1]: kubelet.service: Main process exited, code=exited, status=125/n/a Nov 28 05:50:51 k8s-master-62207847-0 systemd[1]: kubelet.service: Unit entered failed state. Nov 28 05:50:51 k8s-master-62207847-0 systemd[1]: kubelet.service: Failed with result 'exit-code'. Nov 28 05:50:52 k8s-master-62207847-0 systemd[1]: kubelet.service: Service hold-off time over, scheduling restart. Nov 28 05:50:52 k8s-master-62207847-0 systemd[1]: Stopped Kubelet.

[andyzhangx]

that's the issue here, you are using gcrio.azureedge.net to pull image, and in mooncake, it would fail. I would suggest following this guide to deploy k8s cluster by acs-engine: https://github.com/Azure/devops-sample-solution-for-azure-china/blob/master-dev/acs-engine/README.md

In which it would use crproxy.trafficmanager.net:6000/google_containers/hyperkube-amd64 to pull the image, you could also refer to my best practise on mooncake: https://github.com/andyzhangx/Demo/tree/master/acs-engine/mooncake

Just ping me if you have any k8s issue on mooncake, I am the right contact.

[benlam62]

Yes, it works now. Thanks.