VM has reported a failure when processing extension 'vmssCSE'. Error message: "Enable failed: failed to execute command: command terminated with exit status=52 [stdout] aks-system-123266-vmss000000 Connection to mcr.microsoft.com 443 port [tcp/https] succeeded! [stderr] "

Azure / AKS

Azure Kubernetes Service

https://azure.github.io/AKS/

1.97k stars 307 forks source link

VM has reported a failure when processing extension 'vmssCSE'. Error message: "Enable failed: failed to execute command: command terminated with exit status=52 [stdout] aks-system-123266-vmss000000 Connection to mcr.microsoft.com 443 port [tcp/https] succeeded! [stderr] " #2004

Closed megharaikwar closed 3 years ago

megharaikwar commented 3 years ago

What happened: I am trying to create an AKS Cluster using Terraform through Azure Devops pipeline. The pipeline is running for around 18 minutes and then erroring out with the error. Every other resources are created but virtual machine scale sets failed.

What you expected to happen:

How to reproduce it (as minimally and precisely as possible): Create a simple pipeline with terraform init and terraform apply

Anything else we need to know?:

Environment:

Kubernetes version (use kubectl version): 1.18.10
Size of cluster (how many worker nodes are in the cluster?) 2
General description of workloads in the cluster (e.g. HTTP microservices, Java app, Ruby on Rails, machine learning, etc.)
Others:

ghost commented 3 years ago

Hi megharaikwar, AKS bot here :wave: Thank you for posting on the AKS Repo, I'll do my best to get a kind human from the AKS team to assist you.

I might be just a bot, but I'm told my suggestions are normally quite good, as such: 1) If this case is urgent, please open a Support Request so that our 24/7 support team may help you faster. 2) Please abide by the AKS repo Guidelines and Code of Conduct. 3) If you're having an issue, could it be described on the AKS Troubleshooting guides or AKS Diagnostics? 4) Make sure your subscribed to the AKS Release Notes to keep up to date with all that's new on AKS. 5) Make sure there isn't a duplicate of this issue already reported. If there is, feel free to close this one and '+1' the existing issue. 6) If you have a question, do take a look at our AKS FAQ. We place the most common ones there!

smynes commented 3 years ago

Same issue here. Trying to configure a private aks. Using custom DNS, so we've linked the vnet for our DNS servers to the private DNS zone and our DNs servers are using the Azure specified DNS server , 168.63.129.16, as the forwarder / upstream DNS server. The vnets in use for the aks and associated acr both have internet access. The error persists whether we use custom settings for the aks service CIDR/DNS service IP/Docker bridge CIDR or take the defaults. Multiple cases open with support on this. Wil update if any progress.

yangl900 commented 3 years ago

52 means ERR_K8S_API_SERVER_DNS_LOOKUP_FAIL

https://github.com/Azure/AgentBaker/blob/821b8212802c02982a611ea74c4ba46c798f556c/pkg/agent/testdata/AKSUbuntu1604%2BEnablePrivateClusterHostsConfigAgent/line9.sh#L37

That typically means the node was not able to resolve the private DNS name. Was the DNS zone associated to the custom DNS subnet?

AKS also has some improvements on the DNS handling to rollout soon, to avoid the manual attachment. @feiskyer could you comment more?

feiskyer commented 3 years ago

For custom DNS, please refer the docs https://docs.microsoft.com/en-us/azure/aks/private-clusters#hub-and-spoke-with-custom-dns. here is an example of terraform deployment with custom DNS.

feiskyer commented 3 years ago

@megharaikwar I'm not sure. I think you can make some test. It should work if the two jobs are running parallelly.

etiennejournet commented 3 years ago

Solution found here : I had a legacy "dns_servers" field in my network definition.

Sorry for that.