Support custom DNS IP `--dns-service-ip`

ottodeng commented 4 months ago

Version

Karpenter Version: v0.0.0

Kubernetes Version: v1.0.0

Expected Behavior

the pod on new karpenter node, /etc/resolv.conf is 10.0.0.10. not follow az cli create --dns-service-ip 10.97.96.10.

this issue will cause cannot use custom vnet-subnet-id, the pod on karpenter node can not get reponse from default dns nameserver 10.0.0.10.

Actual Behavior

default nodepool

Steps to Reproduce the Problem

each AKS NAP cluster or self-host can reproduce this.

Resource Specs and Logs

(base) ottodeng@Otto-MacBook-Pro-M2.local:/Users/ottodeng/aks/self-host $ kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-deployment-5f7bbf8bb-bkw8z 1/1 Running 0 18m 10.244.4.169 aks-t4-spot-rjz89 nginx-deployment-5f7bbf8bb-hvkvn 1/1 Running 0 18m 10.244.3.132 aks-t4-spot-xg75d nginx-nodepool-5c64bc7fd8-xl92s 1/1 Running 0 18m 10.244.1.82 aks-nodepool1-32105842-vmss000001 nginx-nodepool-5c64bc7fd8-xs2mr 1/1 Running 0 18m 10.244.2.65 aks-nodepool1-32105842-vmss000000 (base) ottodeng@Otto-MacBook-Pro-M2.local:/Users/ottodeng/aks/self-host $ kubectl exec -it nginx-nodepool-5c64bc7fd8-xl92s cat /etc/resolv.conf kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. search default.svc.cluster.local svc.cluster.local cluster.local kdgjdzanqb1uxn5hrh45i34rib.ix.internal.cloudapp.net nameserver 10.97.96.10 options ndots:5 (base) ottodeng@Otto-MacBook-Pro-M2.local:/Users/ottodeng/aks/self-host $ kubectl exec -it nginx-deployment-5f7bbf8bb-bkw8z cat /etc/resolv.conf kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. search default.svc.cluster.local svc.cluster.local cluster.local kdgjdzanqb1uxn5hrh45i34rib.ix.internal.cloudapp.net nameserver 10.0.0.10 options ndots:5 (base) ottodeng@Otto-MacBook-Pro-M2.local:/Users/ottodeng/aks/self-host $

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Bryce-Soghigian commented 4 months ago

im working on a fix for --dns-service-ip here https://github.com/Azure/karpenter-provider-azure/pull/329. For now you will only be able to use the default values for --dns-service-ip until that change gets in.

ottodeng commented 4 months ago

im working on a fix for --dns-service-ip here https://github.com/Azure/karpenter-provider-azure/pull/329. For now you will only be able to use the default values for --dns-service-ip until that change gets in.

Thanks Bryce, will it merge into main in these days?

NMFR commented 2 months ago

Hi,

Any update on this or any workaround?

I am also trying to use Karpenter with a custom subnet and the pod DNS is broken.

gorositopablo commented 1 month ago

Thanks @Bryce-Soghigian . Unfortunately, we have run into the same issue.

Does anyone have any workaround for it?

Thanks

Azure / karpenter-provider-azure