search

rancher / rke2

https://docs.rke2.io/
Apache License 2.0
1.52k stars 265 forks source link

[Release-1.30] - Lack of configuration posibilities to enable DNS cache with Cilium CNI without kubeproxy #6431

Closed thomasferrandiz closed 1 month ago

thomasferrandiz commented 1 month ago

Backport fix for Lack of configuration posibilities to enable DNS cache with Cilium CNI without kubeproxy

fmoral2 commented 1 month ago

Validated on Version:

-$  rke2 version v1.30.3+dev.aaeb5378 (aaeb5378144dac07ceb251373b6ed4297002a369)

Environment Details

Infrastructure Cloud EC2 instance

Node(s) CPU architecture, OS, and Version: PRETTY_NAME="Ubuntu 22.04.1 LTS" NAME="Ubuntu" VERSION_ID="22.04"

Cluster Configuration: 1 node server

Steps to validate the fix

  1. Follow - https://github.com/rancher/rke2/issues/5832#issuecomment-2260030728

    Reproduction Issue:


$ k get pods -A
NAMESPACE     NAME                                                                   READY   STATUS              RESTARTS        AGE
kube-system   cilium-ctbhf                                                           1/1     Running             0               9m1s
kube-system   cilium-operator-84478bd775-6m4bh                                       1/1     Running             3 (106s ago)    9m1s
kube-system   cilium-operator-84478bd775-c46nq                                       0/1     Pending             0               9m1s
kube-system   cloud-controller-manager-ip--.us-east-2.compute.internal   0/1     Running             5 (110s ago)    11m
kube-system   etcd-ip--.us-east-2.compute.internal                       1/1     Running             0               12m
kube-system   helm-install-rke2-cilium-fbhjd                                         0/1     Completed           0               9m38s
kube-system   helm-install-rke2-coredns-p9xrw                                        0/1     Completed           0               9m38s
kube-system   helm-install-rke2-ingress-nginx-ssxsc                                  1/1     Running             0               9m38s
kube-system   helm-install-rke2-metrics-server-sfrdc                                 0/1     Completed           0               9m38s
kube-system   helm-install-rke2-snapshot-controller-crd-lz8qz                        0/1     Completed           0               9m38s
kube-system   helm-install-rke2-snapshot-controller-hhbhj                            0/1     Completed           1               9m38s
kube-system   helm-install-rke2-snapshot-validation-webhook-kxbmf                    0/1     Completed           0               9m38s
kube-system   kube-apiserver-ip--.us-east-2.compute.internal             1/1     Running             0               11m
kube-system   kube-controller-manager-ip--.us-east-2.compute.internal    1/1     Running             3 (3m16s ago)   11m
kube-system   kube-scheduler-ip--.us-east-2.compute.internal             1/1     Running             3 (3m25s ago)   11m
kube-system   node-local-dns-x8pk9                                                   1/1     Running             0               9m1s
kube-system   rke2-coredns-rke2-coredns-64dcf4f58b-ck2kh                             1/1     Running             0               9m1s
kube-system   rke2-coredns-rke2-coredns-autoscaler-5749cd7b8b-x5vbs                  1/1     Running             0               9m1s
kube-system   rke2-ingress-nginx-admission-patch-sqrq4                               0/1     Completed           0               44s
kube-system   rke2-ingress-nginx-controller-67snm                                    0/1     ContainerCreating   0               46s
kube-system   rke2-metrics-server-868fc8795f-6npz8                                   1/1     Running             0               2m1s
kube-system   rke2-snapshot-controller-7dcf5d5b46-5snpr                              1/1     Running             0               103s
kube-system   rke2-snapshot-validation-webhook-bf7bbd6fc-xjqtp                       1/1     Running             0               2m2s

Validation Results:

``` ## install test pod ~$ kubectl run multitool1 --image wbitt/network-multitool ## exec into pod and search dns for any $ kubectl exec multitool1 -- dig suse.com ; <<>> DiG 9.18.16 <<>> suse.com ## query local pod for metrics endpoint. $ curl http:///metrics | grep coredns_dns_requests_total ## only 1 request coredns_dns_requests_total{family="1",proto="udp",server="dns://0.0.0.0:53",type="A",view="",zone="."} 1 ## run dig 6 times $ for i in {1..6}; do kubectl exec multitool1 -- dig suse.com ; done ## query local pod for metrics endpoint. $ curl http:///metrics | grep coredns_dns_requests_total ## 7 requests coredns_dns_requests_total{family="1",proto="udp",server="dns://0.0.0.0:53",type="A",view="",zone="."} 7 k get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE default multitool1 1/1 Running 0 12m kube-system cilium-operator-6df796d759-4h7tb 0/1 Pending 0 22m kube-system cilium-operator-6df796d759-5rblj 1/1 Running 2 (15m ago) 22m kube-system cilium-zt7bv 1/1 Running 0 22m kube-system cloud-controller-manager- .us-east-2.compute.internal 1/1 Running 3 (15m ago) 23m kube-system etcd-ip- .us-east-2.compute.internal 1/1 Running 0 23m kube-system helm-install-rke2-cilium-srkrl 0/1 Completed 0 23m kube-system helm-install-rke2-coredns-sspn9 0/1 Completed 2 23m kube-system helm-install-rke2-ingress-nginx-djl4p 0/1 Completed 0 23m kube-system helm-install-rke2-metrics-server-zcmtb 0/1 Completed 0 23m kube-system helm-install-rke2-snapshot-controller-crd-86dmf 0/1 Completed 0 23m kube-system helm-install-rke2-snapshot-controller-wz8wr 0/1 Completed 1 23m kube-system helm-install-rke2-snapshot-validation-webhook-6wrfw 0/1 Completed 0 23m kube-system kube-apiserver-ip- .us-east-2.compute.internal 1/1 Running 0 22m kube-system kube-controller-manager-ip- .us-east-2.compute.internal 1/1 Running 2 (15m ago) 23m kube-system kube-scheduler-i .us-east-2.compute.internal 1/1 Running 2 (15m ago) 23m kube-system node-local-dns-v5hgk 1/1 Running 0 20m kube-system rke2-coredns-rke2-coredns-6bb85f9dd8-whpsb 1/1 Running 0 20m kube-system rke2-coredns-rke2-coredns-autoscaler-7b9c797d64-ql7bx 1/1 Running 0 20m kube-system rke2-ingress-nginx-controller-rnddg 1/1 Running 0 16m kube-system rke2-metrics-server-868fc8795f-ks69v 1/1 Running 0 17m kube-system rke2-snapshot-controller-7dcf5d5b46-27vrv 1/1 Running 1 (15m ago) 17m kube-system rke2-snapshot-validation-webhook-bf7bbd6fc-wdkvv 1/1 Running 0 17m ```