search

loft-sh / vcluster

vCluster - Create fully functional virtual Kubernetes clusters - Each vcluster runs inside a namespace of the underlying k8s cluster. It's cheaper than creating separate full-blown clusters and it offers better multi-tenancy and isolation than regular namespaces.
https://www.vcluster.com
Apache License 2.0
6.26k stars 398 forks source link

Syncer fails to update CoreDNS manifest after pod restart #358

Closed mixitgit closed 2 years ago

mixitgit commented 2 years ago

When I restart pod with vcluster, syncer is trying to update CoreDNS manifest, but fails Syncer version: 0.5.3

Syncer logs: 

I0204 12:57:28.700104       1 start.go:304] Failed to apply CoreDNS configuration from the manifest file: error when applying patch:
{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apps/v1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{},\"labels\":{\"k8s-app\":\"kube-dns\",\"kubernetes.io/name\":\"CoreDNS\"},\"name\":\"coredns\",\"namespace\":\"kube-system\"},\"spec\":{\"replicas\":1,\"selector\":{\"matchLabels\":{\"k8s-app\":\"kube-dns\"}},\"strategy\":{\"rollingUpdate\":{\"maxUnavailable\":1},\"type\":\"RollingUpdate\"},\"template\":{\"metadata\":{\"labels\":{\"k8s-app\":\"kube-dns\"}},\"spec\":{\"containers\":[{\"args\":[\"-conf\",\"/etc/coredns/Corefile\"],\"image\":\"coredns/coredns:1.8.0\",\"imagePullPolicy\":\"IfNotPresent\",\"livenessProbe\":{\"failureThreshold\":3,\"httpGet\":{\"path\":\"/health\",\"port\":8080,\"scheme\":\"HTTP\"},\"initialDelaySeconds\":60,\"periodSeconds\":10,\"successThreshold\":1,\"timeoutSeconds\":1},\"name\":\"coredns\",\"ports\":[{\"containerPort\":1053,\"name\":\"dns\",\"protocol\":\"UDP\"},{\"containerPort\":1053,\"name\":\"dns-tcp\",\"protocol\":\"TCP\"},{\"containerPort\":9153,\"name\":\"metrics\",\"protocol\":\"TCP\"}],\"readinessProbe\":{\"failureThreshold\":3,\"httpGet\":{\"path\":\"/ready\",\"port\":8181,\"scheme\":\"HTTP\"},\"initialDelaySeconds\":0,\"periodSeconds\":2,\"successThreshold\":1,\"timeoutSeconds\":1},\"resources\":{\"limits\":{\"cpu\":\"1000m\",\"memory\":\"170Mi\"},\"requests\":{\"cpu\":\"100m\",\"memory\":\"70Mi\"}},\"securityContext\":{\"allowPrivilegeEscalation\":false,\"capabilities\":{\"drop\":[\"all\"]},\"readOnlyRootFilesystem\":true,\"runAsNonRoot\":false,\"runAsUser\":0},\"volumeMounts\":[{\"mountPath\":\"/etc/coredns\",\"name\":\"config-volume\",\"readOnly\":true}]}],\"dnsPolicy\":\"Default\",\"nodeSelector\":{\"kubernetes.io/os\":\"linux\"},\"priorityClassName\":\"system-cluster-critical\",\"serviceAccountName\":\"coredns\",\"tolerations\":[{\"key\":\"CriticalAddonsOnly\",\"operator\":\"Exists\"},{\"effect\":\"NoSchedule\",\"key\":\"node-role.kubernetes.io/control-plane\",\"operator\":\"Exists\"},{\"effect\":\"NoSchedule\",\"key\":\"node-role.kubernetes.io/master\",\"operator\":\"Exists\"}],\"topologySpreadConstraints\":[{\"labelSelector\":{\"matchLabels\":{\"k8s-app\":\"kube-dns\"}},\"maxSkew\":1,\"topologyKey\":\"kubernetes.io/hostname\",\"whenUnsatisfiable\":\"DoNotSchedule\"}],\"volumes\":[{\"configMap\":{\"items\":[{\"key\":\"Corefile\",\"path\":\"Corefile\"},{\"key\":\"NodeHosts\",\"path\":\"NodeHosts\"}],\"name\":\"coredns\"},\"name\":\"config-volume\"}]}}}}\n"}},"spec":{"template":{"spec":{"$setElementOrder/containers":[{"name":"coredns"}],"containers":[{"$setElementOrder/ports":[{"containerPort":1053},{"containerPort":1053},{"containerPort":9153}],"image":"coredns/coredns:1.8.0","name":"coredns","ports":[{"containerPort":1053,"name":"dns-tcp","protocol":"TCP"},{"containerPort":1053,"name":"dns","protocol":"UDP"}],"readinessProbe":{"initialDelaySeconds":0},"resources":{"limits":{"cpu":"1000m"}},"securityContext":{"runAsNonRoot":false,"runAsUser":0}}],"nodeSelector":{"kubernetes.io/os":"linux"},"topologySpreadConstraints":[{"labelSelector":{"matchLabels":{"k8s-app":"kube-dns"}},"maxSkew":1,"topologyKey":"kubernetes.io/hostname","whenUnsatisfiable":"DoNotSchedule"}]}}}}
to:
Resource: "apps/v1, Resource=deployments", GroupVersionKind: "apps/v1, Kind=Deployment"
Name: "coredns", Namespace: "kube-system"
for: "manifestString": Deployment.apps "coredns" is invalid: spec.template.spec.containers[0].ports[1].name: Duplicate value: "dns"
I0204 12:57:28.704200       1 logr.go:249] configmap: coredns: updating physical vc-vd-01/coredns-x-kube-system-x-vc-vd-01, because virtual configmap have changed

K3s logs:

time="2022-02-04T13:03:01.504695321Z" level=debug msg="DesiredSet - No change(2) /v1, Kind=ServiceAccount kube-system/coredns for  kube-system/coredns"
time="2022-02-04T13:03:01.508712697Z" level=debug msg="DesiredSet - No change(2) rbac.authorization.k8s.io/v1beta1, Kind=ClusterRole /system:coredns for  kube-system/coredns"
time="2022-02-04T13:03:01.512011878Z" level=debug msg="DesiredSet - No change(2) rbac.authorization.k8s.io/v1beta1, Kind=ClusterRoleBinding /system:coredns for  kube-system/coredns"
time="2022-02-04T13:03:01.514639337Z" level=debug msg="DesiredSet - No change(2) /v1, Kind=ConfigMap kube-system/coredns for  kube-system/coredns"
time="2022-02-04T13:03:01.518752111Z" level=debug msg="DesiredSet - Patch apps/v1, Kind=Deployment kube-system/coredns for  kube-system/coredns -- [{\"spec\":{\"template\":{\"spec\":{\"$setElementOrder/containers\":[{\"name\":\"coredns\"}],\"containers\":[{\"name\":\"coredns\",\"readinessProbe\":{\"initialDelaySeconds\":0}}]}}}}, {\"apiVersion\":\"apps/v1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{\"objectset.rio.cattle.io/applied\":\"H4sIAAAAAAAA/6xV32/bNhD+V4Z7ln8obhNXwB66OFuDLZ6RH30pjOJMnSzOFI8jKTdGoP99OEl2nDRNu2FPpsnvjt/38e70AOj0R/JBs4UM0Lkw2qaQwEbbHDKYkTO8q8hGSKCiiDlGhOwB0FqOGDXbIH959RepGCgOveahwhgNDTWPtCSB5Jvn/MWSH6y3G8hgMwlHJ9s0+el3bfOf3+c52++msFgRZKDYU27DD8GDQyUxm3pFg7ALkSpoEjC4ItOK2kzDAJ3bQ7q8svSWIgXJ1l97zp5m85tXri0xlJDBStHJdHLybjpN07M3ExxPpqe4epuOi5Pi9IyKszcnb8bq7ZkQ+UrSK6SDIyWUAxlSkb2sK4yq/OM1NY1ERo+R1juBeDZG2/WdyzFSl+L+zuIWtcGVIcjSJoG4c0Lg+glW9qlyZh93VCmv+tkccVdsI2pLPkD26QHQr2UBA8W2gARGFNWoN2MkhhfaECwT0BWuhZFHq0rye8yg/83S4XQ4gR63qI1ZsNFqBxlcFnOOC0+hq2+jt2QphIXnVSujQG1qT7elp1CyySGbJFDG6H6jKOcOozzqqCQ0sYQEHPsI2XQ8HScQVEntA364vV2IQdrqqNHMyODuhhTbPEB2Ok7AkdecH7ZSCa6VohCObk4TiLoiruMj8KUiEQqdgQc/Fy2rt5MDukd6jqzYQAZ3M2H4nZBBVO5p2O35i2Hv0qPAiqLXKrwQuEzAE+b6P1kukbtHx9Np+qOOf234yb/w21Pg2itqC9roSsfQFXzFXkoqPRtfaWiBf9cUulPlajkaj6t2ivbQDikNQKr2Ou7O2Ua6b2WiMfxl4fVWG1rTRVBo2mELWYEmUAIKHa600VF3VDDPpVnmF7eff7mczz7fXFx/vDy/kP7IPTs5Q2Ng2XSm/2nN7po5/qoN9VMki76mJoEtm7qiK65tX0eVLBe970dNCEfVZwu9HnSR8HhDn3MpT53bcOi7GRVYG2k5yzndHM2sFUUcPh2xHCADo219L7Y6r7n1ymAI8+76TsBAmTpE8gPlddQKDYizfqsVvVdKNMyf90pkQ37/Efv0ABsSdud9fPvhCa2OBNgJUkjCxb2Wd5XCp6IgFSGDOd+okvLaiPwujUgbeDb0TI80i2czcAYt/a+ZKxT9L6dc7h/2MBsKvb5CJ6brSNUTA/bDNdm322FHqHWgOef0gSX3AfW4Jdc9m0zNN6qlWTZN0/wTAAD//9NEnByICAAA\",\"objectset.rio.cattle.io/id\":\"\",\"objectset.rio.cattle.io/owner-gvk\":\"k3s.cattle.io/v1, Kind=Addon\",\"objectset.rio.cattle.io/owner-name\":\"coredns\",\"objectset.rio.cattle.io/owner-namespace\":\"kube-system\"},\"labels\":{\"k8s-app\":\"kube-dns\",\"kubernetes.io/name\":\"CoreDNS\",\"objectset.rio.cattle.io/hash\":\"bce283298811743a0386ab510f2f67ef74240c57\"},\"name\":\"coredns\",\"namespace\":\"kube-system\"},\"spec\":{\"selector\":{\"matchLabels\":{\"k8s-app\":\"kube-dns\"}},\"strategy\":{\"rollingUpdate\":{\"maxUnavailable\":1},\"type\":\"RollingUpdate\"},\"template\":{\"metadata\":{\"labels\":{\"k8s-app\":\"kube-dns\"}},\"spec\":{\"containers\":[{\"args\":[\"-conf\",\"/etc/coredns/Corefile\"],\"image\":\"rancher/coredns-coredns:1.8.3\",\"imagePullPolicy\":\"IfNotPresent\",\"livenessProbe\":{\"failureThreshold\":3,\"httpGet\":{\"path\":\"/health\",\"port\":8080,\"scheme\":\"HTTP\"},\"initialDelaySeconds\":60,\"periodSeconds\":10,\"successThreshold\":1,\"timeoutSeconds\":1},\"name\":\"coredns\",\"ports\":[{\"containerPort\":53,\"name\":\"dns\",\"protocol\":\"UDP\"},{\"containerPort\":53,\"name\":\"dns-tcp\",\"protocol\":\"TCP\"},{\"containerPort\":9153,\"name\":\"metrics\",\"protocol\":\"TCP\"}],\"readinessProbe\":{\"failureThreshold\":3,\"httpGet\":{\"path\":\"/ready\",\"port\":8181,\"scheme\":\"HTTP\"},\"initialDelaySeconds\":0,\"periodSeconds\":2,\"successThreshold\":1,\"timeoutSeconds\":1},\"resources\":{\"limits\":{\"memory\":\"170Mi\"},\"requests\":{\"cpu\":\"100m\",\"memory\":\"70Mi\"}},\"securityContext\":{\"allowPrivilegeEscalation\":false,\"capabilities\":{\"add\":[\"NET_BIND_SERVICE\"],\"drop\":[\"all\"]},\"readOnlyRootFilesystem\":true},\"volumeMounts\":[{\"mountPath\":\"/etc/coredns\",\"name\":\"config-volume\",\"readOnly\":true}]}],\"dnsPolicy\":\"Default\",\"nodeSelector\":{\"beta.kubernetes.io/os\":\"linux\"},\"priorityClassName\":\"system-cluster-critical\",\"serviceAccountName\":\"coredns\",\"tolerations\":[{\"key\":\"CriticalAddonsOnly\",\"operator\":\"Exists\"},{\"effect\":\"NoSchedule\",\"key\":\"node-role.kubernetes.io/control-plane\",\"operator\":\"Exists\"},{\"effect\":\"NoSchedule\",\"key\":\"node-role.kubernetes.io/master\",\"operator\":\"Exists\"}],\"volumes\":[{\"configMap\":{\"items\":[{\"key\":\"Corefile\",\"path\":\"Corefile\"},{\"key\":\"NodeHosts\",\"path\":\"NodeHosts\"}],\"name\":\"coredns\"},\"name\":\"config-volume\"}]}}}}, {\"apiVersion\":\"apps/v1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{\"objectset.rio.cattle.io/applied\":\"H4sIAAAAAAAA/6xV32/bNhD+V4Z7ln8obhNXwB66OFuDLZ6RH30pjOJMnSzOFI8jKTdGoP99OEl2nDRNu2FPpsnvjt/38e70AOj0R/JBs4UM0Lkw2qaQwEbbHDKYkTO8q8hGSKCiiDlGhOwB0FqOGDXbIH959RepGCgOveahwhgNDTWPtCSB5Jvn/MWSH6y3G8hgMwlHJ9s0+el3bfOf3+c52++msFgRZKDYU27DD8GDQyUxm3pFg7ALkSpoEjC4ItOK2kzDAJ3bQ7q8svSWIgXJ1l97zp5m85tXri0xlJDBStHJdHLybjpN07M3ExxPpqe4epuOi5Pi9IyKszcnb8bq7ZkQ+UrSK6SDIyWUAxlSkb2sK4yq/OM1NY1ERo+R1juBeDZG2/WdyzFSl+L+zuIWtcGVIcjSJoG4c0Lg+glW9qlyZh93VCmv+tkccVdsI2pLPkD26QHQr2UBA8W2gARGFNWoN2MkhhfaECwT0BWuhZFHq0rye8yg/83S4XQ4gR63qI1ZsNFqBxlcFnOOC0+hq2+jt2QphIXnVSujQG1qT7elp1CyySGbJFDG6H6jKOcOozzqqCQ0sYQEHPsI2XQ8HScQVEntA364vV2IQdrqqNHMyODuhhTbPEB2Ok7AkdecH7ZSCa6VohCObk4TiLoiruMj8KUiEQqdgQc/Fy2rt5MDukd6jqzYQAZ3M2H4nZBBVO5p2O35i2Hv0qPAiqLXKrwQuEzAE+b6P1kukbtHx9Np+qOOf234yb/w21Pg2itqC9roSsfQFXzFXkoqPRtfaWiBf9cUulPlajkaj6t2ivbQDikNQKr2Ou7O2Ua6b2WiMfxl4fVWG1rTRVBo2mELWYEmUAIKHa600VF3VDDPpVnmF7eff7mczz7fXFx/vDy/kP7IPTs5Q2Ng2XSm/2nN7po5/qoN9VMki76mJoEtm7qiK65tX0eVLBe970dNCEfVZwu9HnSR8HhDn3MpT53bcOi7GRVYG2k5yzndHM2sFUUcPh2xHCADo219L7Y6r7n1ymAI8+76TsBAmTpE8gPlddQKDYizfqsVvVdKNMyf90pkQ37/Efv0ABsSdud9fPvhCa2OBNgJUkjCxb2Wd5XCp6IgFSGDOd+okvLaiPwujUgbeDb0TI80i2czcAYt/a+ZKxT9L6dc7h/2MBsKvb5CJ6brSNUTA/bDNdm322FHqHWgOef0gSX3AfW4Jdc9m0zNN6qlWTZN0/wTAAD//9NEnByICAAA\",\"objectset.rio.cattle.io/id\":\"\",\"objectset.rio.cattle.io/owner-gvk\":\"k3s.cattle.io/v1, Kind=Addon\",\"objectset.rio.cattle.io/owner-name\":\"coredns\",\"objectset.rio.cattle.io/owner-namespace\":\"kube-system\"},\"labels\":{\"k8s-app\":\"kube-dns\",\"kubernetes.io/name\":\"CoreDNS\",\"objectset.rio.cattle.io/hash\":\"bce283298811743a0386ab510f2f67ef74240c57\"},\"name\":\"coredns\",\"namespace\":\"kube-system\"},\"spec\":{\"selector\":{\"matchLabels\":{\"k8s-app\":\"kube-dns\"}},\"strategy\":{\"rollingUpdate\":{\"maxUnavailable\":1},\"type\":\"RollingUpdate\"},\"template\":{\"metadata\":{\"labels\":{\"k8s-app\":\"kube-dns\"}},\"spec\":{\"containers\":[{\"args\":[\"-conf\",\"/etc/coredns/Corefile\"],\"image\":\"rancher/coredns-coredns:1.8.3\",\"imagePullPolicy\":\"IfNotPresent\",\"livenessProbe\":{\"failureThreshold\":3,\"httpGet\":{\"path\":\"/health\",\"port\":8080,\"scheme\":\"HTTP\"},\"initialDelaySeconds\":60,\"periodSeconds\":10,\"successThreshold\":1,\"timeoutSeconds\":1},\"name\":\"coredns\",\"ports\":[{\"containerPort\":53,\"name\":\"dns\",\"protocol\":\"UDP\"},{\"containerPort\":53,\"name\":\"dns-tcp\",\"protocol\":\"TCP\"},{\"containerPort\":9153,\"name\":\"metrics\",\"protocol\":\"TCP\"}],\"readinessProbe\":{\"failureThreshold\":3,\"httpGet\":{\"path\":\"/ready\",\"port\":8181,\"scheme\":\"HTTP\"},\"initialDelaySeconds\":0,\"periodSeconds\":2,\"successThreshold\":1,\"timeoutSeconds\":1},\"resources\":{\"limits\":{\"memory\":\"170Mi\"},\"requests\":{\"cpu\":\"100m\",\"memory\":\"70Mi\"}},\"securityContext\":{\"allowPrivilegeEscalation\":false,\"capabilities\":{\"add\":[\"NET_BIND_SERVICE\"],\"drop\":[\"all\"]},\"readOnlyRootFilesystem\":true},\"volumeMounts\":[{\"mountPath\":\"/etc/coredns\",\"name\":\"config-volume\",\"readOnly\":true}]}],\"dnsPolicy\":\"Default\",\"nodeSelector\":{\"beta.kubernetes.io/os\":\"linux\"},\"priorityClassName\":\"system-cluster-critical\",\"serviceAccountName\":\"coredns\",\"tolerations\":[{\"key\":\"CriticalAddonsOnly\",\"operator\":\"Exists\"},{\"effect\":\"NoSchedule\",\"key\":\"node-role.kubernetes.io/control-plane\",\"operator\":\"Exists\"},{\"effect\":\"NoSchedule\",\"key\":\"node-role.kubernetes.io/master\",\"operator\":\"Exists\"}],\"volumes\":[{\"configMap\":{\"items\":[{\"key\":\"Corefile\",\"path\":\"Corefile\"},{\"key\":\"NodeHosts\",\"path\":\"NodeHosts\"}],\"name\":\"coredns\"},\"name\":\"config-volume\"}]}}}}, {\"apiVersion\":\"apps/v1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{\"deployment.kubernetes.io/revision\":\"1\",\"objectset.rio.cattle.io/applied\":\"H4sIAAAAAAAA/6xV32/bNhD+V4Z7ln8obhNXwB66OFuDLZ6RH30pjOJMnSzOFI8jKTdGoP99OEl2nDRNu2FPpsnvjt/38e70AOj0R/JBs4UM0Lkw2qaQwEbbHDKYkTO8q8hGSKCiiDlGhOwB0FqOGDXbIH959RepGCgOveahwhgNDTWPtCSB5Jvn/MWSH6y3G8hgMwlHJ9s0+el3bfOf3+c52++msFgRZKDYU27DD8GDQyUxm3pFg7ALkSpoEjC4ItOK2kzDAJ3bQ7q8svSWIgXJ1l97zp5m85tXri0xlJDBStHJdHLybjpN07M3ExxPpqe4epuOi5Pi9IyKszcnb8bq7ZkQ+UrSK6SDIyWUAxlSkb2sK4yq/OM1NY1ERo+R1juBeDZG2/WdyzFSl+L+zuIWtcGVIcjSJoG4c0Lg+glW9qlyZh93VCmv+tkccVdsI2pLPkD26QHQr2UBA8W2gARGFNWoN2MkhhfaECwT0BWuhZFHq0rye8yg/83S4XQ4gR63qI1ZsNFqBxlcFnOOC0+hq2+jt2QphIXnVSujQG1qT7elp1CyySGbJFDG6H6jKOcOozzqqCQ0sYQEHPsI2XQ8HScQVEntA364vV2IQdrqqNHMyODuhhTbPEB2Ok7AkdecH7ZSCa6VohCObk4TiLoiruMj8KUiEQqdgQc/Fy2rt5MDukd6jqzYQAZ3M2H4nZBBVO5p2O35i2Hv0qPAiqLXKrwQuEzAE+b6P1kukbtHx9Np+qOOf234yb/w21Pg2itqC9roSsfQFXzFXkoqPRtfaWiBf9cUulPlajkaj6t2ivbQDikNQKr2Ou7O2Ua6b2WiMfxl4fVWG1rTRVBo2mELWYEmUAIKHa600VF3VDDPpVnmF7eff7mczz7fXFx/vDy/kP7IPTs5Q2Ng2XSm/2nN7po5/qoN9VMki76mJoEtm7qiK65tX0eVLBe970dNCEfVZwu9HnSR8HhDn3MpT53bcOi7GRVYG2k5yzndHM2sFUUcPh2xHCADo219L7Y6r7n1ymAI8+76TsBAmTpE8gPlddQKDYizfqsVvVdKNMyf90pkQ37/Efv0ABsSdud9fPvhCa2OBNgJUkjCxb2Wd5XCp6IgFSGDOd+okvLaiPwujUgbeDb0TI80i2czcAYt/a+ZKxT9L6dc7h/2MBsKvb5CJ6brSNUTA/bDNdm322FHqHWgOef0gSX3AfW4Jdc9m0zNN6qlWTZN0/wTAAD//9NEnByICAAA\",\"objectset.rio.cattle.io/id\":\"\",\"objectset.rio.cattle.io/owner-gvk\":\"k3s.cattle.io/v1, Kind=Addon\",\"objectset.rio.cattle.io/owner-name\":\"coredns\",\"objectset.rio.cattle.io/owner-namespace\":\"kube-system\"},\"creationTimestamp\":\"2022-02-04T12:52:03Z\",\"generation\":1,\"labels\":{\"k8s-app\":\"kube-dns\",\"kubernetes.io/name\":\"CoreDNS\",\"objectset.rio.cattle.io/hash\":\"bce283298811743a0386ab510f2f67ef74240c57\"},\"managedFields\":[{\"apiVersion\":\"apps/v1\",\"fieldsType\":\"FieldsV1\",\"fieldsV1\":{\"f:metadata\":{\"f:annotations\":{\".\":{},\"f:deployment.kubernetes.io/revision\":{},\"f:objectset.rio.cattle.io/applied\":{},\"f:objectset.rio.cattle.io/id\":{},\"f:objectset.rio.cattle.io/owner-gvk\":{},\"f:objectset.rio.cattle.io/owner-name\":{},\"f:objectset.rio.cattle.io/owner-namespace\":{}},\"f:labels\":{\".\":{},\"f:k8s-app\":{},\"f:kubernetes.io/name\":{},\"f:objectset.rio.cattle.io/hash\":{}}},\"f:spec\":{\"f:progressDeadlineSeconds\":{},\"f:replicas\":{},\"f:revisionHistoryLimit\":{},\"f:selector\":{},\"f:strategy\":{\"f:rollingUpdate\":{\".\":{},\"f:maxSurge\":{},\"f:maxUnavailable\":{}},\"f:type\":{}},\"f:template\":{\"f:metadata\":{\"f:labels\":{\".\":{},\"f:k8s-app\":{}}},\"f:spec\":{\"f:containers\":{\"k:{\\\"name\\\":\\\"coredns\\\"}\":{\".\":{},\"f:args\":{},\"f:image\":{},\"f:imagePullPolicy\":{},\"f:livenessProbe\":{\".\":{},\"f:failureThreshold\":{},\"f:httpGet\":{\".\":{},\"f:path\":{},\"f:port\":{},\"f:scheme\":{}},\"f:initialDelaySeconds\":{},\"f:periodSeconds\":{},\"f:successThreshold\":{},\"f:timeoutSeconds\":{}},\"f:name\":{},\"f:ports\":{\".\":{},\"k:{\\\"containerPort\\\":53,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:containerPort\":{},\"f:name\":{},\"f:protocol\":{}},\"k:{\\\"containerPort\\\":53,\\\"protocol\\\":\\\"UDP\\\"}\":{\".\":{},\"f:containerPort\":{},\"f:name\":{},\"f:protocol\":{}},\"k:{\\\"containerPort\\\":9153,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:containerPort\":{},\"f:name\":{},\"f:protocol\":{}}},\"f:readinessProbe\":{\".\":{},\"f:failureThreshold\":{},\"f:httpGet\":{\".\":{},\"f:path\":{},\"f:port\":{},\"f:scheme\":{}},\"f:periodSeconds\":{},\"f:successThreshold\":{},\"f:timeoutSeconds\":{}},\"f:resources\":{\".\":{},\"f:limits\":{\".\":{},\"f:memory\":{}},\"f:requests\":{\".\":{},\"f:cpu\":{},\"f:memory\":{}}},\"f:securityContext\":{\".\":{},\"f:allowPrivilegeEscalation\":{},\"f:capabilities\":{\".\":{},\"f:add\":{},\"f:drop\":{}},\"f:readOnlyRootFilesystem\":{}},\"f:terminationMessagePath\":{},\"f:terminationMessagePolicy\":{},\"f:volumeMounts\":{\".\":{},\"k:{\\\"mountPath\\\":\\\"/etc/coredns\\\"}\":{\".\":{},\"f:mountPath\":{},\"f:name\":{},\"f:readOnly\":{}}}}},\"f:dnsPolicy\":{},\"f:nodeSelector\":{\".\":{},\"f:beta.kubernetes.io/os\":{}},\"f:priorityClassName\":{},\"f:restartPolicy\":{},\"f:schedulerName\":{},\"f:securityContext\":{},\"f:serviceAccount\":{},\"f:serviceAccountName\":{},\"f:terminationGracePeriodSeconds\":{},\"f:tolerations\":{},\"f:volumes\":{\".\":{},\"k:{\\\"name\\\":\\\"config-volume\\\"}\":{\".\":{},\"f:configMap\":{\".\":{},\"f:defaultMode\":{},\"f:items\":{},\"f:name\":{}},\"f:name\":{}}}}}},\"f:status\":{\"f:availableReplicas\":{},\"f:conditions\":{\".\":{},\"k:{\\\"type\\\":\\\"Available\\\"}\":{\".\":{},\"f:lastTransitionTime\":{},\"f:lastUpdateTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}},\"k:{\\\"type\\\":\\\"Progressing\\\"}\":{\".\":{},\"f:lastTransitionTime\":{},\"f:lastUpdateTime\":{},\"f:message\":{},\"f:reason\":{},\"f:status\":{},\"f:type\":{}}},\"f:observedGeneration\":{},\"f:readyReplicas\":{},\"f:replicas\":{},\"f:updatedReplicas\":{}}},\"manager\":\"k3s\",\"operation\":\"Update\",\"time\":\"2022-02-04T12:52:32Z\"}],\"name\":\"coredns\",\"namespace\":\"kube-system\",\"resourceVersion\":\"414\",\"uid\":\"f65c7ff3-c320-48c4-aef9-313c043b0fe0\"},\"spec\":{\"progressDeadlineSeconds\":600,\"replicas\":1,\"revisionHistoryLimit\":10,\"selector\":{\"matchLabels\":{\"k8s-app\":\"kube-dns\"}},\"strategy\":{\"rollingUpdate\":{\"maxSurge\":\"25%\",\"maxUnavailable\":1},\"type\":\"RollingUpdate\"},\"template\":{\"metadata\":{\"creationTimestamp\":null,\"labels\":{\"k8s-app\":\"kube-dns\"}},\"spec\":{\"containers\":[{\"args\":[\"-conf\",\"/etc/coredns/Corefile\"],\"image\":\"rancher/coredns-coredns:1.8.3\",\"imagePullPolicy\":\"IfNotPresent\",\"livenessProbe\":{\"failureThreshold\":3,\"httpGet\":{\"path\":\"/health\",\"port\":8080,\"scheme\":\"HTTP\"},\"initialDelaySeconds\":60,\"periodSeconds\":10,\"successThreshold\":1,\"timeoutSeconds\":1},\"name\":\"coredns\",\"ports\":[{\"containerPort\":53,\"name\":\"dns\",\"protocol\":\"UDP\"},{\"containerPort\":53,\"name\":\"dns-tcp\",\"protocol\":\"TCP\"},{\"containerPort\":9153,\"name\":\"metrics\",\"protocol\":\"TCP\"}],\"readinessProbe\":{\"failureThreshold\":3,\"httpGet\":{\"path\":\"/ready\",\"port\":8181,\"scheme\":\"HTTP\"},\"periodSeconds\":2,\"successThreshold\":1,\"timeoutSeconds\":1},\"resources\":{\"limits\":{\"memory\":\"170Mi\"},\"requests\":{\"cpu\":\"100m\",\"memory\":\"70Mi\"}},\"securityContext\":{\"allowPrivilegeEscalation\":false,\"capabilities\":{\"add\":[\"NET_BIND_SERVICE\"],\"drop\":[\"all\"]},\"readOnlyRootFilesystem\":true},\"terminationMessagePath\":\"/dev/termination-log\",\"terminationMessagePolicy\":\"File\",\"volumeMounts\":[{\"mountPath\":\"/etc/coredns\",\"name\":\"config-volume\",\"readOnly\":true}]}],\"dnsPolicy\":\"Default\",\"nodeSelector\":{\"beta.kubernetes.io/os\":\"linux\"},\"priorityClassName\":\"system-cluster-critical\",\"restartPolicy\":\"Always\",\"schedulerName\":\"default-scheduler\",\"securityContext\":{},\"serviceAccount\":\"coredns\",\"serviceAccountName\":\"coredns\",\"terminationGracePeriodSeconds\":30,\"tolerations\":[{\"key\":\"CriticalAddonsOnly\",\"operator\":\"Exists\"},{\"effect\":\"NoSchedule\",\"key\":\"node-role.kubernetes.io/control-plane\",\"operator\":\"Exists\"},{\"effect\":\"NoSchedule\",\"key\":\"node-role.kubernetes.io/master\",\"operator\":\"Exists\"}],\"volumes\":[{\"configMap\":{\"defaultMode\":420,\"items\":[{\"key\":\"Corefile\",\"path\":\"Corefile\"},{\"key\":\"NodeHosts\",\"path\":\"NodeHosts\"}],\"name\":\"coredns\"},\"name\":\"config-volume\"}]}}},\"status\":{\"availableReplicas\":1,\"conditions\":[{\"lastTransitionTime\":\"2022-02-04T12:52:07Z\",\"lastUpdateTime\":\"2022-02-04T12:52:07Z\",\"message\":\"Deployment has minimum availability.\",\"reason\":\"MinimumReplicasAvailable\",\"status\":\"True\",\"type\":\"Available\"},{\"lastTransitionTime\":\"2022-02-04T12:52:07Z\",\"lastUpdateTime\":\"2022-02-04T12:52:32Z\",\"message\":\"ReplicaSet \\\"coredns-6488c6fcc6\\\" has successfully progressed.\",\"reason\":\"NewReplicaSetAvailable\",\"status\":\"True\",\"type\":\"Progressing\"}],\"observedGeneration\":1,\"readyReplicas\":1,\"replicas\":1,\"updatedReplicas\":1}}]"
time="2022-02-04T13:03:01.520156954Z" level=debug msg="DesiredSet - Updated apps/v1, Kind=Deployment kube-system/coredns for  kube-system/coredns -- application/strategic-merge-patch+json {\"spec\":{\"template\":{\"spec\":{\"$setElementOrder/containers\":[{\"name\":\"coredns\"}],\"containers\":[{\"name\":\"coredns\",\"readinessProbe\":{\"initialDelaySeconds\":0}}]}}}}"
time="2022-02-04T13:03:01.531142631Z" level=debug msg="DesiredSet - Patch /v1, Kind=Service kube-system/kube-dns for  kube-system/coredns -- [{\"spec\":{\"clusterIP\":\"10.247.0.10\"}}, {\"apiVersion\":\"v1\",\"kind\":\"Service\",\"metadata\":{\"annotations\":{\"objectset.rio.cattle.io/applied\":\"{\\\"apiVersion\\\":\\\"v1\\\",\\\"kind\\\":\\\"Service\\\",\\\"metadata\\\":{\\\"annotations\\\":{\\\"objectset.rio.cattle.io/id\\\":\\\"\\\",\\\"objectset.rio.cattle.io/owner-gvk\\\":\\\"k3s.cattle.io/v1, Kind=Addon\\\",\\\"objectset.rio.cattle.io/owner-name\\\":\\\"coredns\\\",\\\"objectset.rio.cattle.io/owner-namespace\\\":\\\"kube-system\\\",\\\"prometheus.io/port\\\":\\\"9153\\\",\\\"prometheus.io/scrape\\\":\\\"true\\\"},\\\"labels\\\":{\\\"k8s-app\\\":\\\"kube-dns\\\",\\\"kubernetes.io/cluster-service\\\":\\\"true\\\",\\\"kubernetes.io/name\\\":\\\"CoreDNS\\\",\\\"objectset.rio.cattle.io/hash\\\":\\\"bce283298811743a0386ab510f2f67ef74240c57\\\"},\\\"name\\\":\\\"kube-dns\\\",\\\"namespace\\\":\\\"kube-system\\\"},\\\"spec\\\":{\\\"clusterIP\\\":\\\"10.247.0.10\\\",\\\"ports\\\":[{\\\"name\\\":\\\"dns\\\",\\\"port\\\":53,\\\"protocol\\\":\\\"UDP\\\"},{\\\"name\\\":\\\"dns-tcp\\\",\\\"port\\\":53,\\\"protocol\\\":\\\"TCP\\\"},{\\\"name\\\":\\\"metrics\\\",\\\"port\\\":9153,\\\"protocol\\\":\\\"TCP\\\"}],\\\"selector\\\":{\\\"k8s-app\\\":\\\"kube-dns\\\"}}}\",\"objectset.rio.cattle.io/id\":\"\",\"objectset.rio.cattle.io/owner-gvk\":\"k3s.cattle.io/v1, Kind=Addon\",\"objectset.rio.cattle.io/owner-name\":\"coredns\",\"objectset.rio.cattle.io/owner-namespace\":\"kube-system\",\"prometheus.io/port\":\"9153\",\"prometheus.io/scrape\":\"true\"},\"labels\":{\"k8s-app\":\"kube-dns\",\"kubernetes.io/cluster-service\":\"true\",\"kubernetes.io/name\":\"CoreDNS\",\"objectset.rio.cattle.io/hash\":\"bce283298811743a0386ab510f2f67ef74240c57\"},\"name\":\"kube-dns\",\"namespace\":\"kube-system\"},\"spec\":{\"clusterIP\":\"10.247.0.10\",\"ports\":[{\"name\":\"dns\",\"port\":53,\"protocol\":\"UDP\"},{\"name\":\"dns-tcp\",\"port\":53,\"protocol\":\"TCP\"},{\"name\":\"metrics\",\"port\":9153,\"protocol\":\"TCP\"}],\"selector\":{\"k8s-app\":\"kube-dns\"}}}, {\"apiVersion\":\"v1\",\"kind\":\"Service\",\"metadata\":{\"annotations\":{\"objectset.rio.cattle.io/applied\":\"{\\\"apiVersion\\\":\\\"v1\\\",\\\"kind\\\":\\\"Service\\\",\\\"metadata\\\":{\\\"annotations\\\":{\\\"objectset.rio.cattle.io/id\\\":\\\"\\\",\\\"objectset.rio.cattle.io/owner-gvk\\\":\\\"k3s.cattle.io/v1, Kind=Addon\\\",\\\"objectset.rio.cattle.io/owner-name\\\":\\\"coredns\\\",\\\"objectset.rio.cattle.io/owner-namespace\\\":\\\"kube-system\\\",\\\"prometheus.io/port\\\":\\\"9153\\\",\\\"prometheus.io/scrape\\\":\\\"true\\\"},\\\"labels\\\":{\\\"k8s-app\\\":\\\"kube-dns\\\",\\\"kubernetes.io/cluster-service\\\":\\\"true\\\",\\\"kubernetes.io/name\\\":\\\"CoreDNS\\\",\\\"objectset.rio.cattle.io/hash\\\":\\\"bce283298811743a0386ab510f2f67ef74240c57\\\"},\\\"name\\\":\\\"kube-dns\\\",\\\"namespace\\\":\\\"kube-system\\\"},\\\"spec\\\":{\\\"clusterIP\\\":\\\"10.247.0.10\\\",\\\"ports\\\":[{\\\"name\\\":\\\"dns\\\",\\\"port\\\":53,\\\"protocol\\\":\\\"UDP\\\"},{\\\"name\\\":\\\"dns-tcp\\\",\\\"port\\\":53,\\\"protocol\\\":\\\"TCP\\\"},{\\\"name\\\":\\\"metrics\\\",\\\"port\\\":9153,\\\"protocol\\\":\\\"TCP\\\"}],\\\"selector\\\":{\\\"k8s-app\\\":\\\"kube-dns\\\"}}}\",\"objectset.rio.cattle.io/id\":\"\",\"objectset.rio.cattle.io/owner-gvk\":\"k3s.cattle.io/v1, Kind=Addon\",\"objectset.rio.cattle.io/owner-name\":\"coredns\",\"objectset.rio.cattle.io/owner-namespace\":\"kube-system\",\"prometheus.io/port\":\"9153\",\"prometheus.io/scrape\":\"true\"},\"labels\":{\"k8s-app\":\"kube-dns\",\"kubernetes.io/cluster-service\":\"true\",\"kubernetes.io/name\":\"CoreDNS\",\"objectset.rio.cattle.io/hash\":\"bce283298811743a0386ab510f2f67ef74240c57\"},\"name\":\"kube-dns\",\"namespace\":\"kube-system\"},\"spec\":{\"clusterIP\":\"10.247.0.10\",\"ports\":[{\"name\":\"dns\",\"port\":53,\"protocol\":\"UDP\"},{\"name\":\"dns-tcp\",\"port\":53,\"protocol\":\"TCP\"},{\"name\":\"metrics\",\"port\":9153,\"protocol\":\"TCP\"}],\"selector\":{\"k8s-app\":\"kube-dns\"}}}, {\"apiVersion\":\"v1\",\"kind\":\"Service\",\"metadata\":{\"annotations\":{\"kubectl.kubernetes.io/last-applied-configuration\":\"{\\\"apiVersion\\\":\\\"v1\\\",\\\"kind\\\":\\\"Service\\\",\\\"metadata\\\":{\\\"annotations\\\":{\\\"prometheus.io/port\\\":\\\"9153\\\",\\\"prometheus.io/scrape\\\":\\\"true\\\"},\\\"labels\\\":{\\\"k8s-app\\\":\\\"kube-dns\\\",\\\"kubernetes.io/cluster-service\\\":\\\"true\\\",\\\"kubernetes.io/name\\\":\\\"CoreDNS\\\"},\\\"name\\\":\\\"kube-dns\\\",\\\"namespace\\\":\\\"kube-system\\\"},\\\"spec\\\":{\\\"ports\\\":[{\\\"name\\\":\\\"dns\\\",\\\"port\\\":53,\\\"protocol\\\":\\\"UDP\\\",\\\"targetPort\\\":1053},{\\\"name\\\":\\\"dns-tcp\\\",\\\"port\\\":53,\\\"protocol\\\":\\\"TCP\\\",\\\"targetPort\\\":1053},{\\\"name\\\":\\\"metrics\\\",\\\"port\\\":9153,\\\"protocol\\\":\\\"TCP\\\"}],\\\"selector\\\":{\\\"k8s-app\\\":\\\"kube-dns\\\"}}}\\n\",\"objectset.rio.cattle.io/applied\":\"{\\\"apiVersion\\\":\\\"v1\\\",\\\"kind\\\":\\\"Service\\\",\\\"metadata\\\":{\\\"annotations\\\":{\\\"objectset.rio.cattle.io/id\\\":\\\"\\\",\\\"objectset.rio.cattle.io/owner-gvk\\\":\\\"k3s.cattle.io/v1, Kind=Addon\\\",\\\"objectset.rio.cattle.io/owner-name\\\":\\\"coredns\\\",\\\"objectset.rio.cattle.io/owner-namespace\\\":\\\"kube-system\\\",\\\"prometheus.io/port\\\":\\\"9153\\\",\\\"prometheus.io/scrape\\\":\\\"true\\\"},\\\"labels\\\":{\\\"k8s-app\\\":\\\"kube-dns\\\",\\\"kubernetes.io/cluster-service\\\":\\\"true\\\",\\\"kubernetes.io/name\\\":\\\"CoreDNS\\\",\\\"objectset.rio.cattle.io/hash\\\":\\\"bce283298811743a0386ab510f2f67ef74240c57\\\"},\\\"name\\\":\\\"kube-dns\\\",\\\"namespace\\\":\\\"kube-system\\\"},\\\"spec\\\":{\\\"clusterIP\\\":\\\"10.247.0.10\\\",\\\"ports\\\":[{\\\"name\\\":\\\"dns\\\",\\\"port\\\":53,\\\"protocol\\\":\\\"UDP\\\"},{\\\"name\\\":\\\"dns-tcp\\\",\\\"port\\\":53,\\\"protocol\\\":\\\"TCP\\\"},{\\\"name\\\":\\\"metrics\\\",\\\"port\\\":9153,\\\"protocol\\\":\\\"TCP\\\"}],\\\"selector\\\":{\\\"k8s-app\\\":\\\"kube-dns\\\"}}}\",\"objectset.rio.cattle.io/id\":\"\",\"objectset.rio.cattle.io/owner-gvk\":\"k3s.cattle.io/v1, Kind=Addon\",\"objectset.rio.cattle.io/owner-name\":\"coredns\",\"objectset.rio.cattle.io/owner-namespace\":\"kube-system\",\"prometheus.io/port\":\"9153\",\"prometheus.io/scrape\":\"true\"},\"creationTimestamp\":\"2022-02-04T12:52:09Z\",\"labels\":{\"k8s-app\":\"kube-dns\",\"kubernetes.io/cluster-service\":\"true\",\"kubernetes.io/name\":\"CoreDNS\",\"objectset.rio.cattle.io/hash\":\"bce283298811743a0386ab510f2f67ef74240c57\"},\"managedFields\":[{\"apiVersion\":\"v1\",\"fieldsType\":\"FieldsV1\",\"fieldsV1\":{\"f:metadata\":{\"f:annotations\":{\".\":{},\"f:kubectl.kubernetes.io/last-applied-configuration\":{},\"f:objectset.rio.cattle.io/applied\":{},\"f:objectset.rio.cattle.io/id\":{},\"f:objectset.rio.cattle.io/owner-gvk\":{},\"f:objectset.rio.cattle.io/owner-name\":{},\"f:objectset.rio.cattle.io/owner-namespace\":{},\"f:prometheus.io/port\":{},\"f:prometheus.io/scrape\":{}},\"f:labels\":{\".\":{},\"f:k8s-app\":{},\"f:kubernetes.io/cluster-service\":{},\"f:kubernetes.io/name\":{},\"f:objectset.rio.cattle.io/hash\":{}}},\"f:spec\":{\"f:clusterIP\":{},\"f:ports\":{\".\":{},\"k:{\\\"port\\\":53,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:name\":{},\"f:port\":{},\"f:protocol\":{},\"f:targetPort\":{}},\"k:{\\\"port\\\":53,\\\"protocol\\\":\\\"UDP\\\"}\":{\".\":{},\"f:name\":{},\"f:port\":{},\"f:protocol\":{},\"f:targetPort\":{}},\"k:{\\\"port\\\":9153,\\\"protocol\\\":\\\"TCP\\\"}\":{\".\":{},\"f:name\":{},\"f:port\":{},\"f:protocol\":{},\"f:targetPort\":{}}},\"f:selector\":{\".\":{},\"f:k8s-app\":{}},\"f:sessionAffinity\":{},\"f:type\":{}}},\"manager\":\"vcluster\",\"operation\":\"Update\",\"time\":\"2022-02-04T12:52:09Z\"}],\"name\":\"kube-dns\",\"namespace\":\"kube-system\",\"resourceVersion\":\"348\",\"uid\":\"c6c8de62-015d-4a29-9590-f2ab2ea9e2e3\"},\"spec\":{\"clusterIP\":\"10.247.141.211\",\"clusterIPs\":[\"10.247.141.211\"],\"ports\":[{\"name\":\"dns\",\"port\":53,\"protocol\":\"UDP\",\"targetPort\":1053},{\"name\":\"dns-tcp\",\"port\":53,\"protocol\":\"TCP\",\"targetPort\":53},{\"name\":\"metrics\",\"port\":9153,\"protocol\":\"TCP\",\"targetPort\":9153}],\"selector\":{\"k8s-app\":\"kube-dns\"},\"sessionAffinity\":\"None\",\"type\":\"ClusterIP\"},\"status\":{\"loadBalancer\":{}}}]"
time="2022-02-04T13:03:01.531510146Z" level=debug msg="DesiredSet - Updated /v1, Kind=Service kube-system/kube-dns for  kube-system/coredns -- application/strategic-merge-patch+json {\"spec\":{\"clusterIP\":\"10.247.0.10\"}}"
time="2022-02-04T13:03:01.537905133Z" level=debug msg="DesiredSet - No change(2) rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding /kube-apiserver-kubelet-admin for  kube-system/rolebindings"
time="2022-02-04T13:03:01.538176568Z" level=debug msg="DesiredSet - No change(2) rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding /system:k3s-controller for  kube-system/rolebindings"
time="2022-02-04T13:03:01.541717993Z" level=debug msg="DesiredSet - No change(2) rbac.authorization.k8s.io/v1, Kind=ClusterRole /system:k3s-controller for  kube-system/rolebindings"
time="2022-02-04T13:03:01.544321679Z" level=error msg="Failed to process config: failed to process /data/server/manifests/coredns.yaml: failed to update kube-system/kube-dns /v1, Kind=Service for  kube-system/coredns: Service \"kube-dns\" is invalid: spec.clusterIPs[0]: Invalid value: []string{\"10.247.0.10\"}: may not change once set"
W0204 13:03:04.419449      19 lease.go:233] Resetting endpoints for master service "kubernetes" to [172.16.1.123]
FabianKramm commented 2 years ago

@mixitgit thanks for creating this issue! how did you install vcluster, is there anything special in your vcluster setup? Does it work with the v0.6.0-beta.0 version?

mixitgit commented 2 years ago

It appeared that the issue was caused because we have migrated to 0.5.3 version of vcluster from 0.4.5, but haven't turned off coredns creation in k3s.