Open n-guitar opened 2 years ago
Describe the bug A clear and concise description of what the bug is.
To Reproduce
$ k get pod -n kube-system NAME READY STATUS RESTARTS AGE calico-kube-controllers-685b65ddf9-7tvmv 1/1 Running 1 16h calico-node-4nf8n 1/1 Running 1 16h calico-node-gg6dv 1/1 Running 1 15h coredns-78fcd69978-79s2b 1/1 Running 1 16h coredns-78fcd69978-9nxwg 1/1 Running 1 16h etcd-master-node 1/1 Running 1 16h kube-apiserver-master-node 1/1 Running 1 16h kube-controller-manager-master-node 1/1 Running 1 16h kube-proxy-4r6lj 1/1 Running 1 15h kube-proxy-ft9xn 1/1 Running 1 16h kube-scheduler-master-node 1/1 Running 1 14h metrics-server-599cd48df7-n4qhn 0/1 Running 0 41m $ k get deploy -n kube-system NAME READY UP-TO-DATE AVAILABLE AGE calico-kube-controllers 1/1 1 1 16h coredns 2/2 2 2 16h metrics-server 0/1 1 0 16h
describe
$ k describe pod -n kube-system metrics-server-599cd48df7-n4qhn Name: metrics-server-599cd48df7-n4qhn Namespace: kube-system Priority: 2000000000 Priority Class Name: system-cluster-critical Node: worker-node01/192.168.200.11 Start Time: Wed, 15 Jun 2022 02:08:07 +0000 Labels: k8s-app=metrics-server pod-template-hash=599cd48df7 Annotations: cni.projectcalico.org/containerID: e4107da65cabe1676da1cbc981ccc19a9407d450eb5e93f4e803a057345d7261 cni.projectcalico.org/podIP: 10.244.87.193/32 cni.projectcalico.org/podIPs: 10.244.87.193/32 Status: Running IP: 10.244.87.193 IPs: IP: 10.244.87.193 Controlled By: ReplicaSet/metrics-server-599cd48df7 Containers: metrics-server: Container ID: cri-o://d20ccd24163e718cf45cc34ee3514255c5cbc3e7ab02516c74459d6bfce155d0 Image: k8s.gcr.io/metrics-server/metrics-server:v0.6.1 Image ID: k8s.gcr.io/metrics-server/metrics-server@sha256:5ddc6458eb95f5c70bd13fdab90cbd7d6ad1066e5b528ad1dcb28b76c5fb2f00 Port: 4443/TCP Host Port: 0/TCP Args: --cert-dir=/tmp --secure-port=4443 --kubelet-insecure-tls --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-status-port State: Running Started: Wed, 15 Jun 2022 02:08:08 +0000 Ready: False Restart Count: 0 Liveness: http-get https://:https/livez delay=0s timeout=1s period=10s #success=1 #failure=3 Readiness: http-get https://:https/readyz delay=0s timeout=1s period=10s #success=1 #failure=3 Environment: <none> Mounts: /tmp from tmp-dir (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-wcf82 (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: tmp-dir: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: <unset> kube-api-access-wcf82: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: <nil> DownwardAPI: true QoS Class: BestEffort Node-Selectors: kubernetes.io/os=linux Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedScheduling 42m default-scheduler 0/2 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate, 1 node(s) had taint {node.kubernetes.io/unreachable: }, that the pod didn't tolerate. Warning FailedScheduling 41m default-scheduler 0/2 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate, 1 node(s) had taint {node.kubernetes.io/unreachable: }, that the pod didn't tolerate. Normal Scheduled 32m default-scheduler Successfully assigned kube-system/metrics-server-599cd48df7-n4qhn to worker-node01 Normal Pulled 32m kubelet Container image "k8s.gcr.io/metrics-server/metrics-server:v0.6.1" already present on machine Normal Created 32m kubelet Created container metrics-server Normal Started 32m kubelet Started container metrics-server Warning Unhealthy 32m kubelet Readiness probe failed: Get "https://10.244.87.193:4443/readyz": dial tcp 10.244.87.193:4443: connect: connection refused Warning Unhealthy 2m8s (x206 over 32m) kubelet Readiness probe failed: HTTP probe failed with statuscode: 500
log
$ k logs -n kube-system metrics-server-599cd48df7-n4qhn I0615 02:08:09.769826 1 serving.go:342] Generated self-signed cert (/tmp/apiserver.crt, /tmp/apiserver.key) I0615 02:08:10.761258 1 secure_serving.go:266] Serving securely on [::]:4443 I0615 02:08:10.761368 1 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController I0615 02:08:10.761398 1 shared_informer.go:240] Waiting for caches to sync for RequestHeaderAuthRequestController I0615 02:08:10.761461 1 dynamic_serving_content.go:131] "Starting controller" name="serving-cert::/tmp/apiserver.crt::/tmp/apiserver.key" I0615 02:08:10.775563 1 tlsconfig.go:240] "Starting DynamicServingCertificateController" W0615 02:08:10.780771 1 shared_informer.go:372] The sharedIndexInformer has started, run more than once is not allowed I0615 02:08:10.780912 1 configmap_cafile_content.go:201] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0615 02:08:10.780946 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0615 02:08:10.780982 1 configmap_cafile_content.go:201] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0615 02:08:10.781006 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0615 02:08:10.782265 1 server.go:187] "Failed probe" probe="metric-storage-ready" err="no metrics to serve" E0615 02:08:10.804171 1 scraper.go:140] "Failed to scrape node" err="request failed, status: \"403 Forbidden\"" node="worker-node01" E0615 02:08:10.807165 1 scraper.go:140] "Failed to scrape node" err="request failed, status: \"403 Forbidden\"" node="master-node" I0615 02:08:10.861696 1 shared_informer.go:247] Caches are synced for RequestHeaderAuthRequestController I0615 02:08:10.874058 1 server.go:187] "Failed probe" probe="metric-storage-ready" err="no metrics to serve" I0615 02:08:10.881560 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0615 02:08:10.881665 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0615 02:08:17.500979 1 server.go:187] "Failed probe" probe="metric-storage-ready" err="no metrics to serve" I0615 02:08:27.507726 1 server.go:187] "Failed probe" probe="metric-storage-ready" err="no metrics to serve" I0615 02:08:37.499359 1 server.go:187] "Failed probe" probe="metric-storage-ready" err="no metrics to serve" I0615 02:08:47.499155 1 server.go:187] "Failed probe" probe="metric-storage-ready" err="no metrics to serve" I0615 02:08:57.498266 1 server.go:187] "Failed probe" probe="metric-storage-ready" err="no metrics to serve" I0615 02:09:07.500739 1 server.go:187] "Failed probe" probe="metric-storage-ready" err="no metrics to serve" E0615 02:09:10.775717 1 scraper.go:140] "Failed to scrape node" err="request failed, status: \"403 Forbidden\"" node="master-node" E0615 02:09:10.776257 1 scraper.go:140] "Failed to scrape node" err="request failed, status: \"403 Forbidden\"" node="worker-node01"
sw vars (please complete the following information):
$ kubectl get nodes -o custom-columns=Name:metadata.name,Status:status.conditions[4].type,kubeletVersion:status.nodeInfo.kubeletVersion,Architecture:status.nodeInfo.architecture,RuntimeVersion:status.nodeInfo.containerRuntimeVersion,kernelVersion:status.nodeInfo.kernelVersion,osImage:status.nodeInfo.osImage,CPU:status.capacity.cpu,MEMORY:status.capacity.memory Name Status kubeletVersion Architecture RuntimeVersion kernelVersion osImage CPU MEMORY master-node Ready v1.22.10 amd64 cri-o://1.22.5 5.13.0-22-generic Ubuntu 21.10 2 3971068Ki worker-node01 Ready v1.22.10 amd64 cri-o://1.22.5 5.13.0-22-generic Ubuntu 21.10 1 2025456Ki
Additional context Add any other context about the problem here.
Macbook pro
sw_vers ProductName: macOS ProductVersion: 11.6.4 BuildVersion: 20G417
vagrant
Vagrant 2.2.10
https://github.com/kubernetes-sigs/metrics-server/blob/master/README.md#requirements Requirementsが十分か確認
Describe the bug A clear and concise description of what the bug is.
To Reproduce
describe
log
sw vars (please complete the following information):
Additional context Add any other context about the problem here.
Macbook pro
vagrant