search

kubernetes-sigs / kind

Kubernetes IN Docker - local clusters for testing Kubernetes
https://kind.sigs.k8s.io/
Apache License 2.0
13.47k stars 1.56k forks source link

Snapshotting a running kind node and reusing it #2328

Closed nitishm closed 2 years ago

nitishm commented 3 years ago

What is the best way to snapshot a running kind node? I have tried the following -

  1. Create the cluster
kind create cluster --name orkestra
  1. Have it download some docker images when pods are spun up.
docker commit orkestra-control-plane nmalhotra/kind:v1.20.2
docker push  nmalhotra/kind:v1.20.2
  1. Delete the old cluster and recreate it with the snapshot image 
    kind delete cluster --name orkestra
kind create cluster --name orkestra --image nmalhotra/kind:v1.20.2

    , but this fails during the controlplane creation, with the following error log

    Creating cluster "orkestra-2" ...
✓ Ensuring node image (nmalhotra/kind:v1.20.2-new) đŸ–ŧ
✓ Preparing nodes đŸ“Ļ
✓ Writing configuration 📜
✗ Starting control-plane 🕹ī¸
ERROR: failed to create cluster: failed to init node with kubeadm: command "docker exec --privileged orkestra-2-control-plane kubeadm init --skip-phases=preflight --config=/kind/kubeadm.conf --skip-token-print --v=6" failed with error: exit status 1
Command Output: I0624 18:37:52.265787     234 initconfiguration.go:201] loading configuration from "/kind/kubeadm.conf"
[config] WARNING: Ignored YAML document with GroupVersionKind kubeadm.k8s.io/v1beta2, Kind=JoinConfiguration
I0624 18:37:52.292068     234 certs.go:474] validating certificate period for CA certificate
I0624 18:37:52.294173     234 certs.go:474] validating certificate period for front-proxy CA certificate
[init] Using Kubernetes version: v1.20.2
[certs] Using certificateDir folder "/etc/kubernetes/pki"
I0624 18:37:52.295881     234 certs.go:474] validating certificate period for ca certificate
[certs] Using existing ca certificate authority
I0624 18:37:52.297376     234 certs.go:474] validating certificate period for apiserver certificate
[certs] Using existing apiserver certificate and key on disk
I0624 18:37:52.299574     234 certs.go:474] validating certificate period for apiserver-kubelet-client certificate
[certs] Using existing apiserver-kubelet-client certificate and key on disk
I0624 18:37:52.300677     234 certs.go:474] validating certificate period for front-proxy-ca certificate
[certs] Using existing front-proxy-ca certificate authority
I0624 18:37:52.301590     234 certs.go:474] validating certificate period for front-proxy-client certificate
[certs] Using existing front-proxy-client certificate and key on disk
I0624 18:37:52.302488     234 certs.go:474] validating certificate period for etcd/ca certificate
[certs] Using existing etcd/ca certificate authority
I0624 18:37:52.303688     234 certs.go:474] validating certificate period for etcd/server certificate
[certs] Using existing etcd/server certificate and key on disk
I0624 18:37:52.304627     234 certs.go:474] validating certificate period for etcd/peer certificate
[certs] Using existing etcd/peer certificate and key on disk
I0624 18:37:52.305865     234 certs.go:474] validating certificate period for etcd/healthcheck-client certificate
[certs] Using existing etcd/healthcheck-client certificate and key on disk
I0624 18:37:52.306626     234 certs.go:474] validating certificate period for apiserver-etcd-client certificate
I0624 18:37:52.307102     234 certs.go:76] creating new public/private key files for signing service account users
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0624 18:37:52.307310     234 kubeconfig.go:101] creating kubeconfig file for admin.conf
I0624 18:37:52.389193     234 loader.go:379] Config loaded from file:  /etc/kubernetes/admin.conf
W0624 18:37:52.389293     234 kubeconfig.go:246] a kubeconfig file "/etc/kubernetes/admin.conf" exists already but has an unexpected API Server URL: expected: https://orkestra-2-control-plane:6443, got: https://orkestra-control-plane:6443
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/admin.conf"
I0624 18:37:52.389447     234 kubeconfig.go:101] creating kubeconfig file for kubelet.conf
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/kubelet.conf"
I0624 18:37:52.611390     234 loader.go:379] Config loaded from file:  /etc/kubernetes/kubelet.conf
W0624 18:37:52.611443     234 kubeconfig.go:246] a kubeconfig file "/etc/kubernetes/kubelet.conf" exists already but has an unexpected API Server URL: expected: https://orkestra-2-control-plane:6443, got: https://orkestra-control-plane:6443
I0624 18:37:52.611469     234 kubeconfig.go:101] creating kubeconfig file for controller-manager.conf
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/controller-manager.conf"
I0624 18:37:52.797677     234 loader.go:379] Config loaded from file:  /etc/kubernetes/controller-manager.conf
W0624 18:37:52.797725     234 kubeconfig.go:246] a kubeconfig file "/etc/kubernetes/controller-manager.conf" exists already but has an unexpected API Server URL: expected: https://172.18.0.4:6443, got: https://172.18.0.2:6443
I0624 18:37:52.797745     234 kubeconfig.go:101] creating kubeconfig file for scheduler.conf
I0624 18:37:53.054522     234 loader.go:379] Config loaded from file:  /etc/kubernetes/scheduler.conf
W0624 18:37:53.054576     234 kubeconfig.go:246] a kubeconfig file "/etc/kubernetes/scheduler.conf" exists already but has an unexpected API Server URL: expected: https://172.18.0.4:6443, got: https://172.18.0.2:6443
I0624 18:37:53.054605     234 kubelet.go:63] Stopping the kubelet
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/scheduler.conf"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
I0624 18:37:53.183272     234 manifests.go:96] [control-plane] getting StaticPodSpecs
I0624 18:37:53.184366     234 manifests.go:109] [control-plane] adding volume "ca-certs" for component "kube-apiserver"
I0624 18:37:53.184418     234 manifests.go:109] [control-plane] adding volume "etc-ca-certificates" for component "kube-apiserver"
I0624 18:37:53.184434     234 manifests.go:109] [control-plane] adding volume "k8s-certs" for component "kube-apiserver"
I0624 18:37:53.184452     234 manifests.go:109] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-apiserver"
I0624 18:37:53.184469     234 manifests.go:109] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-apiserver"
I0624 18:37:53.190928     234 manifests.go:126] [control-plane] wrote static Pod manifest for component "kube-apiserver" to "/etc/kubernetes/manifests/kube-apiserver.yaml"
I0624 18:37:53.190988     234 manifests.go:96] [control-plane] getting StaticPodSpecs
[control-plane] Creating static Pod manifest for "kube-controller-manager"
I0624 18:37:53.191350     234 manifests.go:109] [control-plane] adding volume "ca-certs" for component "kube-controller-manager"
I0624 18:37:53.191401     234 manifests.go:109] [control-plane] adding volume "etc-ca-certificates" for component "kube-controller-manager"
I0624 18:37:53.191448     234 manifests.go:109] [control-plane] adding volume "flexvolume-dir" for component "kube-controller-manager"
I0624 18:37:53.191590     234 manifests.go:109] [control-plane] adding volume "k8s-certs" for component "kube-controller-manager"
I0624 18:37:53.191767     234 manifests.go:109] [control-plane] adding volume "kubeconfig" for component "kube-controller-manager"
I0624 18:37:53.191790     234 manifests.go:109] [control-plane] adding volume "usr-local-share-ca-certificates" for component "kube-controller-manager"
I0624 18:37:53.191942     234 manifests.go:109] [control-plane] adding volume "usr-share-ca-certificates" for component "kube-controller-manager"
I0624 18:37:53.193002     234 manifests.go:126] [control-plane] wrote static Pod manifest for component "kube-controller-manager" to "/etc/kubernetes/manifests/kube-controller-manager.yaml"
I0624 18:37:53.193063     234 manifests.go:96] [control-plane] getting StaticPodSpecs
[control-plane] Creating static Pod manifest for "kube-scheduler"
I0624 18:37:53.193417     234 manifests.go:109] [control-plane] adding volume "kubeconfig" for component "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0624 18:37:53.194241     234 manifests.go:126] [control-plane] wrote static Pod manifest for component "kube-scheduler" to "/etc/kubernetes/manifests/kube-scheduler.yaml"
I0624 18:37:53.195283     234 local.go:74] [etcd] wrote Static Pod manifest for a local etcd member to "/etc/kubernetes/manifests/etcd.yaml"
I0624 18:37:53.195341     234 waitcontrolplane.go:87] [wait-control-plane] Waiting for the API server to be healthy
I0624 18:37:53.195919     234 loader.go:379] Config loaded from file:  /etc/kubernetes/admin.conf
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0624 18:37:53.212087     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/healthz?timeout=10s 200 OK in 14 milliseconds
[apiclient] All control plane components are healthy after 0.015518 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
I0624 18:37:53.212486     234 uploadconfig.go:108] [upload-config] Uploading the kubeadm ClusterConfiguration to a ConfigMap
I0624 18:37:53.224944     234 round_trippers.go:445] POST https://orkestra-control-plane:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 409 Conflict in 10 milliseconds
I0624 18:37:53.229857     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config?timeout=10s 200 OK in 4 milliseconds
I0624 18:37:53.236851     234 round_trippers.go:445] PUT https://orkestra-control-plane:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config?timeout=10s 200 OK in 5 milliseconds
I0624 18:37:53.246599     234 round_trippers.go:445] POST https://orkestra-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 409 Conflict in 8 milliseconds
I0624 18:37:53.251381     234 round_trippers.go:445] PUT https://orkestra-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles/kubeadm:nodes-kubeadm-config?timeout=10s 200 OK in 4 milliseconds
I0624 18:37:53.262534     234 round_trippers.go:445] POST https://orkestra-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 409 Conflict in 10 milliseconds
I0624 18:37:53.267726     234 round_trippers.go:445] PUT https://orkestra-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings/kubeadm:nodes-kubeadm-config?timeout=10s 200 OK in 4 milliseconds
[kubelet] Creating a ConfigMap "kubelet-config-1.20" in namespace kube-system with the configuration for the kubelets in the cluster
I0624 18:37:53.268672     234 uploadconfig.go:122] [upload-config] Uploading the kubelet component config to a ConfigMap
I0624 18:37:53.278303     234 round_trippers.go:445] POST https://orkestra-control-plane:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 409 Conflict in 8 milliseconds
I0624 18:37:53.283844     234 round_trippers.go:445] PUT https://orkestra-control-plane:6443/api/v1/namespaces/kube-system/configmaps/kubelet-config-1.20?timeout=10s 200 OK in 4 milliseconds
I0624 18:37:53.292271     234 round_trippers.go:445] POST https://orkestra-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 409 Conflict in 7 milliseconds
I0624 18:37:53.298727     234 round_trippers.go:445] PUT https://orkestra-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles/kubeadm:kubelet-config-1.20?timeout=10s 200 OK in 5 milliseconds
I0624 18:37:53.309281     234 round_trippers.go:445] POST https://orkestra-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 409 Conflict in 9 milliseconds
I0624 18:37:53.315625     234 round_trippers.go:445] PUT https://orkestra-control-plane:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings/kubeadm:kubelet-config-1.20?timeout=10s 200 OK in 5 milliseconds
I0624 18:37:53.315822     234 uploadconfig.go:127] [upload-config] Preserving the CRISocket information for the control-plane node
I0624 18:37:53.315843     234 patchnode.go:30] [patchnode] Uploading the CRI Socket information "unix:///run/containerd/containerd.sock" to the Node API object "orkestra-2-control-plane" as an annotation
I0624 18:37:53.820075     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:37:54.320348     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:37:54.821158     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:37:55.322211     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:37:55.821936     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:37:56.320032     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:37:56.822001     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:37:57.320300     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:37:57.820779     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:37:58.320748     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:37:58.820850     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:37:59.323439     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:37:59.820789     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:00.320273     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:00.821071     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:01.321810     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:38:01.823385     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 6 milliseconds
I0624 18:38:02.321455     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:02.823099     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:38:03.320882     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:03.819452     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:04.321867     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:04.823819     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 6 milliseconds
I0624 18:38:05.322666     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:38:05.824254     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:38:06.284264     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:06.784288     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:07.284337     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:07.784325     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:08.284228     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:08.784240     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:09.286372     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:09.784130     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:10.284062     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:10.785113     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:11.283801     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:11.786213     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:12.283782     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:12.784388     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:13.284425     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:13.784080     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:14.284043     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:14.784460     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:15.284871     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:15.784885     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:16.285588     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:16.786269     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:17.285321     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:17.786185     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:38:18.284144     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:18.785183     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:19.287119     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:38:19.785239     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:20.285496     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:20.785082     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:21.284685     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:21.783648     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 2 milliseconds
I0624 18:38:22.284106     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:22.785396     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:23.284165     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:23.786038     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:38:24.284879     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:24.786719     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:25.285609     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:25.784374     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:26.284222     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:26.786119     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:27.285195     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:27.784941     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:28.284240     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:28.784403     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:29.284155     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:29.784422     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:30.283657     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:30.784700     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:31.285576     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:31.783997     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:32.284688     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:32.784845     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0624 18:38:33.284392     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:33.784753     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:34.285379     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:34.786500     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:38:35.286105     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:38:35.785029     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:36.248276     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:36.751976     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 6 milliseconds
I0624 18:38:37.248142     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:37.749375     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0624 18:38:38.249034     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:38.748485     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:39.247379     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:39.750847     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:38:40.248525     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:40.748971     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:41.248227     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:41.749095     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:42.248215     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:42.749714     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:43.249559     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:43.748531     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:44.248777     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:44.747245     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 2 milliseconds
I0624 18:38:45.248008     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:45.750504     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 6 milliseconds
I0624 18:38:46.266067     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 21 milliseconds
I0624 18:38:46.748917     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:47.298225     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 53 milliseconds
I0624 18:38:47.750972     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 6 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0624 18:38:48.313824     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 69 milliseconds
I0624 18:38:48.750359     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:38:49.249311     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:49.751612     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 6 milliseconds
I0624 18:38:50.252863     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 8 milliseconds
I0624 18:38:50.750121     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:38:51.487129     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 242 milliseconds
I0624 18:38:51.755340     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 9 milliseconds
I0624 18:38:52.251494     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 6 milliseconds
I0624 18:38:52.960671     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 215 milliseconds
I0624 18:38:53.248933     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:53.748096     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:54.247870     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:38:54.750315     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:38:55.249218     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:55.751788     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 7 milliseconds
I0624 18:38:56.248954     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:38:56.751033     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 6 milliseconds
I0624 18:38:57.250709     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:38:57.751365     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 6 milliseconds
I0624 18:38:58.252187     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 7 milliseconds
I0624 18:38:58.749884     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:38:59.251659     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 6 milliseconds
I0624 18:38:59.754723     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 9 milliseconds
I0624 18:39:00.251071     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 6 milliseconds
I0624 18:39:00.749913     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:01.250913     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 6 milliseconds
I0624 18:39:01.791212     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 45 milliseconds
I0624 18:39:02.250107     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:02.750538     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:03.251048     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 6 milliseconds
I0624 18:39:03.754491     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 9 milliseconds
I0624 18:39:04.249329     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:04.750601     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:05.252031     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 7 milliseconds
I0624 18:39:05.750814     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 6 milliseconds
I0624 18:39:06.214015     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:06.713434     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:07.212752     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:07.716073     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 7 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0624 18:39:08.215398     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 6 milliseconds
I0624 18:39:08.714908     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 6 milliseconds
I0624 18:39:09.213934     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:09.713800     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:10.212269     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:10.713303     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:11.212331     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:11.712155     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:12.212508     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:12.712352     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:13.212437     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:13.712909     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:14.212023     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:14.712861     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:15.213946     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:15.713909     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:16.215876     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 6 milliseconds
I0624 18:39:16.712838     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:17.213154     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:17.713361     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:18.212026     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:18.713479     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:19.215045     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:19.714285     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:20.212180     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:20.713736     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:21.212708     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:21.715391     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:22.212972     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:22.712781     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:23.212779     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:23.712698     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:24.212756     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:24.712255     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:25.214723     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:25.714839     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:26.214445     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:26.712372     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:27.212559     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:27.713092     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:28.212435     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:28.712881     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:29.213143     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:29.712648     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:30.212749     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:30.712686     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:31.215806     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 6 milliseconds
I0624 18:39:31.713713     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:32.212886     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:32.713303     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:33.213905     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:33.714525     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:34.216600     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 7 milliseconds
I0624 18:39:34.712255     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:35.212205     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:35.711841     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:36.177307     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:36.677039     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:37.178167     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:37.678408     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:38.176247     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:38.676174     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:39.176674     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:39.676525     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:40.176614     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:40.675935     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:41.174957     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 2 milliseconds
I0624 18:39:41.677445     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:42.176773     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:42.678134     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:43.176111     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:43.675778     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:44.176411     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:44.679337     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:45.178094     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:45.675816     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:46.176649     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:46.676565     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:47.178363     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:47.677210     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
I0624 18:39:48.176881     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:48.675711     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:49.177465     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:49.678905     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:50.177190     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:50.678399     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 5 milliseconds
I0624 18:39:51.176756     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:51.676022     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:52.177409     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:52.677272     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 4 milliseconds
I0624 18:39:53.175707     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
I0624 18:39:53.179454     234 round_trippers.go:445] GET https://orkestra-control-plane:6443/api/v1/nodes/orkestra-2-control-plane?timeout=10s 404 Not Found in 3 milliseconds
timed out waiting for the condition
Error writing Crisocket information for the control-plane node
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/init.runUploadKubeletConfig
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/init/uploadconfig.go:129
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow/runner.go:234
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow/runner.go:421
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow/runner.go:207
k8s.io/kubernetes/cmd/kubeadm/app/cmd.newCmdInit.func1
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/init.go:151
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).execute
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:850
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).ExecuteC
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:958
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).Execute
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:895
k8s.io/kubernetes/cmd/kubeadm/app.Run
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/kubeadm.go:50
main.main
_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/kubeadm.go:25
runtime.main
/usr/local/go/src/runtime/proc.go:204
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1374
error execution phase upload-config/kubelet
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow/runner.go:235
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow/runner.go:421
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow/runner.go:207
k8s.io/kubernetes/cmd/kubeadm/app/cmd.newCmdInit.func1
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/init.go:151
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).execute
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:850
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).ExecuteC
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:958
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).Execute
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:895
k8s.io/kubernetes/cmd/kubeadm/app.Run
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/kubeadm.go:50
main.main
_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/kubeadm.go:25
runtime.main
/usr/local/go/src/runtime/proc.go:204
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1374
BenTheElder commented 3 years ago

What is the best way to snapshot a running kind node?

I'm not aware of one that actually works. @mauilion had some experiments with persisting one via etcd but I never got up to speed on that

aojea commented 3 years ago

I think that at minimum you have to restore the image with the same ip address it had when you snapshotted that ... there is an ip flag in docker to assign ips to the containers