rancher / rke2

https://docs.rke2.io/
Apache License 2.0
1.52k stars 264 forks source link

Restore Snapshot is failing with Panic exception #1084

Closed adeepsri closed 3 years ago

adeepsri commented 3 years ago

Environmental Info: RKE2 Version: rke2 version: v1.20.7+rke2r2 (ea6514775f816b0b4972f832de7d98eba8dff55c)

Node(s) CPU architecture, OS, and Version: [root@server0 ~]# uname -a Linux server0 4.18.0-193.47.1.el8_2.x86_64 #1 SMP Thu Mar 4 03:03:32 EST 2021 x86_64 x86_64 x86_64 GNU/Linux

Cluster Configuration: Single Node Cluster

Describe the bug: Following Backup & Restore steps provided at https://docs.rke2.io/backup_restore/ is failing to restore snapshot with exception and none of the resources got created.

Steps To Reproduce:

Expected behavior: Snapshot restore should have etcd restored

Actual behavior: ETCD restore throw exception of observed panic exception, and once etcd restore completes, checking kubectl command, none of the resources got restored.

Additional context / logs:

W0531 11:35:27.004674  236858 pod_container_deletor.go:79] Container "6680d04a2929b55bbddbcef4e4aec959f97f0c167738a5f9a03c64409b1d2984" not found in pod's containers
W0531 11:35:27.004745  236858 pod_container_deletor.go:79] Container "ac256db2fd8193de21cdfb3d5ea7ef881876a7f20c590a9816d50cc2d494e7eb" not found in pod's containers
W0531 11:35:27.004864  236858 pod_container_deletor.go:79] Container "5c41f3a7c446e46ed904398c49abf4b412a105ecb6ce3856db4a7ac1beb4c57e" not found in pod's containers
W0531 11:35:27.004891  236858 pod_container_deletor.go:79] Container "b0e456e1626c305d909135080b576f7b65daecbf608c3ae1f47265be6068797d" not found in pod's containers
E0531 11:35:27.067038  236858 csi_plugin.go:148] kubernetes.io/csi: registrationHandler.RegisterPlugin failed to unregister plugin due to previous error: kubernetes.io/csi: Error uninstalling CSI driver: error uninstalling CSI driver from CSINode object error getting CSI client
I0531 11:35:27.071902  236858 reconciler.go:157] Reconciler: start to sync state
E0531 11:35:27.074222  236858 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
goroutine 190 [running]:
k8s.io/apimachinery/pkg/util/runtime.logPanic(0x4151120, 0x6e5d3a0)
        /go/src/kubernetes/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:74 +0x95
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
        /go/src/kubernetes/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:48 +0x89
panic(0x4151120, 0x6e5d3a0)
        /usr/local/go/src/runtime/panic.go:969 +0x1b9
k8s.io/kubernetes/pkg/kubelet/volumemanager/reconciler.(*reconciler).updateDevicePath(0xc0004e0600, 0xc001659c20)
        /go/src/kubernetes/pkg/kubelet/volumemanager/reconciler/reconciler.go:601 +0x3e
k8s.io/kubernetes/pkg/kubelet/volumemanager/reconciler.(*reconciler).updateStates(0xc0004e0600, 0xc001659c20, 0xc0010d6de0, 0x5e)
        /go/src/kubernetes/pkg/kubelet/volumemanager/reconciler/reconciler.go:632 +0x50
k8s.io/kubernetes/pkg/kubelet/volumemanager/reconciler.(*reconciler).syncStates(0xc0004e0600)
        /go/src/kubernetes/pkg/kubelet/volumemanager/reconciler/reconciler.go:436 +0x397
k8s.io/kubernetes/pkg/kubelet/volumemanager/reconciler.(*reconciler).sync(0xc0004e0600)
        /go/src/kubernetes/pkg/kubelet/volumemanager/reconciler/reconciler.go:348 +0x4e
k8s.io/kubernetes/pkg/kubelet/volumemanager/reconciler.(*reconciler).reconciliationLoopFunc.func1()
        /go/src/kubernetes/pkg/kubelet/volumemanager/reconciler/reconciler.go:158 +0x117
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc0007a0cb0)
        /go/src/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc0007a0cb0, 0x507fa80, 0xc000c6d0b0, 0x42dcb01, 0xc00010e0c0)
        /go/src/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156 +0xad
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0007a0cb0, 0x5f5e100, 0x0, 0x1, 0xc00010e0c0)
        /go/src/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.Until(...)
        /go/src/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90
k8s.io/kubernetes/pkg/kubelet/volumemanager/reconciler.(*reconciler).Run(0xc0004e0600, 0xc00010e0c0)
        /go/src/kubernetes/pkg/kubelet/volumemanager/reconciler/reconciler.go:146 +0x5f
created by k8s.io/kubernetes/pkg/kubelet/volumemanager.(*volumeManager).Run
        /go/src/kubernetes/pkg/kubelet/volumemanager/volume_manager.go:272 +0x1ae
W0531 11:35:27.089437  236858 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {/var/lib/kubelet/plugins_registry/driver.longhorn.io-reg.sock  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial unix /var/lib/kubelet/plugins_registry/driver.longhorn.io-reg.sock: operation was canceled". Reconnecting...
E0531 11:35:27.089451  236858 goroutinemap.go:150] Operation for "/var/lib/kubelet/plugins_registry/driver.longhorn.io-reg.sock" failed. No retries permitted until 2021-05-31 11:35:27.589409509 +0000 UTC m=+6.611643825 (durationBeforeRetry 500ms). Error: "RegisterPlugin error -- plugin registration failed with err: error updating Node object with CSI driver node info: error updating node: timed out waiting for the condition; caused by: error getting kube client: rpc error: code = Unavailable desc = transport is closing"
INFO[2021-05-31T11:35:27.089606422Z] Finish piping stdout of container "41e474f3f2995d5eff262daf7a83ea16086d8b58ab04c806f6e54abecf0d3125"
INFO[2021-05-31T11:35:27.089652424Z] Finish piping stderr of container "41e474f3f2995d5eff262daf7a83ea16086d8b58ab04c806f6e54abecf0d3125"
INFO[2021-05-31T11:35:27.112087529Z] TaskExit event &TaskExit{ContainerID:41e474f3f2995d5eff262daf7a83ea16086d8b58ab04c806f6e54abecf0d3125,ID:41e474f3f2995d5eff262daf7a83ea16086d8b58ab04c806f6e54abecf0d3125,Pid:204187,ExitStatus:1,ExitedAt:2021-05-31 11:35:27.11178921 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:27.189918902Z] shim disconnected                             id=41e474f3f2995d5eff262daf7a83ea16086d8b58ab04c806f6e54abecf0d3125
ERRO[2021-05-31T11:35:27.190009307Z] copy shim log                                 error="read /proc/self/fd/50: file already closed"
brandond commented 3 years ago

Please post the entire log output from the restore command, not just the failure portion. It's important to be able to see what it does beforehand.

adeepsri commented 3 years ago

@brandond Please find entire logs of etcd restore

[root@server0 ~]# ~/UiPath_Installer/install-uipath.sh -i ~/UiPath_Installer/input_restore.json -o ~/UiPath_Installer/output.json -r
jq-1.5-12.el8.x86_64
package helm is not installed
/root/UiPath_Installer/Infra_Installer/disaster_recovery/longhorn_helper.sh: line 3: SCRIPT_DIR: readonly variable
[INFO] [2021-05-31T11:35:15+0000]: Using input parameters file: /root/UiPath_Installer/input_restore.json
[INFO] [2021-05-31T11:35:15+0000]: Using output parameters file: /root/UiPath_Installer/restore_output.json
[INFO] [2021-05-31T11:35:15+0000]: Using defaults parameters file: /root/UiPath_Installer/Infra_Installer/defaults.json
[INFO] [2021-05-31T11:35:15+0000]: Check if cluster is ready to use...
server0   Ready    control-plane,etcd,master   20m   v1.20.7+rke2r2
Kubernetes control plane is running at https://127.0.0.1:6443
CoreDNS is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/rke2-coredns-rke2-coredns:udp-53/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
[INFO] [2021-05-31T11:35:16+0000]: Longhorn RestApi Endpoint: http://10.43.212.69:9500
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3249    0  3249    0     0  3172k      0 --:--:-- --:--:-- --:--:-- 3172k
[INFO] [2021-05-31T11:35:16+0000]: Longhorn Backend is Accessible
[INFO] [2021-05-31T11:35:16+0000]: Start Restoring Cluster for hostname server0
mkdir: cannot create directory ‘/mnt/restore’: File exists
ls: cannot access '/mnt/restore/cluster0/server0/longhorn_disk/*longhorn-disk.cfg*': No such file or directory
[WARN] [2021-05-31T11:35:16+0000]: Longhorn default disk Files for ETCD Snapshot Backup for hostname server0 not available in backup store.
[INFO] [2021-05-31T11:35:16+0000]: NFS Server connection & mountpath successfully validated.
[INFO] [2021-05-31T11:35:16+0000]: Restoring ETCD for Hostname cluster0 started...
[INFO] [2021-05-31T11:35:16+0000]: Disable ETCD backup in the setup if any
[INFO] [2021-05-31T11:35:18+0000]: Disable Longhorn volume Backup
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   794  100   710  100    84   115k  14000 --:--:-- --:--:-- --:--:--  129k
[INFO] [2021-05-31T11:35:18+0000]: Updated longhorn default settings default-longhorn-static-storage-class with longhorn successfully.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   424  100   372  100    52  93000  13000 --:--:-- --:--:-- --:--:--  138k
[INFO] [2021-05-31T11:35:18+0000]: Updated longhorn default settings backup-target with  successfully.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   551  100   490  100    61   119k  15250 --:--:-- --:--:-- --:--:--  134k
[INFO] [2021-05-31T11:35:18+0000]: Updated longhorn default settings default-replica-count with 1 successfully.
[INFO] [2021-05-31T11:35:18+0000]: Stopping RKE2 Server...
[INFO] [2021-05-31T11:35:20+0000]: Stopped RKE2 Server successfully.
[INFO] [2021-05-31T11:35:20+0000]: Restore ETCD with snapshot etcd-snapshot-server0-1622460001 started...
236836
WARN[0000] not running in CIS mode
INFO[0000] Running temporary kubelet /var/lib/rancher/rke2/bin/kubelet --fail-swap-on=false --container-runtime=remote --containerd=/run/k3s/containerd/containerd.sock --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --pod-manifest-path=/var/lib/rancher/rke2/agent/pod-manifests
INFO[0000] Running temporary containerd /var/lib/rancher/rke2/bin/containerd -c /var/lib/rancher/rke2/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/rke2/agent/containerd
INFO[2021-05-31T11:35:20.994179013Z] starting containerd                           revision=2d0c2063337a633b3d637c4e0344e0fe4981ee06 version=v1.4.4-k3s2
INFO[2021-05-31T11:35:21.022945814Z] loading plugin "io.containerd.content.v1.content"...  type=io.containerd.content.v1
INFO[2021-05-31T11:35:21.023036719Z] loading plugin "io.containerd.snapshotter.v1.aufs"...  type=io.containerd.snapshotter.v1
INFO[2021-05-31T11:35:21.025736688Z] skip loading plugin "io.containerd.snapshotter.v1.aufs"...  error="aufs is not supported (modprobe aufs failed: exit status 1 \"modprobe: FATAL: Module aufs not found in directory /lib/modules/4.18.0-193.47.1.el8_2.x86_64\\n\"): skip plugin" type=io.containerd.snapshotter.v1
INFO[2021-05-31T11:35:21.025778891Z] loading plugin "io.containerd.snapshotter.v1.btrfs"...  type=io.containerd.snapshotter.v1
INFO[2021-05-31T11:35:21.028692973Z] skip loading plugin "io.containerd.snapshotter.v1.btrfs"...  error="path /var/lib/rancher/rke2/agent/containerd/io.containerd.snapshotter.v1.btrfs (xfs) must be a btrfs filesystem to be used with the btrfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
INFO[2021-05-31T11:35:21.028743077Z] loading plugin "io.containerd.snapshotter.v1.devmapper"...  type=io.containerd.snapshotter.v1
WARN[2021-05-31T11:35:21.028774479Z] failed to load plugin io.containerd.snapshotter.v1.devmapper  error="devmapper not configured"
INFO[2021-05-31T11:35:21.028790380Z] loading plugin "io.containerd.snapshotter.v1.native"...  type=io.containerd.snapshotter.v1
INFO[2021-05-31T11:35:21.028836182Z] loading plugin "io.containerd.snapshotter.v1.overlayfs"...  type=io.containerd.snapshotter.v1
INFO[2021-05-31T11:35:21.029027294Z] loading plugin "io.containerd.snapshotter.v1.zfs"...  type=io.containerd.snapshotter.v1
INFO[2021-05-31T11:35:21.030850109Z] skip loading plugin "io.containerd.snapshotter.v1.zfs"...  error="path /var/lib/rancher/rke2/agent/containerd/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
INFO[2021-05-31T11:35:21.030879910Z] loading plugin "io.containerd.metadata.v1.bolt"...  type=io.containerd.metadata.v1
WARN[2021-05-31T11:35:21.030909912Z] could not use snapshotter devmapper in metadata plugin  error="devmapper not configured"
INFO[2021-05-31T11:35:21.030925513Z] metadata content store policy set             policy=shared
INFO[2021-05-31T11:35:21.031054821Z] loading plugin "io.containerd.differ.v1.walking"...  type=io.containerd.differ.v1
INFO[2021-05-31T11:35:21.031080823Z] loading plugin "io.containerd.gc.v1.scheduler"...  type=io.containerd.gc.v1
INFO[2021-05-31T11:35:21.031121025Z] loading plugin "io.containerd.service.v1.introspection-service"...  type=io.containerd.service.v1
INFO[2021-05-31T11:35:21.031158628Z] loading plugin "io.containerd.service.v1.containers-service"...  type=io.containerd.service.v1
INFO[2021-05-31T11:35:21.031177029Z] loading plugin "io.containerd.service.v1.content-service"...  type=io.containerd.service.v1
INFO[2021-05-31T11:35:21.031193530Z] loading plugin "io.containerd.service.v1.diff-service"...  type=io.containerd.service.v1
INFO[2021-05-31T11:35:21.031211931Z] loading plugin "io.containerd.service.v1.images-service"...  type=io.containerd.service.v1
INFO[2021-05-31T11:35:21.031230032Z] loading plugin "io.containerd.service.v1.leases-service"...  type=io.containerd.service.v1
INFO[2021-05-31T11:35:21.031247433Z] loading plugin "io.containerd.service.v1.namespaces-service"...  type=io.containerd.service.v1
INFO[2021-05-31T11:35:21.031265735Z] loading plugin "io.containerd.service.v1.snapshots-service"...  type=io.containerd.service.v1
INFO[2021-05-31T11:35:21.031279435Z] loading plugin "io.containerd.runtime.v1.linux"...  type=io.containerd.runtime.v1
INFO[2021-05-31T11:35:21.031341339Z] loading plugin "io.containerd.runtime.v2.task"...  type=io.containerd.runtime.v2
Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --containerd has been deprecated, This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.
Flag --pod-manifest-path has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
I0531 11:35:21.065613  236858 server.go:416] Version: v1.20.7+rke2r2
W0531 11:35:21.065948  236858 server.go:558] standalone mode, no API client
W0531 11:35:21.066145  236858 server.go:614] failed to get the kubelet's cgroup: cpu and memory cgroup hierarchy not unified.  cpu: /user.slice, memory: /user.slice/user-1000.slice/session-315.scope.  Kubelet system container metrics may be missing.
INFO[2021-05-31T11:35:21.088599924Z] loading plugin "io.containerd.monitor.v1.cgroups"...  type=io.containerd.monitor.v1
INFO[2021-05-31T11:35:21.089126957Z] loading plugin "io.containerd.service.v1.tasks-service"...  type=io.containerd.service.v1
INFO[2021-05-31T11:35:21.089234664Z] loading plugin "io.containerd.internal.v1.restart"...  type=io.containerd.internal.v1
INFO[2021-05-31T11:35:21.089288567Z] loading plugin "io.containerd.grpc.v1.containers"...  type=io.containerd.grpc.v1
INFO[2021-05-31T11:35:21.089307368Z] loading plugin "io.containerd.grpc.v1.content"...  type=io.containerd.grpc.v1
INFO[2021-05-31T11:35:21.089324969Z] loading plugin "io.containerd.grpc.v1.diff"...  type=io.containerd.grpc.v1
INFO[2021-05-31T11:35:21.089342070Z] loading plugin "io.containerd.grpc.v1.events"...  type=io.containerd.grpc.v1
INFO[2021-05-31T11:35:21.089358671Z] loading plugin "io.containerd.grpc.v1.healthcheck"...  type=io.containerd.grpc.v1
INFO[2021-05-31T11:35:21.089377773Z] loading plugin "io.containerd.grpc.v1.images"...  type=io.containerd.grpc.v1
INFO[2021-05-31T11:35:21.089394774Z] loading plugin "io.containerd.grpc.v1.leases"...  type=io.containerd.grpc.v1
INFO[2021-05-31T11:35:21.089412375Z] loading plugin "io.containerd.grpc.v1.namespaces"...  type=io.containerd.grpc.v1
INFO[2021-05-31T11:35:21.089429276Z] loading plugin "io.containerd.internal.v1.opt"...  type=io.containerd.internal.v1
INFO[2021-05-31T11:35:21.089490580Z] loading plugin "io.containerd.grpc.v1.snapshots"...  type=io.containerd.grpc.v1
INFO[2021-05-31T11:35:21.089510681Z] loading plugin "io.containerd.grpc.v1.tasks"...  type=io.containerd.grpc.v1
INFO[2021-05-31T11:35:21.089527882Z] loading plugin "io.containerd.grpc.v1.version"...  type=io.containerd.grpc.v1
INFO[2021-05-31T11:35:21.089543083Z] loading plugin "io.containerd.grpc.v1.cri"...  type=io.containerd.grpc.v1
INFO[2021-05-31T11:35:21.089730695Z] Start cri plugin with config {PluginConfig:{ContainerdConfig:{Snapshotter:overlayfs DefaultRuntimeName:runc DefaultRuntime:{Type: Engine: PodAnnotations:[] ContainerAnnotations:[] Root: Options:<nil> PrivilegedWithoutHostDevices:false BaseRuntimeSpec:} UntrustedWorkloadRuntime:{Type: Engine: PodAnnotations:[] ContainerAnnotations:[] Root: Options:<nil> PrivilegedWithoutHostDevices:false BaseRuntimeSpec:} Runtimes:map[runc:{Type:io.containerd.runc.v2 Engine: PodAnnotations:[] ContainerAnnotations:[] Root: Options:<nil> PrivilegedWithoutHostDevices:false BaseRuntimeSpec:}] NoPivot:false DisableSnapshotAnnotations:true DiscardUnpackedLayers:false} CniConfig:{NetworkPluginBinDir:/opt/cni/bin NetworkPluginConfDir:/etc/cni/net.d NetworkPluginMaxConfNum:1 NetworkPluginConfTemplate:} Registry:{Mirrors:map[docker-registry.docker-registry.svc.cluster.local:5000:{Endpoints:[http://localhost:30071] Rewrites:map[]} docker.io:{Endpoints:[https://registry-1.docker.io] Rewrites:map[]} sfbrdevhelmweacr.azurecr.io:{Endpoints:[https://sfbrdevhelmweacr.azurecr.io] Rewrites:map[]}] Configs:map[sfbrdevhelmweacr.azurecr.io:{Auth:0xc00011ff80 TLS:<nil>}] Auths:map[] Headers:map[]} ImageDecryption:{KeyModel:} DisableTCPService:true StreamServerAddress:127.0.0.1 StreamServerPort:10010 StreamIdleTimeout:4h0m0s EnableSelinux:false SelinuxCategoryRange:1024 SandboxImage:index.docker.io/rancher/pause:3.2 StatsCollectPeriod:10 SystemdCgroup:false EnableTLSStreaming:false X509KeyPairStreaming:{TLSCertFile: TLSKeyFile:} MaxContainerLogLineSize:16384 DisableCgroup:false DisableApparmor:false RestrictOOMScoreAdj:false MaxConcurrentDownloads:3 DisableProcMount:false UnsetSeccompProfile: TolerateMissingHugetlbController:true DisableHugetlbController:true IgnoreImageDefinedVolumes:false} ContainerdRootDir:/var/lib/rancher/rke2/agent/containerd ContainerdEndpoint:/run/k3s/containerd/containerd.sock RootDir:/var/lib/rancher/rke2/agent/containerd/io.containerd.grpc.v1.cri StateDir:/run/k3s/containerd/io.containerd.grpc.v1.cri}
INFO[2021-05-31T11:35:21.089772897Z] Connect containerd service
INFO[2021-05-31T11:35:21.089832401Z] Get image filesystem path "/var/lib/rancher/rke2/agent/containerd/io.containerd.snapshotter.v1.overlayfs"
INFO[2021-05-31T11:35:21.090396036Z] loading plugin "io.containerd.grpc.v1.introspection"...  type=io.containerd.grpc.v1
INFO[2021-05-31T11:35:21.090463441Z] Start subscribing containerd event
INFO[2021-05-31T11:35:21.090527045Z] Start recovering state
INFO[2021-05-31T11:35:21.090863166Z] serving...                                    address=/run/k3s/containerd/containerd.sock.ttrpc
INFO[2021-05-31T11:35:21.091530307Z] serving...                                    address=/run/k3s/containerd/containerd.sock
INFO[2021-05-31T11:35:21.091582411Z] containerd successfully booted in 0.102048s
INFO[2021-05-31T11:35:21.442504980Z] Start event monitor
INFO[2021-05-31T11:35:21.442548183Z] Start snapshots syncer
INFO[2021-05-31T11:35:21.442557184Z] Start cni network conf syncer
INFO[2021-05-31T11:35:21.442563484Z] Start streaming server
W0531 11:35:26.107359  236858 server.go:473] No api server defined - no events will be sent to API server.
I0531 11:35:26.107389  236858 server.go:645] --cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /
I0531 11:35:26.108872  236858 container_manager_linux.go:274] container manager verified user specified cgroup-root exists: []
I0531 11:35:26.108907  236858 container_manager_linux.go:279] Creating Container Manager object based on Node Config: {RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:remote CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: ReservedSystemCPUs: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:imagefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.15} GracePeriod:0s MinReclaim:<nil>} {Signal:memory.available Operator:LessThan Value:{Quantity:100Mi Percentage:0} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.1} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.inodesFree Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>}]} QOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalTopologyManagerScope:container ExperimentalCPUManagerReconcilePeriod:10s ExperimentalPodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms ExperimentalTopologyManagerPolicy:none}
I0531 11:35:26.108991  236858 topology_manager.go:120] [topologymanager] Creating topology manager with none policy per container scope
I0531 11:35:26.109005  236858 container_manager_linux.go:310] [topologymanager] Initializing Topology Manager with none policy and container-level scope
I0531 11:35:26.109012  236858 container_manager_linux.go:315] Creating device plugin manager: true
I0531 11:35:26.109421  236858 remote_runtime.go:62] parsed scheme: ""
I0531 11:35:26.109440  236858 remote_runtime.go:62] scheme "" not registered, fallback to default scheme
I0531 11:35:26.109491  236858 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/run/k3s/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}
I0531 11:35:26.109510  236858 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0531 11:35:26.109590  236858 remote_image.go:50] parsed scheme: ""
I0531 11:35:26.109604  236858 remote_image.go:50] scheme "" not registered, fallback to default scheme
I0531 11:35:26.109625  236858 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/run/k3s/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}
I0531 11:35:26.109638  236858 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0531 11:35:26.109705  236858 kubelet.go:400] Kubelet is running in standalone mode, will skip API server sync
I0531 11:35:26.109723  236858 kubelet.go:262] Adding pod path: /var/lib/rancher/rke2/agent/pod-manifests
I0531 11:35:26.112927  236858 kuberuntime_manager.go:216] Container runtime containerd initialized, version: v1.4.4-k3s2, apiVersion: v1alpha2
E0531 11:35:26.264609  236858 aws_credentials.go:77] while getting AWS credentials NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors
W0531 11:35:26.264949  236858 volume_host.go:75] kubeClient is nil. Skip initialization of CSIDriverLister
W0531 11:35:26.265525  236858 csi_plugin.go:191] kubernetes.io/csi: kubeclient not set, assuming standalone kubelet
W0531 11:35:26.265556  236858 csi_plugin.go:265] Skipping CSINode initialization, kubelet running in standalone mode
I0531 11:35:26.265843  236858 server.go:1176] Started kubelet
W0531 11:35:26.265878  236858 kubelet.go:1396] No api server defined - no node status update will be sent.
I0531 11:35:26.265937  236858 server.go:148] Starting to listen on 0.0.0.0:10250
E0531 11:35:26.266850  236858 cri_stats_provider.go:376] Failed to get the info of the filesystem with mountpoint "/var/lib/rancher/rke2/agent/containerd/io.containerd.snapshotter.v1.overlayfs": unable to find data in memory cache.
E0531 11:35:26.266899  236858 kubelet.go:1291] Image garbage collection failed once. Stats initialization may not have completed yet: invalid capacity 0 on image filesystem
I0531 11:35:26.267173  236858 server.go:410] Adding debug handlers to kubelet server.
I0531 11:35:26.270782  236858 fs_resource_analyzer.go:64] Starting FS ResourceAnalyzer
I0531 11:35:26.270956  236858 volume_manager.go:271] Starting Kubelet Volume Manager
I0531 11:35:26.270985  236858 desired_state_of_world_populator.go:142] Desired state populator starts to run
I0531 11:35:26.271691  236858 client.go:86] parsed scheme: "unix"
I0531 11:35:26.271724  236858 client.go:86] scheme "unix" not registered, fallback to default scheme
I0531 11:35:26.271804  236858 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{unix:///run/k3s/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}
I0531 11:35:26.271820  236858 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0531 11:35:26.303943  236858 kubelet_network_linux.go:56] Initialized IPv4 iptables rules.
I0531 11:35:26.303990  236858 status_manager.go:154] Kubernetes client is nil, not starting status manager.
I0531 11:35:26.304006  236858 kubelet.go:1828] Starting kubelet main sync loop.
E0531 11:35:26.304156  236858 kubelet.go:1852] skipping pod synchronization - [container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]
E0531 11:35:26.404209  236858 kubelet.go:1852] skipping pod synchronization - container runtime status check may not have completed yet
E0531 11:35:26.604320  236858 kubelet.go:1852] skipping pod synchronization - container runtime status check may not have completed yet
I0531 11:35:26.743024  236858 cpu_manager.go:193] [cpumanager] starting with none policy
I0531 11:35:26.743071  236858 cpu_manager.go:194] [cpumanager] reconciling every 10s
I0531 11:35:26.743100  236858 state_mem.go:36] [cpumanager] initializing new in-memory state store
I0531 11:35:26.743342  236858 state_mem.go:88] [cpumanager] updated default cpuset: ""
I0531 11:35:26.743358  236858 state_mem.go:96] [cpumanager] updated cpuset assignments: "map[]"
I0531 11:35:26.743375  236858 policy_none.go:43] [cpumanager] none policy: Start
W0531 11:35:26.746709  236858 manager.go:594] Failed to retrieve checkpoint for "kubelet_internal_checkpoint": checkpoint is not found
E0531 11:35:26.747366  236858 container_manager_linux.go:533] failed to find cgroups of kubelet - cpu and memory cgroup hierarchy not unified.  cpu: /user.slice, memory: /user.slice/user-1000.slice/session-315.scope
I0531 11:35:26.747629  236858 plugin_manager.go:114] Starting Kubelet Plugin Manager
I0531 11:35:26.747828  236858 operation_generator.go:181] parsed scheme: ""
I0531 11:35:26.747845  236858 operation_generator.go:181] scheme "" not registered, fallback to default scheme
I0531 11:35:26.747864  236858 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/var/lib/kubelet/plugins_registry/driver.longhorn.io-reg.sock  <nil> 0 <nil>}] <nil> <nil>}
I0531 11:35:26.747881  236858 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0531 11:35:26.748624  236858 csi_plugin.go:101] kubernetes.io/csi: Trying to validate a new CSI Driver with name: driver.longhorn.io endpoint: /var/lib/kubelet/obsoleted-longhorn-plugins/driver.longhorn.io/csi.sock versions: 1.0.0
I0531 11:35:26.748901  236858 csi_plugin.go:114] kubernetes.io/csi: Register new plugin with name: driver.longhorn.io at endpoint: /var/lib/kubelet/obsoleted-longhorn-plugins/driver.longhorn.io/csi.sock
I0531 11:35:26.748963  236858 clientconn.go:106] parsed scheme: ""
I0531 11:35:26.748983  236858 clientconn.go:106] scheme "" not registered, fallback to default scheme
I0531 11:35:26.749179  236858 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/var/lib/kubelet/obsoleted-longhorn-plugins/driver.longhorn.io/csi.sock  <nil> 0 <nil>}] <nil> <nil>}
I0531 11:35:26.749206  236858 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0531 11:35:26.749259  236858 clientconn.go:897] blockingPicker: the picked transport is not ready, loop back to repick
W0531 11:35:27.004674  236858 pod_container_deletor.go:79] Container "6680d04a2929b55bbddbcef4e4aec959f97f0c167738a5f9a03c64409b1d2984" not found in pod's containers
W0531 11:35:27.004745  236858 pod_container_deletor.go:79] Container "ac256db2fd8193de21cdfb3d5ea7ef881876a7f20c590a9816d50cc2d494e7eb" not found in pod's containers
W0531 11:35:27.004864  236858 pod_container_deletor.go:79] Container "5c41f3a7c446e46ed904398c49abf4b412a105ecb6ce3856db4a7ac1beb4c57e" not found in pod's containers
W0531 11:35:27.004891  236858 pod_container_deletor.go:79] Container "b0e456e1626c305d909135080b576f7b65daecbf608c3ae1f47265be6068797d" not found in pod's containers
E0531 11:35:27.067038  236858 csi_plugin.go:148] kubernetes.io/csi: registrationHandler.RegisterPlugin failed to unregister plugin due to previous error: kubernetes.io/csi: Error uninstalling CSI driver: error uninstalling CSI driver from CSINode object error getting CSI client
I0531 11:35:27.071902  236858 reconciler.go:157] Reconciler: start to sync state
E0531 11:35:27.074222  236858 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
goroutine 190 [running]:
k8s.io/apimachinery/pkg/util/runtime.logPanic(0x4151120, 0x6e5d3a0)
        /go/src/kubernetes/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:74 +0x95
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
        /go/src/kubernetes/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:48 +0x89
panic(0x4151120, 0x6e5d3a0)
        /usr/local/go/src/runtime/panic.go:969 +0x1b9
k8s.io/kubernetes/pkg/kubelet/volumemanager/reconciler.(*reconciler).updateDevicePath(0xc0004e0600, 0xc001659c20)
        /go/src/kubernetes/pkg/kubelet/volumemanager/reconciler/reconciler.go:601 +0x3e
k8s.io/kubernetes/pkg/kubelet/volumemanager/reconciler.(*reconciler).updateStates(0xc0004e0600, 0xc001659c20, 0xc0010d6de0, 0x5e)
        /go/src/kubernetes/pkg/kubelet/volumemanager/reconciler/reconciler.go:632 +0x50
k8s.io/kubernetes/pkg/kubelet/volumemanager/reconciler.(*reconciler).syncStates(0xc0004e0600)
        /go/src/kubernetes/pkg/kubelet/volumemanager/reconciler/reconciler.go:436 +0x397
k8s.io/kubernetes/pkg/kubelet/volumemanager/reconciler.(*reconciler).sync(0xc0004e0600)
        /go/src/kubernetes/pkg/kubelet/volumemanager/reconciler/reconciler.go:348 +0x4e
k8s.io/kubernetes/pkg/kubelet/volumemanager/reconciler.(*reconciler).reconciliationLoopFunc.func1()
        /go/src/kubernetes/pkg/kubelet/volumemanager/reconciler/reconciler.go:158 +0x117
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc0007a0cb0)
        /go/src/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc0007a0cb0, 0x507fa80, 0xc000c6d0b0, 0x42dcb01, 0xc00010e0c0)
        /go/src/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156 +0xad
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0007a0cb0, 0x5f5e100, 0x0, 0x1, 0xc00010e0c0)
        /go/src/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.Until(...)
        /go/src/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90
k8s.io/kubernetes/pkg/kubelet/volumemanager/reconciler.(*reconciler).Run(0xc0004e0600, 0xc00010e0c0)
        /go/src/kubernetes/pkg/kubelet/volumemanager/reconciler/reconciler.go:146 +0x5f
created by k8s.io/kubernetes/pkg/kubelet/volumemanager.(*volumeManager).Run
        /go/src/kubernetes/pkg/kubelet/volumemanager/volume_manager.go:272 +0x1ae
W0531 11:35:27.089437  236858 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {/var/lib/kubelet/plugins_registry/driver.longhorn.io-reg.sock  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial unix /var/lib/kubelet/plugins_registry/driver.longhorn.io-reg.sock: operation was canceled". Reconnecting...
E0531 11:35:27.089451  236858 goroutinemap.go:150] Operation for "/var/lib/kubelet/plugins_registry/driver.longhorn.io-reg.sock" failed. No retries permitted until 2021-05-31 11:35:27.589409509 +0000 UTC m=+6.611643825 (durationBeforeRetry 500ms). Error: "RegisterPlugin error -- plugin registration failed with err: error updating Node object with CSI driver node info: error updating node: timed out waiting for the condition; caused by: error getting kube client: rpc error: code = Unavailable desc = transport is closing"
INFO[2021-05-31T11:35:27.089606422Z] Finish piping stdout of container "41e474f3f2995d5eff262daf7a83ea16086d8b58ab04c806f6e54abecf0d3125"
INFO[2021-05-31T11:35:27.089652424Z] Finish piping stderr of container "41e474f3f2995d5eff262daf7a83ea16086d8b58ab04c806f6e54abecf0d3125"
INFO[2021-05-31T11:35:27.112087529Z] TaskExit event &TaskExit{ContainerID:41e474f3f2995d5eff262daf7a83ea16086d8b58ab04c806f6e54abecf0d3125,ID:41e474f3f2995d5eff262daf7a83ea16086d8b58ab04c806f6e54abecf0d3125,Pid:204187,ExitStatus:1,ExitedAt:2021-05-31 11:35:27.11178921 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:27.189918902Z] shim disconnected                             id=41e474f3f2995d5eff262daf7a83ea16086d8b58ab04c806f6e54abecf0d3125
ERRO[2021-05-31T11:35:27.190009307Z] copy shim log                                 error="read /proc/self/fd/50: file already closed"
INFO[2021-05-31T11:35:28.315946598Z] StopContainer for "b339e15cd86e8095e17408d7bd37abfafc346f460feeeda471b01f84d79570fe" with timeout 30 (s)
INFO[2021-05-31T11:35:28.315992700Z] StopContainer for "a4907d95317ec33b9e18aefc2cd35948223a0af6fc5ac5df2671fa1b8ac4f40b" with timeout 30 (s)
INFO[2021-05-31T11:35:28.315946197Z] StopContainer for "168cf0a156e6deea9792a794248fc62bc60712ff820878712bc9d2468b27be0b" with timeout 30 (s)
INFO[2021-05-31T11:35:28.316002801Z] StopContainer for "cd037ddca1cf79d703a16c6046deaf4d639247b736e9a29b07688ce931e886dd" with timeout 30 (s)
INFO[2021-05-31T11:35:28.315968199Z] StopContainer for "b10aa220c3246c0b8317a22fb79d8d1676ac88dd135d6521880b0e0756544f0d" with timeout 30 (s)
INFO[2021-05-31T11:35:28.315974099Z] StopContainer for "7f99f0de5096ddac1c6658063e307ba13a680a76e96685a76f8370728a7f159f" with timeout 30 (s)
INFO[2021-05-31T11:35:28.316624240Z] Stop container "168cf0a156e6deea9792a794248fc62bc60712ff820878712bc9d2468b27be0b" with signal terminated
INFO[2021-05-31T11:35:28.316737147Z] StopContainer for "484b5f26c648ae1e8a14b6e321b85df898543dda324d589c9654845a25202858" with timeout 30 (s)
INFO[2021-05-31T11:35:28.316790650Z] Stop container "b339e15cd86e8095e17408d7bd37abfafc346f460feeeda471b01f84d79570fe" with signal terminated
INFO[2021-05-31T11:35:28.316848054Z] StopContainer for "4ad62ff9625721b50946986e0f546a40cf40cfc693582eceb1f1c3cb1f7e3084" with timeout 30 (s)
INFO[2021-05-31T11:35:28.316904057Z] StopContainer for "a95a719aa4b0d17316076cfe39f3fe7adebc8b99db1e5ab07f70b6df30f4618d" with timeout 30 (s)
INFO[2021-05-31T11:35:28.316977062Z] StopContainer for "ebcc58d0a3b98ba775f22ac16b396e58e625e98f0995ff601020a5116a31a3c1" with timeout 30 (s)
INFO[2021-05-31T11:35:28.317015164Z] StopContainer for "abd46405e7365b01a15eb71e3fd97595811086fa89d804ff89baa5925e5c09c0" with timeout 30 (s)
INFO[2021-05-31T11:35:28.317111070Z] Stop container "b10aa220c3246c0b8317a22fb79d8d1676ac88dd135d6521880b0e0756544f0d" with signal terminated
INFO[2021-05-31T11:35:28.316857655Z] StopContainer for "0b52af0a7848f8b2c7d022e3848d93b93921e3d2bf1a655aa6dfb3d26618587b" with timeout 30 (s)
INFO[2021-05-31T11:35:28.316808051Z] StopContainer for "4d7b1966552f191aeba7d827cf4fb02a2811525ec67105111ce3c43f34e97853" with timeout 30 (s)
INFO[2021-05-31T11:35:28.317125671Z] StopContainer for "a02772fcef07c90a33dd85eb05aab5a282370d8c89f9ac2c6c3daed985b12d49" with timeout 30 (s)
INFO[2021-05-31T11:35:28.317355686Z] Stop container "4ad62ff9625721b50946986e0f546a40cf40cfc693582eceb1f1c3cb1f7e3084" with signal terminated
INFO[2021-05-31T11:35:28.316903157Z] Stop container "a4907d95317ec33b9e18aefc2cd35948223a0af6fc5ac5df2671fa1b8ac4f40b" with signal terminated
INFO[2021-05-31T11:35:28.317463092Z] Stop container "ebcc58d0a3b98ba775f22ac16b396e58e625e98f0995ff601020a5116a31a3c1" with signal terminated
INFO[2021-05-31T11:35:28.317157873Z] StopContainer for "28ab98f2bc95b24782c5ecfe15e14eeab2d3da7d73d42a43a61ab39b9043fc5d" with timeout 2 (s)
INFO[2021-05-31T11:35:28.317859817Z] Stop container "abd46405e7365b01a15eb71e3fd97595811086fa89d804ff89baa5925e5c09c0" with signal terminated
INFO[2021-05-31T11:35:28.317869718Z] Stop container "484b5f26c648ae1e8a14b6e321b85df898543dda324d589c9654845a25202858" with signal terminated
INFO[2021-05-31T11:35:28.317964424Z] Stop container "4d7b1966552f191aeba7d827cf4fb02a2811525ec67105111ce3c43f34e97853" with signal terminated
INFO[2021-05-31T11:35:28.317991226Z] Stop container "a95a719aa4b0d17316076cfe39f3fe7adebc8b99db1e5ab07f70b6df30f4618d" with signal terminated
INFO[2021-05-31T11:35:28.317067768Z] StopContainer for "a45842958375fbb490dcbb1b02df255224280d3f6c7a77794256dd0ab043493a" with timeout 30 (s)
INFO[2021-05-31T11:35:28.316829853Z] StopContainer for "dfb4378a6838ba4057b39ce35ea286ecd7877d1ef2ebed56407656a9ec2df1d6" with timeout 30 (s)
INFO[2021-05-31T11:35:28.318092732Z] Stop container "a02772fcef07c90a33dd85eb05aab5a282370d8c89f9ac2c6c3daed985b12d49" with signal terminated
INFO[2021-05-31T11:35:28.316851854Z] Stop container "cd037ddca1cf79d703a16c6046deaf4d639247b736e9a29b07688ce931e886dd" with signal terminated
INFO[2021-05-31T11:35:28.316896557Z] StopContainer for "fa0fcd4feacc9f4f74f8c39e3f512cd029e47e8e86b6062e9e5f767b2beb2f65" with timeout 30 (s)
INFO[2021-05-31T11:35:28.316896257Z] StopContainer for "43d3cefdad766490e38623465108038771ee54b1053b921a83fea3da0d0943db" with timeout 30 (s)
INFO[2021-05-31T11:35:28.318820477Z] Stop container "a45842958375fbb490dcbb1b02df255224280d3f6c7a77794256dd0ab043493a" with signal terminated
INFO[2021-05-31T11:35:28.316920759Z] StopContainer for "55373d72ab04ce7cd951686d8e8152307aa0b340bb1f58b6d670f868fe7a12bc" with timeout 30 (s)
INFO[2021-05-31T11:35:28.317194276Z] StopContainer for "693a32131387a0d8303d6df80693d3de6a9019e9a66edd3d203187d88edc06f2" with timeout 30 (s)
INFO[2021-05-31T11:35:28.316986563Z] StopContainer for "e276d005b51326c3bc7a88995e40611a3df12be1bc7653ac8707df4341615085" with timeout 2 (s)
INFO[2021-05-31T11:35:28.316963961Z] StopContainer for "ff4f7aa2c9a53c3df3cf67ed2f5b76ea9f7dff200b97dfa898f92a3c1b763d3a" with timeout 30 (s)
INFO[2021-05-31T11:35:28.316952861Z] StopContainer for "7240611c0a2661f07cde3d3d0022e1115e85064397a7f4f8165e11952930c309" with timeout 30 (s)
INFO[2021-05-31T11:35:28.333463494Z] Stop container "43d3cefdad766490e38623465108038771ee54b1053b921a83fea3da0d0943db" with signal terminated
INFO[2021-05-31T11:35:28.316985063Z] StopContainer for "35b48bbfd52bbaab2073ae7f2cb49279b0ef94f6586ff81304aac891cb6b3c3d" with timeout 30 (s)
INFO[2021-05-31T11:35:28.317018365Z] StopContainer for "307f709c72d167bf70b186a526cb562fa4837c790be984f433ee132536b5822c" with timeout 30 (s)
INFO[2021-05-31T11:35:28.317083569Z] StopContainer for "5196b8846626eaad2bfa309478ff1f5ab5731ad78c141b1b77cdbea8a1e57eaa" with timeout 30 (s)
INFO[2021-05-31T11:35:28.332926161Z] Stop container "dfb4378a6838ba4057b39ce35ea286ecd7877d1ef2ebed56407656a9ec2df1d6" with signal terminated
INFO[2021-05-31T11:35:28.317113471Z] Stop container "7f99f0de5096ddac1c6658063e307ba13a680a76e96685a76f8370728a7f159f" with signal terminated
INFO[2021-05-31T11:35:28.317183075Z] StopContainer for "fcb78381d2248c96acfb5f1d4f0da188d3fd7db0dd2e1cafbf26f67959908459" with timeout 30 (s)
INFO[2021-05-31T11:35:28.332968363Z] Stop container "0b52af0a7848f8b2c7d022e3848d93b93921e3d2bf1a655aa6dfb3d26618587b" with signal terminated
INFO[2021-05-31T11:35:28.332988664Z] Stop container "28ab98f2bc95b24782c5ecfe15e14eeab2d3da7d73d42a43a61ab39b9043fc5d" with signal terminated
INFO[2021-05-31T11:35:28.333077170Z] Stop container "55373d72ab04ce7cd951686d8e8152307aa0b340bb1f58b6d670f868fe7a12bc" with signal terminated
INFO[2021-05-31T11:35:28.333978626Z] Stop container "fa0fcd4feacc9f4f74f8c39e3f512cd029e47e8e86b6062e9e5f767b2beb2f65" with signal terminated
INFO[2021-05-31T11:35:28.316924259Z] ExecSync for "bef614a8c136075240c887d23cc202965eb8c7b4218501b2459eab892021e302" with command [/bin/sh -c rm -f /csi//*] and timeout 0 (s)
INFO[2021-05-31T11:35:28.334982989Z] Stop container "ff4f7aa2c9a53c3df3cf67ed2f5b76ea9f7dff200b97dfa898f92a3c1b763d3a" with signal terminated
INFO[2021-05-31T11:35:28.335070295Z] Stop container "307f709c72d167bf70b186a526cb562fa4837c790be984f433ee132536b5822c" with signal terminated
INFO[2021-05-31T11:35:28.335150400Z] Stop container "7240611c0a2661f07cde3d3d0022e1115e85064397a7f4f8165e11952930c309" with signal terminated
INFO[2021-05-31T11:35:28.335175701Z] Stop container "e276d005b51326c3bc7a88995e40611a3df12be1bc7653ac8707df4341615085" with signal terminated
INFO[2021-05-31T11:35:28.335237505Z] Stop container "fcb78381d2248c96acfb5f1d4f0da188d3fd7db0dd2e1cafbf26f67959908459" with signal terminated
INFO[2021-05-31T11:35:28.335261907Z] Stop container "35b48bbfd52bbaab2073ae7f2cb49279b0ef94f6586ff81304aac891cb6b3c3d" with signal terminated
INFO[2021-05-31T11:35:28.335316510Z] Stop container "693a32131387a0d8303d6df80693d3de6a9019e9a66edd3d203187d88edc06f2" with signal quit
INFO[2021-05-31T11:35:28.335343712Z] Stop container "5196b8846626eaad2bfa309478ff1f5ab5731ad78c141b1b77cdbea8a1e57eaa" with signal terminated
INFO[2021-05-31T11:35:28.352781204Z] Finish piping stderr of container "7240611c0a2661f07cde3d3d0022e1115e85064397a7f4f8165e11952930c309"
INFO[2021-05-31T11:35:28.352857208Z] Finish piping stdout of container "7240611c0a2661f07cde3d3d0022e1115e85064397a7f4f8165e11952930c309"
INFO[2021-05-31T11:35:28.353528450Z] Finish piping stderr of container "168cf0a156e6deea9792a794248fc62bc60712ff820878712bc9d2468b27be0b"
INFO[2021-05-31T11:35:28.353599755Z] Finish piping stdout of container "168cf0a156e6deea9792a794248fc62bc60712ff820878712bc9d2468b27be0b"
INFO[2021-05-31T11:35:28.353706662Z] Finish piping stderr of container "4ad62ff9625721b50946986e0f546a40cf40cfc693582eceb1f1c3cb1f7e3084"
INFO[2021-05-31T11:35:28.353764265Z] Finish piping stdout of container "4ad62ff9625721b50946986e0f546a40cf40cfc693582eceb1f1c3cb1f7e3084"
INFO[2021-05-31T11:35:28.354826732Z] TaskExit event &TaskExit{ContainerID:168cf0a156e6deea9792a794248fc62bc60712ff820878712bc9d2468b27be0b,ID:168cf0a156e6deea9792a794248fc62bc60712ff820878712bc9d2468b27be0b,Pid:204088,ExitStatus:2,ExitedAt:2021-05-31 11:35:28.354217894 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:28.364107913Z] Finish piping stderr of container "abd46405e7365b01a15eb71e3fd97595811086fa89d804ff89baa5925e5c09c0"
INFO[2021-05-31T11:35:28.364195718Z] Finish piping stdout of container "abd46405e7365b01a15eb71e3fd97595811086fa89d804ff89baa5925e5c09c0"
INFO[2021-05-31T11:35:28.364251522Z] Finish piping stderr of container "a4907d95317ec33b9e18aefc2cd35948223a0af6fc5ac5df2671fa1b8ac4f40b"
INFO[2021-05-31T11:35:28.364296524Z] Finish piping stdout of container "a4907d95317ec33b9e18aefc2cd35948223a0af6fc5ac5df2671fa1b8ac4f40b"
INFO[2021-05-31T11:35:28.366014332Z] Finish piping stderr of container "484b5f26c648ae1e8a14b6e321b85df898543dda324d589c9654845a25202858"
INFO[2021-05-31T11:35:28.366154441Z] Finish piping stdout of container "484b5f26c648ae1e8a14b6e321b85df898543dda324d589c9654845a25202858"
INFO[2021-05-31T11:35:28.373356592Z] Finish piping stderr of container "ebcc58d0a3b98ba775f22ac16b396e58e625e98f0995ff601020a5116a31a3c1"
INFO[2021-05-31T11:35:28.373433797Z] Finish piping stdout of container "ebcc58d0a3b98ba775f22ac16b396e58e625e98f0995ff601020a5116a31a3c1"
INFO[2021-05-31T11:35:28.375452823Z] Finish piping stderr of container "cd037ddca1cf79d703a16c6046deaf4d639247b736e9a29b07688ce931e886dd"
INFO[2021-05-31T11:35:28.375517727Z] Finish piping stdout of container "cd037ddca1cf79d703a16c6046deaf4d639247b736e9a29b07688ce931e886dd"
INFO[2021-05-31T11:35:28.375606033Z] Finish piping stderr of container "a02772fcef07c90a33dd85eb05aab5a282370d8c89f9ac2c6c3daed985b12d49"
INFO[2021-05-31T11:35:28.375656036Z] Finish piping stdout of container "a02772fcef07c90a33dd85eb05aab5a282370d8c89f9ac2c6c3daed985b12d49"
INFO[2021-05-31T11:35:28.382752980Z] Finish piping stderr of container "fa0fcd4feacc9f4f74f8c39e3f512cd029e47e8e86b6062e9e5f767b2beb2f65"
INFO[2021-05-31T11:35:28.382825785Z] Finish piping stdout of container "fa0fcd4feacc9f4f74f8c39e3f512cd029e47e8e86b6062e9e5f767b2beb2f65"
INFO[2021-05-31T11:35:28.388326129Z] Finish piping stderr of container "b339e15cd86e8095e17408d7bd37abfafc346f460feeeda471b01f84d79570fe"
INFO[2021-05-31T11:35:28.388424635Z] Finish piping stdout of container "b339e15cd86e8095e17408d7bd37abfafc346f460feeeda471b01f84d79570fe"
INFO[2021-05-31T11:35:28.389712416Z] Finish piping stderr of container "55373d72ab04ce7cd951686d8e8152307aa0b340bb1f58b6d670f868fe7a12bc"
INFO[2021-05-31T11:35:28.389784120Z] Finish piping stdout of container "55373d72ab04ce7cd951686d8e8152307aa0b340bb1f58b6d670f868fe7a12bc"
INFO[2021-05-31T11:35:28.397638912Z] Finish piping stderr of container "28ab98f2bc95b24782c5ecfe15e14eeab2d3da7d73d42a43a61ab39b9043fc5d"
INFO[2021-05-31T11:35:28.398421161Z] Finish piping stderr of container "43d3cefdad766490e38623465108038771ee54b1053b921a83fea3da0d0943db"
INFO[2021-05-31T11:35:28.398464564Z] Finish piping stdout of container "43d3cefdad766490e38623465108038771ee54b1053b921a83fea3da0d0943db"
INFO[2021-05-31T11:35:28.401983884Z] Finish piping stdout of container "a95a719aa4b0d17316076cfe39f3fe7adebc8b99db1e5ab07f70b6df30f4618d"
INFO[2021-05-31T11:35:28.402074590Z] Finish piping stderr of container "a95a719aa4b0d17316076cfe39f3fe7adebc8b99db1e5ab07f70b6df30f4618d"
INFO[2021-05-31T11:35:28.405933431Z] Finish piping stdout of container "0b52af0a7848f8b2c7d022e3848d93b93921e3d2bf1a655aa6dfb3d26618587b"
INFO[2021-05-31T11:35:28.406031837Z] Finish piping stderr of container "0b52af0a7848f8b2c7d022e3848d93b93921e3d2bf1a655aa6dfb3d26618587b"
INFO[2021-05-31T11:35:28.427789900Z] Finish piping stderr of container "307f709c72d167bf70b186a526cb562fa4837c790be984f433ee132536b5822c"
INFO[2021-05-31T11:35:28.427846903Z] Finish piping stdout of container "307f709c72d167bf70b186a526cb562fa4837c790be984f433ee132536b5822c"
INFO[2021-05-31T11:35:28.428171423Z] Finish piping stdout of container "ff4f7aa2c9a53c3df3cf67ed2f5b76ea9f7dff200b97dfa898f92a3c1b763d3a"
INFO[2021-05-31T11:35:28.428208326Z] Finish piping stderr of container "ff4f7aa2c9a53c3df3cf67ed2f5b76ea9f7dff200b97dfa898f92a3c1b763d3a"
INFO[2021-05-31T11:35:28.428954372Z] Finish piping stderr of container "7f99f0de5096ddac1c6658063e307ba13a680a76e96685a76f8370728a7f159f"
INFO[2021-05-31T11:35:28.428976474Z] Finish piping stdout of container "7f99f0de5096ddac1c6658063e307ba13a680a76e96685a76f8370728a7f159f"
INFO[2021-05-31T11:35:28.432782212Z] Finish piping stderr of container "a45842958375fbb490dcbb1b02df255224280d3f6c7a77794256dd0ab043493a"
INFO[2021-05-31T11:35:28.432803913Z] Finish piping stdout of container "a45842958375fbb490dcbb1b02df255224280d3f6c7a77794256dd0ab043493a"
INFO[2021-05-31T11:35:28.436713458Z] Finish piping stderr of container "dfb4378a6838ba4057b39ce35ea286ecd7877d1ef2ebed56407656a9ec2df1d6"
INFO[2021-05-31T11:35:28.436782163Z] Finish piping stdout of container "dfb4378a6838ba4057b39ce35ea286ecd7877d1ef2ebed56407656a9ec2df1d6"
INFO[2021-05-31T11:35:28.436841466Z] Finish piping stdout of container "28ab98f2bc95b24782c5ecfe15e14eeab2d3da7d73d42a43a61ab39b9043fc5d"
INFO[2021-05-31T11:35:28.437066880Z] Finish piping stdout of container "5196b8846626eaad2bfa309478ff1f5ab5731ad78c141b1b77cdbea8a1e57eaa"
INFO[2021-05-31T11:35:28.437083681Z] Finish piping stderr of container "5196b8846626eaad2bfa309478ff1f5ab5731ad78c141b1b77cdbea8a1e57eaa"
INFO[2021-05-31T11:35:28.437475006Z] Finish piping stderr of container "35b48bbfd52bbaab2073ae7f2cb49279b0ef94f6586ff81304aac891cb6b3c3d"
INFO[2021-05-31T11:35:28.437550611Z] Finish piping stdout of container "35b48bbfd52bbaab2073ae7f2cb49279b0ef94f6586ff81304aac891cb6b3c3d"
INFO[2021-05-31T11:35:28.440834716Z] Finish piping stderr of container "fcb78381d2248c96acfb5f1d4f0da188d3fd7db0dd2e1cafbf26f67959908459"
INFO[2021-05-31T11:35:28.440860218Z] Finish piping stdout of container "fcb78381d2248c96acfb5f1d4f0da188d3fd7db0dd2e1cafbf26f67959908459"
INFO[2021-05-31T11:35:28.453750325Z] shim disconnected                             id=168cf0a156e6deea9792a794248fc62bc60712ff820878712bc9d2468b27be0b
ERRO[2021-05-31T11:35:28.453814129Z] copy shim log                                 error="read /proc/self/fd/26: file already closed"
INFO[2021-05-31T11:35:28.458391915Z] TaskExit event &TaskExit{ContainerID:4ad62ff9625721b50946986e0f546a40cf40cfc693582eceb1f1c3cb1f7e3084,ID:4ad62ff9625721b50946986e0f546a40cf40cfc693582eceb1f1c3cb1f7e3084,Pid:203889,ExitStatus:2,ExitedAt:2021-05-31 11:35:28.358283948 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:28.458454119Z] StopContainer for "168cf0a156e6deea9792a794248fc62bc60712ff820878712bc9d2468b27be0b" returns successfully
INFO[2021-05-31T11:35:28.459145863Z] StopPodSandbox for "db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97"
INFO[2021-05-31T11:35:28.459233268Z] Container to stop "168cf0a156e6deea9792a794248fc62bc60712ff820878712bc9d2468b27be0b" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:28.463712148Z] Finish piping "stderr" of container exec "50ebdbbf976119d9426c329d19a4b442b3a1eb8a3da8abaa4d80ad5928adf02e"
INFO[2021-05-31T11:35:28.463739950Z] Finish piping "stdout" of container exec "50ebdbbf976119d9426c329d19a4b442b3a1eb8a3da8abaa4d80ad5928adf02e"
INFO[2021-05-31T11:35:28.463910761Z] Exec process "50ebdbbf976119d9426c329d19a4b442b3a1eb8a3da8abaa4d80ad5928adf02e" exits with exit code 0 and error <nil>
INFO[2021-05-31T11:35:28.465354551Z] ExecSync for "bef614a8c136075240c887d23cc202965eb8c7b4218501b2459eab892021e302" returns with exit code 0
INFO[2021-05-31T11:35:28.465769577Z] StopContainer for "bef614a8c136075240c887d23cc202965eb8c7b4218501b2459eab892021e302" with timeout 30 (s)
INFO[2021-05-31T11:35:28.466112299Z] Stop container "bef614a8c136075240c887d23cc202965eb8c7b4218501b2459eab892021e302" with signal terminated
INFO[2021-05-31T11:35:28.467789404Z] Finish piping stdout of container "693a32131387a0d8303d6df80693d3de6a9019e9a66edd3d203187d88edc06f2"
INFO[2021-05-31T11:35:28.467790704Z] Finish piping stderr of container "693a32131387a0d8303d6df80693d3de6a9019e9a66edd3d203187d88edc06f2"
INFO[2021-05-31T11:35:28.503125216Z] Finish piping stderr of container "bef614a8c136075240c887d23cc202965eb8c7b4218501b2459eab892021e302"
INFO[2021-05-31T11:35:28.503142417Z] Finish piping stdout of container "bef614a8c136075240c887d23cc202965eb8c7b4218501b2459eab892021e302"
INFO[2021-05-31T11:35:28.514250112Z] shim disconnected                             id=4ad62ff9625721b50946986e0f546a40cf40cfc693582eceb1f1c3cb1f7e3084
ERRO[2021-05-31T11:35:28.514332118Z] copy shim log                                 error="read /proc/self/fd/62: file already closed"
INFO[2021-05-31T11:35:28.518617286Z] TaskExit event &TaskExit{ContainerID:dceb9a712dc8e63224ff37a893615edc49d7e0809f2663ffca5b51a1b8f6b7d9,ID:dceb9a712dc8e63224ff37a893615edc49d7e0809f2663ffca5b51a1b8f6b7d9,Pid:193413,ExitStatus:137,ExitedAt:2021-05-31 11:35:28.361312538 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:28.518676390Z] StopContainer for "4ad62ff9625721b50946986e0f546a40cf40cfc693582eceb1f1c3cb1f7e3084" returns successfully
INFO[2021-05-31T11:35:28.519120617Z] StopPodSandbox for "b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48"
INFO[2021-05-31T11:35:28.519179221Z] Container to stop "4ad62ff9625721b50946986e0f546a40cf40cfc693582eceb1f1c3cb1f7e3084" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:28.564643167Z] shim disconnected                             id=dceb9a712dc8e63224ff37a893615edc49d7e0809f2663ffca5b51a1b8f6b7d9
ERRO[2021-05-31T11:35:28.564741173Z] copy shim log                                 error="read /proc/self/fd/152: file already closed"
INFO[2021-05-31T11:35:28.564867281Z] TaskExit event &TaskExit{ContainerID:077488a0b3aadb9b64c2bbbf2bd74eb939d500ac9598ebafa08b93640df69686,ID:077488a0b3aadb9b64c2bbbf2bd74eb939d500ac9598ebafa08b93640df69686,Pid:193221,ExitStatus:137,ExitedAt:2021-05-31 11:35:28.363543477 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:28.632291703Z] shim disconnected                             id=077488a0b3aadb9b64c2bbbf2bd74eb939d500ac9598ebafa08b93640df69686
ERRO[2021-05-31T11:35:28.632389309Z] copy shim log                                 error="read /proc/self/fd/8: file already closed"
INFO[2021-05-31T11:35:28.632467714Z] TaskExit event &TaskExit{ContainerID:41c5bc1acf3f41342d5b140009a18a4435782b38e177a1153c42bceacfba1ed7,ID:41c5bc1acf3f41342d5b140009a18a4435782b38e177a1153c42bceacfba1ed7,Pid:193036,ExitStatus:137,ExitedAt:2021-05-31 11:35:28.357186379 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:28.680235904Z] shim disconnected                             id=41c5bc1acf3f41342d5b140009a18a4435782b38e177a1153c42bceacfba1ed7
ERRO[2021-05-31T11:35:28.680340711Z] copy shim log                                 error="read /proc/self/fd/47: file already closed"
INFO[2021-05-31T11:35:28.680495020Z] TaskExit event &TaskExit{ContainerID:abd46405e7365b01a15eb71e3fd97595811086fa89d804ff89baa5925e5c09c0,ID:abd46405e7365b01a15eb71e3fd97595811086fa89d804ff89baa5925e5c09c0,Pid:203869,ExitStatus:2,ExitedAt:2021-05-31 11:35:28.373981431 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:28.732473675Z] shim disconnected                             id=abd46405e7365b01a15eb71e3fd97595811086fa89d804ff89baa5925e5c09c0
ERRO[2021-05-31T11:35:28.732551579Z] copy shim log                                 error="read /proc/self/fd/113: file already closed"
INFO[2021-05-31T11:35:28.742130979Z] TaskExit event &TaskExit{ContainerID:cd037ddca1cf79d703a16c6046deaf4d639247b736e9a29b07688ce931e886dd,ID:cd037ddca1cf79d703a16c6046deaf4d639247b736e9a29b07688ce931e886dd,Pid:203930,ExitStatus:2,ExitedAt:2021-05-31 11:35:28.37461357 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:28.742201684Z] StopContainer for "abd46405e7365b01a15eb71e3fd97595811086fa89d804ff89baa5925e5c09c0" returns successfully
INFO[2021-05-31T11:35:28.742645711Z] StopPodSandbox for "407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749"
INFO[2021-05-31T11:35:28.742739817Z] Container to stop "abd46405e7365b01a15eb71e3fd97595811086fa89d804ff89baa5925e5c09c0" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:28.791658680Z] shim disconnected                             id=cd037ddca1cf79d703a16c6046deaf4d639247b736e9a29b07688ce931e886dd
ERRO[2021-05-31T11:35:28.791743085Z] copy shim log                                 error="read /proc/self/fd/143: file already closed"
INFO[2021-05-31T11:35:28.795732435Z] TaskExit event &TaskExit{ContainerID:a02772fcef07c90a33dd85eb05aab5a282370d8c89f9ac2c6c3daed985b12d49,ID:a02772fcef07c90a33dd85eb05aab5a282370d8c89f9ac2c6c3daed985b12d49,Pid:193473,ExitStatus:137,ExitedAt:2021-05-31 11:35:28.377149629 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:28.795793039Z] StopContainer for "cd037ddca1cf79d703a16c6046deaf4d639247b736e9a29b07688ce931e886dd" returns successfully
INFO[2021-05-31T11:35:28.796203064Z] StopPodSandbox for "6414e3c3a9d87f08fbb4b5fa1dc2f6bb2dbb0aba79218c139560cb06f0c4b88e"
INFO[2021-05-31T11:35:28.796280369Z] Container to stop "cd037ddca1cf79d703a16c6046deaf4d639247b736e9a29b07688ce931e886dd" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:28.856650749Z] shim disconnected                             id=a02772fcef07c90a33dd85eb05aab5a282370d8c89f9ac2c6c3daed985b12d49
ERRO[2021-05-31T11:35:28.856700352Z] copy shim log                                 error="read /proc/self/fd/98: file already closed"
INFO[2021-05-31T11:35:28.860680301Z] TaskExit event &TaskExit{ContainerID:ebcc58d0a3b98ba775f22ac16b396e58e625e98f0995ff601020a5116a31a3c1,ID:ebcc58d0a3b98ba775f22ac16b396e58e625e98f0995ff601020a5116a31a3c1,Pid:203906,ExitStatus:2,ExitedAt:2021-05-31 11:35:28.376822409 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:28.860737005Z] StopContainer for "a02772fcef07c90a33dd85eb05aab5a282370d8c89f9ac2c6c3daed985b12d49" returns successfully
INFO[2021-05-31T11:35:28.861109128Z] StopPodSandbox for "dceb9a712dc8e63224ff37a893615edc49d7e0809f2663ffca5b51a1b8f6b7d9"
INFO[2021-05-31T11:35:28.861179632Z] Container to stop "a02772fcef07c90a33dd85eb05aab5a282370d8c89f9ac2c6c3daed985b12d49" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:28.878180097Z] TearDown network for sandbox "dceb9a712dc8e63224ff37a893615edc49d7e0809f2663ffca5b51a1b8f6b7d9" successfully
INFO[2021-05-31T11:35:28.878209198Z] StopPodSandbox for "dceb9a712dc8e63224ff37a893615edc49d7e0809f2663ffca5b51a1b8f6b7d9" returns successfully
INFO[2021-05-31T11:35:28.912394339Z] shim disconnected                             id=ebcc58d0a3b98ba775f22ac16b396e58e625e98f0995ff601020a5116a31a3c1
ERRO[2021-05-31T11:35:28.912463643Z] copy shim log                                 error="read /proc/self/fd/167: file already closed"
INFO[2021-05-31T11:35:28.917247242Z] TaskExit event &TaskExit{ContainerID:a4907d95317ec33b9e18aefc2cd35948223a0af6fc5ac5df2671fa1b8ac4f40b,ID:a4907d95317ec33b9e18aefc2cd35948223a0af6fc5ac5df2671fa1b8ac4f40b,Pid:203675,ExitStatus:2,ExitedAt:2021-05-31 11:35:28.381707115 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:28.917314847Z] StopContainer for "ebcc58d0a3b98ba775f22ac16b396e58e625e98f0995ff601020a5116a31a3c1" returns successfully
INFO[2021-05-31T11:35:28.917702171Z] StopPodSandbox for "ad9d7f314ca4d3b3ed19c04e52b4ab9efacc24659668765f3ab9e13644c9a4ba"
INFO[2021-05-31T11:35:28.917785276Z] Container to stop "ebcc58d0a3b98ba775f22ac16b396e58e625e98f0995ff601020a5116a31a3c1" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:28.963061611Z] shim disconnected                             id=a4907d95317ec33b9e18aefc2cd35948223a0af6fc5ac5df2671fa1b8ac4f40b
ERRO[2021-05-31T11:35:28.963135415Z] copy shim log                                 error="read /proc/self/fd/104: file already closed"
INFO[2021-05-31T11:35:28.973200345Z] TaskExit event &TaskExit{ContainerID:fa0fcd4feacc9f4f74f8c39e3f512cd029e47e8e86b6062e9e5f767b2beb2f65,ID:fa0fcd4feacc9f4f74f8c39e3f512cd029e47e8e86b6062e9e5f767b2beb2f65,Pid:193114,ExitStatus:137,ExitedAt:2021-05-31 11:35:28.390651674 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:28.973263749Z] StopContainer for "a4907d95317ec33b9e18aefc2cd35948223a0af6fc5ac5df2671fa1b8ac4f40b" returns successfully
INFO[2021-05-31T11:35:28.973682776Z] StopPodSandbox for "3236f91258244df5401386921d30f79b459fb42e2e573df2f9b26ac6646b12cd"
INFO[2021-05-31T11:35:28.973742379Z] Container to stop "a4907d95317ec33b9e18aefc2cd35948223a0af6fc5ac5df2671fa1b8ac4f40b" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:29.024358948Z] shim disconnected                             id=fa0fcd4feacc9f4f74f8c39e3f512cd029e47e8e86b6062e9e5f767b2beb2f65
ERRO[2021-05-31T11:35:29.024446154Z] copy shim log                                 error="read /proc/self/fd/173: file already closed"
INFO[2021-05-31T11:35:29.028212189Z] TaskExit event &TaskExit{ContainerID:55373d72ab04ce7cd951686d8e8152307aa0b340bb1f58b6d670f868fe7a12bc,ID:55373d72ab04ce7cd951686d8e8152307aa0b340bb1f58b6d670f868fe7a12bc,Pid:203909,ExitStatus:2,ExitedAt:2021-05-31 11:35:28.39392828 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:29.028301695Z] StopContainer for "fa0fcd4feacc9f4f74f8c39e3f512cd029e47e8e86b6062e9e5f767b2beb2f65" returns successfully
INFO[2021-05-31T11:35:29.028847229Z] StopPodSandbox for "41c5bc1acf3f41342d5b140009a18a4435782b38e177a1153c42bceacfba1ed7"
INFO[2021-05-31T11:35:29.028913233Z] Container to stop "fa0fcd4feacc9f4f74f8c39e3f512cd029e47e8e86b6062e9e5f767b2beb2f65" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:29.040176038Z] TearDown network for sandbox "41c5bc1acf3f41342d5b140009a18a4435782b38e177a1153c42bceacfba1ed7" successfully
INFO[2021-05-31T11:35:29.040204440Z] StopPodSandbox for "41c5bc1acf3f41342d5b140009a18a4435782b38e177a1153c42bceacfba1ed7" returns successfully
INFO[2021-05-31T11:35:29.080670374Z] shim disconnected                             id=55373d72ab04ce7cd951686d8e8152307aa0b340bb1f58b6d670f868fe7a12bc
ERRO[2021-05-31T11:35:29.080757679Z] copy shim log                                 error="read /proc/self/fd/71: file already closed"
INFO[2021-05-31T11:35:29.085613683Z] TaskExit event &TaskExit{ContainerID:b339e15cd86e8095e17408d7bd37abfafc346f460feeeda471b01f84d79570fe,ID:b339e15cd86e8095e17408d7bd37abfafc346f460feeeda471b01f84d79570fe,Pid:195098,ExitStatus:2,ExitedAt:2021-05-31 11:35:28.397026174 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:29.085681887Z] StopContainer for "55373d72ab04ce7cd951686d8e8152307aa0b340bb1f58b6d670f868fe7a12bc" returns successfully
INFO[2021-05-31T11:35:29.086104914Z] StopPodSandbox for "0b1a2e7b98b8a529a9bfc4d32182299d790a5c9c4f0c3df1b71122a7c6cf3042"
INFO[2021-05-31T11:35:29.086166818Z] Container to stop "55373d72ab04ce7cd951686d8e8152307aa0b340bb1f58b6d670f868fe7a12bc" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:29.132606425Z] shim disconnected                             id=b339e15cd86e8095e17408d7bd37abfafc346f460feeeda471b01f84d79570fe
ERRO[2021-05-31T11:35:29.132692731Z] copy shim log                                 error="read /proc/self/fd/122: file already closed"
INFO[2021-05-31T11:35:29.136569573Z] TaskExit event &TaskExit{ContainerID:a95a719aa4b0d17316076cfe39f3fe7adebc8b99db1e5ab07f70b6df30f4618d,ID:a95a719aa4b0d17316076cfe39f3fe7adebc8b99db1e5ab07f70b6df30f4618d,Pid:199143,ExitStatus:0,ExitedAt:2021-05-31 11:35:28.405637613 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:29.136634177Z] StopContainer for "b339e15cd86e8095e17408d7bd37abfafc346f460feeeda471b01f84d79570fe" returns successfully
INFO[2021-05-31T11:35:29.137062704Z] StopPodSandbox for "e1899ad16d337576f1ffd41beb2c464450daf90a50d2138263b0485cfdef8860"
INFO[2021-05-31T11:35:29.137131808Z] Container to stop "b339e15cd86e8095e17408d7bd37abfafc346f460feeeda471b01f84d79570fe" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:29.184663484Z] shim disconnected                             id=a95a719aa4b0d17316076cfe39f3fe7adebc8b99db1e5ab07f70b6df30f4618d
ERRO[2021-05-31T11:35:29.184734989Z] copy shim log                                 error="read /proc/self/fd/110: file already closed"
INFO[2021-05-31T11:35:29.189125864Z] TaskExit event &TaskExit{ContainerID:0b52af0a7848f8b2c7d022e3848d93b93921e3d2bf1a655aa6dfb3d26618587b,ID:0b52af0a7848f8b2c7d022e3848d93b93921e3d2bf1a655aa6dfb3d26618587b,Pid:193252,ExitStatus:137,ExitedAt:2021-05-31 11:35:28.406628875 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:29.189188267Z] StopContainer for "a95a719aa4b0d17316076cfe39f3fe7adebc8b99db1e5ab07f70b6df30f4618d" returns successfully
INFO[2021-05-31T11:35:29.189615394Z] StopPodSandbox for "bf8bb075f1119e600352ea3d1be12b899016c5cb5706bc7b2d406d77095f3c8b"
INFO[2021-05-31T11:35:29.189675098Z] Container to stop "a95a719aa4b0d17316076cfe39f3fe7adebc8b99db1e5ab07f70b6df30f4618d" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:29.242015675Z] shim disconnected                             id=0b52af0a7848f8b2c7d022e3848d93b93921e3d2bf1a655aa6dfb3d26618587b
ERRO[2021-05-31T11:35:29.242149083Z] copy shim log                                 error="read /proc/self/fd/17: file already closed"
INFO[2021-05-31T11:35:29.245623901Z] TaskExit event &TaskExit{ContainerID:7240611c0a2661f07cde3d3d0022e1115e85064397a7f4f8165e11952930c309,ID:7240611c0a2661f07cde3d3d0022e1115e85064397a7f4f8165e11952930c309,Pid:193455,ExitStatus:137,ExitedAt:2021-05-31 11:35:28.409148132 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:29.245715906Z] StopContainer for "0b52af0a7848f8b2c7d022e3848d93b93921e3d2bf1a655aa6dfb3d26618587b" returns successfully
INFO[2021-05-31T11:35:29.246179935Z] StopPodSandbox for "077488a0b3aadb9b64c2bbbf2bd74eb939d500ac9598ebafa08b93640df69686"
INFO[2021-05-31T11:35:29.246265041Z] Container to stop "0b52af0a7848f8b2c7d022e3848d93b93921e3d2bf1a655aa6dfb3d26618587b" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:29.259171849Z] TearDown network for sandbox "077488a0b3aadb9b64c2bbbf2bd74eb939d500ac9598ebafa08b93640df69686" successfully
INFO[2021-05-31T11:35:29.259202651Z] StopPodSandbox for "077488a0b3aadb9b64c2bbbf2bd74eb939d500ac9598ebafa08b93640df69686" returns successfully
INFO[2021-05-31T11:35:29.288899410Z] shim disconnected                             id=7240611c0a2661f07cde3d3d0022e1115e85064397a7f4f8165e11952930c309
ERRO[2021-05-31T11:35:29.288960914Z] copy shim log                                 error="read /proc/self/fd/89: file already closed"
INFO[2021-05-31T11:35:29.295135900Z] TaskExit event &TaskExit{ContainerID:08dcbc707213a202f1bfc9a47bbbd5742726a38390e96e5cf52cddccd05fc1dc,ID:08dcbc707213a202f1bfc9a47bbbd5742726a38390e96e5cf52cddccd05fc1dc,Pid:193402,ExitStatus:137,ExitedAt:2021-05-31 11:35:28.409638363 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:29.295193604Z] StopContainer for "7240611c0a2661f07cde3d3d0022e1115e85064397a7f4f8165e11952930c309" returns successfully
INFO[2021-05-31T11:35:29.295648032Z] StopPodSandbox for "08dcbc707213a202f1bfc9a47bbbd5742726a38390e96e5cf52cddccd05fc1dc"
INFO[2021-05-31T11:35:29.295705436Z] Container to stop "7240611c0a2661f07cde3d3d0022e1115e85064397a7f4f8165e11952930c309" must be in running or unknown state, current state "CONTAINER_EXITED"
W0531 11:35:29.349855  236858 pod_container_deletor.go:79] Container "dceb9a712dc8e63224ff37a893615edc49d7e0809f2663ffca5b51a1b8f6b7d9" not found in pod's containers
W0531 11:35:29.355457  236858 pod_container_deletor.go:79] Container "41c5bc1acf3f41342d5b140009a18a4435782b38e177a1153c42bceacfba1ed7" not found in pod's containers
W0531 11:35:29.359394  236858 pod_container_deletor.go:79] Container "077488a0b3aadb9b64c2bbbf2bd74eb939d500ac9598ebafa08b93640df69686" not found in pod's containers
INFO[2021-05-31T11:35:29.360920019Z] shim disconnected                             id=08dcbc707213a202f1bfc9a47bbbd5742726a38390e96e5cf52cddccd05fc1dc
ERRO[2021-05-31T11:35:29.360989523Z] copy shim log                                 error="read /proc/self/fd/11: file already closed"
INFO[2021-05-31T11:35:29.361173635Z] TaskExit event &TaskExit{ContainerID:484b5f26c648ae1e8a14b6e321b85df898543dda324d589c9654845a25202858,ID:484b5f26c648ae1e8a14b6e321b85df898543dda324d589c9654845a25202858,Pid:203663,ExitStatus:0,ExitedAt:2021-05-31 11:35:28.419031451 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:29.361238439Z] TearDown network for sandbox "08dcbc707213a202f1bfc9a47bbbd5742726a38390e96e5cf52cddccd05fc1dc" successfully
INFO[2021-05-31T11:35:29.361262140Z] StopPodSandbox for "08dcbc707213a202f1bfc9a47bbbd5742726a38390e96e5cf52cddccd05fc1dc" returns successfully
INFO[2021-05-31T11:35:29.410564027Z] shim disconnected                             id=484b5f26c648ae1e8a14b6e321b85df898543dda324d589c9654845a25202858
ERRO[2021-05-31T11:35:29.410628031Z] copy shim log                                 error="read /proc/self/fd/59: file already closed"
INFO[2021-05-31T11:35:29.417170340Z] StopContainer for "484b5f26c648ae1e8a14b6e321b85df898543dda324d589c9654845a25202858" returns successfully
INFO[2021-05-31T11:35:29.417106936Z] TaskExit event &TaskExit{ContainerID:307f709c72d167bf70b186a526cb562fa4837c790be984f433ee132536b5822c,ID:307f709c72d167bf70b186a526cb562fa4837c790be984f433ee132536b5822c,Pid:203697,ExitStatus:2,ExitedAt:2021-05-31 11:35:28.429259192 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:29.417585466Z] StopPodSandbox for "a7bfead0eec2fc630fa99ca6e28e1d893d7c28a47d96fec8385487934a6ff18c"
INFO[2021-05-31T11:35:29.417660071Z] Container to stop "473e345c2545aed47f219e5b646672215d4c5011fe0eba43bf1c72e8b36506f8" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:29.417706974Z] Container to stop "484b5f26c648ae1e8a14b6e321b85df898543dda324d589c9654845a25202858" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:29.459631399Z] shim disconnected                             id=307f709c72d167bf70b186a526cb562fa4837c790be984f433ee132536b5822c
ERRO[2021-05-31T11:35:29.459710804Z] copy shim log                                 error="read /proc/self/fd/35: file already closed"
INFO[2021-05-31T11:35:29.463877965Z] TaskExit event &TaskExit{ContainerID:7f99f0de5096ddac1c6658063e307ba13a680a76e96685a76f8370728a7f159f,ID:7f99f0de5096ddac1c6658063e307ba13a680a76e96685a76f8370728a7f159f,Pid:198787,ExitStatus:0,ExitedAt:2021-05-31 11:35:28.429703819 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:29.463955569Z] StopContainer for "307f709c72d167bf70b186a526cb562fa4837c790be984f433ee132536b5822c" returns successfully
INFO[2021-05-31T11:35:29.464447100Z] StopPodSandbox for "dc7672cfde39f812fc8f29c3906cf998d197e62862efb74a37742c0abb5a8684"
INFO[2021-05-31T11:35:29.464522805Z] Container to stop "307f709c72d167bf70b186a526cb562fa4837c790be984f433ee132536b5822c" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:29.507442892Z] shim disconnected                             id=7f99f0de5096ddac1c6658063e307ba13a680a76e96685a76f8370728a7f159f
ERRO[2021-05-31T11:35:29.507523397Z] copy shim log                                 error="read /proc/self/fd/92: file already closed"
INFO[2021-05-31T11:35:29.511752762Z] TaskExit event &TaskExit{ContainerID:ff4f7aa2c9a53c3df3cf67ed2f5b76ea9f7dff200b97dfa898f92a3c1b763d3a,ID:ff4f7aa2c9a53c3df3cf67ed2f5b76ea9f7dff200b97dfa898f92a3c1b763d3a,Pid:203712,ExitStatus:2,ExitedAt:2021-05-31 11:35:28.430344059 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:29.511829467Z] StopContainer for "7f99f0de5096ddac1c6658063e307ba13a680a76e96685a76f8370728a7f159f" returns successfully
INFO[2021-05-31T11:35:29.512247193Z] StopPodSandbox for "c7131a99d594d9de61450820e4bb74080d1916a4623a7561d9981d6127986bd7"
INFO[2021-05-31T11:35:29.512314697Z] Container to stop "fcddf6b37679ac40690c76cf8dea0f4b54d37511a8c4b3df1ed69ed02abd8484" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:29.512344599Z] Container to stop "7f99f0de5096ddac1c6658063e307ba13a680a76e96685a76f8370728a7f159f" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:29.561832897Z] shim disconnected                             id=ff4f7aa2c9a53c3df3cf67ed2f5b76ea9f7dff200b97dfa898f92a3c1b763d3a
ERRO[2021-05-31T11:35:29.561898101Z] copy shim log                                 error="read /proc/self/fd/179: file already closed"
INFO[2021-05-31T11:35:29.565776144Z] TaskExit event &TaskExit{ContainerID:a45842958375fbb490dcbb1b02df255224280d3f6c7a77794256dd0ab043493a,ID:a45842958375fbb490dcbb1b02df255224280d3f6c7a77794256dd0ab043493a,Pid:204146,ExitStatus:2,ExitedAt:2021-05-31 11:35:28.434639328 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:29.565833948Z] StopContainer for "ff4f7aa2c9a53c3df3cf67ed2f5b76ea9f7dff200b97dfa898f92a3c1b763d3a" returns successfully
INFO[2021-05-31T11:35:29.566236273Z] StopPodSandbox for "b97deb11b14184ded631c24ad2c966dd806651354269091802c1570090874899"
INFO[2021-05-31T11:35:29.566303477Z] Container to stop "ff4f7aa2c9a53c3df3cf67ed2f5b76ea9f7dff200b97dfa898f92a3c1b763d3a" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:29.610721058Z] shim disconnected                             id=a45842958375fbb490dcbb1b02df255224280d3f6c7a77794256dd0ab043493a
ERRO[2021-05-31T11:35:29.610809263Z] copy shim log                                 error="read /proc/self/fd/101: file already closed"
INFO[2021-05-31T11:35:29.616215202Z] TaskExit event &TaskExit{ContainerID:dfb4378a6838ba4057b39ce35ea286ecd7877d1ef2ebed56407656a9ec2df1d6,ID:dfb4378a6838ba4057b39ce35ea286ecd7877d1ef2ebed56407656a9ec2df1d6,Pid:196906,ExitStatus:2,ExitedAt:2021-05-31 11:35:28.438114546 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:29.616275206Z] StopContainer for "a45842958375fbb490dcbb1b02df255224280d3f6c7a77794256dd0ab043493a" returns successfully
INFO[2021-05-31T11:35:29.616636628Z] StopPodSandbox for "600dccf452c5ce4361371721a73d39a537acd01f2f2d733d078eaad99e250d1d"
INFO[2021-05-31T11:35:29.616698932Z] Container to stop "a45842958375fbb490dcbb1b02df255224280d3f6c7a77794256dd0ab043493a" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:29.663975192Z] shim disconnected                             id=dfb4378a6838ba4057b39ce35ea286ecd7877d1ef2ebed56407656a9ec2df1d6
ERRO[2021-05-31T11:35:29.664064297Z] copy shim log                                 error="read /proc/self/fd/158: file already closed"
INFO[2021-05-31T11:35:29.668453872Z] TaskExit event &TaskExit{ContainerID:43d3cefdad766490e38623465108038771ee54b1053b921a83fea3da0d0943db,ID:43d3cefdad766490e38623465108038771ee54b1053b921a83fea3da0d0943db,Pid:199118,ExitStatus:0,ExitedAt:2021-05-31 11:35:28.437617115 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:29.668511576Z] StopContainer for "dfb4378a6838ba4057b39ce35ea286ecd7877d1ef2ebed56407656a9ec2df1d6" returns successfully
INFO[2021-05-31T11:35:29.668895900Z] StopPodSandbox for "6c8528f9b0babcb539f8aacea4371feb25a773eee2bac8a1c1a41249ec727d53"
INFO[2021-05-31T11:35:29.668951303Z] Container to stop "dfb4378a6838ba4057b39ce35ea286ecd7877d1ef2ebed56407656a9ec2df1d6" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:29.724230264Z] shim disconnected                             id=43d3cefdad766490e38623465108038771ee54b1053b921a83fea3da0d0943db
ERRO[2021-05-31T11:35:29.724300669Z] copy shim log                                 error="read /proc/self/fd/53: file already closed"
INFO[2021-05-31T11:35:29.727989100Z] TaskExit event &TaskExit{ContainerID:28ab98f2bc95b24782c5ecfe15e14eeab2d3da7d73d42a43a61ab39b9043fc5d,ID:28ab98f2bc95b24782c5ecfe15e14eeab2d3da7d73d42a43a61ab39b9043fc5d,Pid:195596,ExitStatus:0,ExitedAt:2021-05-31 11:35:28.438866693 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:29.728069605Z] StopContainer for "43d3cefdad766490e38623465108038771ee54b1053b921a83fea3da0d0943db" returns successfully
INFO[2021-05-31T11:35:29.729286481Z] StopPodSandbox for "69fec1b6339031dc3233048afa88191db808156968a754e109c19c2189901b5a"
INFO[2021-05-31T11:35:29.729459492Z] Container to stop "43d3cefdad766490e38623465108038771ee54b1053b921a83fea3da0d0943db" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:29.791841097Z] shim disconnected                             id=28ab98f2bc95b24782c5ecfe15e14eeab2d3da7d73d42a43a61ab39b9043fc5d
ERRO[2021-05-31T11:35:29.791906901Z] copy shim log                                 error="read /proc/self/fd/29: file already closed"
INFO[2021-05-31T11:35:29.795877850Z] TaskExit event &TaskExit{ContainerID:35b48bbfd52bbaab2073ae7f2cb49279b0ef94f6586ff81304aac891cb6b3c3d,ID:35b48bbfd52bbaab2073ae7f2cb49279b0ef94f6586ff81304aac891cb6b3c3d,Pid:204197,ExitStatus:2,ExitedAt:2021-05-31 11:35:28.439173912 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:29.795931053Z] StopContainer for "28ab98f2bc95b24782c5ecfe15e14eeab2d3da7d73d42a43a61ab39b9043fc5d" returns successfully
INFO[2021-05-31T11:35:29.842921395Z] shim disconnected                             id=35b48bbfd52bbaab2073ae7f2cb49279b0ef94f6586ff81304aac891cb6b3c3d
ERRO[2021-05-31T11:35:29.843021701Z] copy shim log                                 error="read /proc/self/fd/41: file already closed"
INFO[2021-05-31T11:35:29.855637391Z] TaskExit event &TaskExit{ContainerID:5196b8846626eaad2bfa309478ff1f5ab5731ad78c141b1b77cdbea8a1e57eaa,ID:5196b8846626eaad2bfa309478ff1f5ab5731ad78c141b1b77cdbea8a1e57eaa,Pid:203833,ExitStatus:2,ExitedAt:2021-05-31 11:35:28.439363324 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:29.855687694Z] StopContainer for "35b48bbfd52bbaab2073ae7f2cb49279b0ef94f6586ff81304aac891cb6b3c3d" returns successfully
INFO[2021-05-31T11:35:29.856142123Z] StopPodSandbox for "4572d3396b52957ae5e832ede8f6041fc706889fd258e2783a9075cb3700a075"
INFO[2021-05-31T11:35:29.856232828Z] Container to stop "35b48bbfd52bbaab2073ae7f2cb49279b0ef94f6586ff81304aac891cb6b3c3d" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:29.900256184Z] shim disconnected                             id=5196b8846626eaad2bfa309478ff1f5ab5731ad78c141b1b77cdbea8a1e57eaa
ERRO[2021-05-31T11:35:29.900331189Z] copy shim log                                 error="read /proc/self/fd/68: file already closed"
INFO[2021-05-31T11:35:29.904141628Z] TaskExit event &TaskExit{ContainerID:fcb78381d2248c96acfb5f1d4f0da188d3fd7db0dd2e1cafbf26f67959908459,ID:fcb78381d2248c96acfb5f1d4f0da188d3fd7db0dd2e1cafbf26f67959908459,Pid:197876,ExitStatus:0,ExitedAt:2021-05-31 11:35:28.445336698 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:29.904215332Z] StopContainer for "5196b8846626eaad2bfa309478ff1f5ab5731ad78c141b1b77cdbea8a1e57eaa" returns successfully
INFO[2021-05-31T11:35:29.904643059Z] StopPodSandbox for "de39add8ab0c0df611e1cb366f26227313dda568ebe82871a1f5ca387c49d8f1"
INFO[2021-05-31T11:35:29.904723664Z] Container to stop "5196b8846626eaad2bfa309478ff1f5ab5731ad78c141b1b77cdbea8a1e57eaa" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:29.953999549Z] shim disconnected                             id=fcb78381d2248c96acfb5f1d4f0da188d3fd7db0dd2e1cafbf26f67959908459
ERRO[2021-05-31T11:35:29.954079654Z] copy shim log                                 error="read /proc/self/fd/176: file already closed"
INFO[2021-05-31T11:35:29.958560535Z] TaskExit event &TaskExit{ContainerID:693a32131387a0d8303d6df80693d3de6a9019e9a66edd3d203187d88edc06f2,ID:693a32131387a0d8303d6df80693d3de6a9019e9a66edd3d203187d88edc06f2,Pid:197800,ExitStatus:0,ExitedAt:2021-05-31 11:35:28.469035982 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:29.958633639Z] StopContainer for "fcb78381d2248c96acfb5f1d4f0da188d3fd7db0dd2e1cafbf26f67959908459" returns successfully
INFO[2021-05-31T11:35:29.959096968Z] StopPodSandbox for "f13274e2c989d8f7f0f0f64c9489416b8c21e08d6250734a25fbf236ce6f5f46"
INFO[2021-05-31T11:35:29.959174473Z] Container to stop "fcb78381d2248c96acfb5f1d4f0da188d3fd7db0dd2e1cafbf26f67959908459" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:30.002050257Z] shim disconnected                             id=693a32131387a0d8303d6df80693d3de6a9019e9a66edd3d203187d88edc06f2
ERRO[2021-05-31T11:35:30.002125662Z] copy shim log                                 error="read /proc/self/fd/80: file already closed"
INFO[2021-05-31T11:35:30.006221818Z] TaskExit event &TaskExit{ContainerID:bef614a8c136075240c887d23cc202965eb8c7b4218501b2459eab892021e302,ID:bef614a8c136075240c887d23cc202965eb8c7b4218501b2459eab892021e302,Pid:204265,ExitStatus:143,ExitedAt:2021-05-31 11:35:28.521338956 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:30.006279722Z] StopContainer for "693a32131387a0d8303d6df80693d3de6a9019e9a66edd3d203187d88edc06f2" returns successfully
INFO[2021-05-31T11:35:30.006677947Z] StopPodSandbox for "b8b3446d73a12d2dd76f254b3e3c8b7275e188b1cac00105f3a9a4a071a4d10a"
INFO[2021-05-31T11:35:30.006764852Z] Container to stop "693a32131387a0d8303d6df80693d3de6a9019e9a66edd3d203187d88edc06f2" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:30.047751919Z] shim disconnected                             id=bef614a8c136075240c887d23cc202965eb8c7b4218501b2459eab892021e302
ERRO[2021-05-31T11:35:30.047837524Z] copy shim log                                 error="read /proc/self/fd/134: file already closed"
INFO[2021-05-31T11:35:30.053520580Z] TaskExit event &TaskExit{ContainerID:db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97,ID:db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97,Pid:202712,ExitStatus:137,ExitedAt:2021-05-31 11:35:28.52219451 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:30.053594184Z] StopContainer for "bef614a8c136075240c887d23cc202965eb8c7b4218501b2459eab892021e302" returns successfully
INFO[2021-05-31T11:35:30.054010410Z] StopPodSandbox for "2cd5b03f21adbac5af8ee9067f717bebf6a63563aaf7d30367c4905d64cc423f"
INFO[2021-05-31T11:35:30.054106416Z] Container to stop "bef614a8c136075240c887d23cc202965eb8c7b4218501b2459eab892021e302" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:30.054138418Z] Container to stop "41e474f3f2995d5eff262daf7a83ea16086d8b58ab04c806f6e54abecf0d3125" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:30.099595464Z] shim disconnected                             id=db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97
ERRO[2021-05-31T11:35:30.099690370Z] copy shim log                                 error="read /proc/self/fd/146: file already closed"
INFO[2021-05-31T11:35:30.099856381Z] TaskExit event &TaskExit{ContainerID:b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48,ID:b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48,Pid:202203,ExitStatus:137,ExitedAt:2021-05-31 11:35:28.581373315 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:30.146846222Z] shim disconnected                             id=b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48
ERRO[2021-05-31T11:35:30.146946329Z] copy shim log                                 error="read /proc/self/fd/125: file already closed"
INFO[2021-05-31T11:35:30.147260148Z] TaskExit event &TaskExit{ContainerID:407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749,ID:407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749,Pid:202192,ExitStatus:137,ExitedAt:2021-05-31 11:35:28.802488058 +0000 UTC,XXX_unrecognized:[],}
ERRO[2021-05-31T11:35:30.151596120Z] StopPodSandbox for "db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97" failed  error="failed to destroy network for sandbox \"db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:30.151853  236858 remote_runtime.go:143] StopPodSandbox "db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:30.151914  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97"}
E0531 11:35:30.151968  236858 kubelet_pods.go:1246] Failed killing the pod "csi-snapshotter-5f6d68fffd-h4m2j": failed to "KillPodSandbox" for "cb6cfdea-2375-4a9f-9c7b-b0060b958196" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
INFO[2021-05-31T11:35:30.198512757Z] shim disconnected                             id=407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749
ERRO[2021-05-31T11:35:30.198617764Z] copy shim log                                 error="read /proc/self/fd/44: file already closed"
INFO[2021-05-31T11:35:30.198734071Z] TaskExit event &TaskExit{ContainerID:6414e3c3a9d87f08fbb4b5fa1dc2f6bb2dbb0aba79218c139560cb06f0c4b88e,ID:6414e3c3a9d87f08fbb4b5fa1dc2f6bb2dbb0aba79218c139560cb06f0c4b88e,Pid:201906,ExitStatus:137,ExitedAt:2021-05-31 11:35:28.834397256 +0000 UTC,XXX_unrecognized:[],}
ERRO[2021-05-31T11:35:30.205787212Z] StopPodSandbox for "b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48" failed  error="failed to destroy network for sandbox \"b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:30.206077  236858 remote_runtime.go:143] StopPodSandbox "b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:30.206146  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48"}
E0531 11:35:30.206191  236858 kubelet_pods.go:1246] Failed killing the pod "csi-resizer-67c44f4c59-l9cr2": failed to "KillPodSandbox" for "5e7593ef-b495-4dc2-bc62-ca9a52d15b76" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
ERRO[2021-05-31T11:35:30.247810143Z] StopPodSandbox for "407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749" failed  error="failed to destroy network for sandbox \"407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:30.248080  236858 remote_runtime.go:143] StopPodSandbox "407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:30.248192  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749"}
E0531 11:35:30.248227  236858 kubelet_pods.go:1246] Failed killing the pod "csi-resizer-67c44f4c59-hr4gp": failed to "KillPodSandbox" for "eaa5431c-29cf-432a-a1cf-2302986663ef" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
INFO[2021-05-31T11:35:30.262254848Z] shim disconnected                             id=6414e3c3a9d87f08fbb4b5fa1dc2f6bb2dbb0aba79218c139560cb06f0c4b88e
ERRO[2021-05-31T11:35:30.262346753Z] copy shim log                                 error="read /proc/self/fd/77: file already closed"
INFO[2021-05-31T11:35:30.262568067Z] TaskExit event &TaskExit{ContainerID:ad9d7f314ca4d3b3ed19c04e52b4ab9efacc24659668765f3ab9e13644c9a4ba,ID:ad9d7f314ca4d3b3ed19c04e52b4ab9efacc24659668765f3ab9e13644c9a4ba,Pid:201951,ExitStatus:137,ExitedAt:2021-05-31 11:35:28.985658225 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:30.312191274Z] StopContainer for "fa0fcd4feacc9f4f74f8c39e3f512cd029e47e8e86b6062e9e5f767b2beb2f65" with timeout 30 (s)
INFO[2021-05-31T11:35:30.312250378Z] Container to stop "fa0fcd4feacc9f4f74f8c39e3f512cd029e47e8e86b6062e9e5f767b2beb2f65" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:30.312284380Z] StopContainer for "fa0fcd4feacc9f4f74f8c39e3f512cd029e47e8e86b6062e9e5f767b2beb2f65" returns successfully
INFO[2021-05-31T11:35:30.312328983Z] StopContainer for "a02772fcef07c90a33dd85eb05aab5a282370d8c89f9ac2c6c3daed985b12d49" with timeout 30 (s)
INFO[2021-05-31T11:35:30.312359784Z] Container to stop "a02772fcef07c90a33dd85eb05aab5a282370d8c89f9ac2c6c3daed985b12d49" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:30.312403687Z] StopContainer for "a02772fcef07c90a33dd85eb05aab5a282370d8c89f9ac2c6c3daed985b12d49" returns successfully
INFO[2021-05-31T11:35:30.312440590Z] StopContainer for "abd46405e7365b01a15eb71e3fd97595811086fa89d804ff89baa5925e5c09c0" with timeout 30 (s)
INFO[2021-05-31T11:35:30.312462891Z] Container to stop "abd46405e7365b01a15eb71e3fd97595811086fa89d804ff89baa5925e5c09c0" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:30.312478592Z] StopContainer for "abd46405e7365b01a15eb71e3fd97595811086fa89d804ff89baa5925e5c09c0" returns successfully
INFO[2021-05-31T11:35:30.312527495Z] StopContainer for "0b52af0a7848f8b2c7d022e3848d93b93921e3d2bf1a655aa6dfb3d26618587b" with timeout 30 (s)
INFO[2021-05-31T11:35:30.312563097Z] Container to stop "0b52af0a7848f8b2c7d022e3848d93b93921e3d2bf1a655aa6dfb3d26618587b" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:30.312593699Z] StopContainer for "0b52af0a7848f8b2c7d022e3848d93b93921e3d2bf1a655aa6dfb3d26618587b" returns successfully
INFO[2021-05-31T11:35:30.312628701Z] StopContainer for "168cf0a156e6deea9792a794248fc62bc60712ff820878712bc9d2468b27be0b" with timeout 30 (s)
INFO[2021-05-31T11:35:30.312663003Z] Container to stop "168cf0a156e6deea9792a794248fc62bc60712ff820878712bc9d2468b27be0b" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:30.312700906Z] StopContainer for "168cf0a156e6deea9792a794248fc62bc60712ff820878712bc9d2468b27be0b" returns successfully
INFO[2021-05-31T11:35:30.312676804Z] StopContainer for "4ad62ff9625721b50946986e0f546a40cf40cfc693582eceb1f1c3cb1f7e3084" with timeout 30 (s)
INFO[2021-05-31T11:35:30.312761410Z] Container to stop "4ad62ff9625721b50946986e0f546a40cf40cfc693582eceb1f1c3cb1f7e3084" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:30.312793612Z] StopContainer for "4ad62ff9625721b50946986e0f546a40cf40cfc693582eceb1f1c3cb1f7e3084" returns successfully
INFO[2021-05-31T11:35:30.312676804Z] StopContainer for "7240611c0a2661f07cde3d3d0022e1115e85064397a7f4f8165e11952930c309" with timeout 30 (s)
INFO[2021-05-31T11:35:30.312851715Z] Container to stop "7240611c0a2661f07cde3d3d0022e1115e85064397a7f4f8165e11952930c309" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:30.312874717Z] StopPodSandbox for "dceb9a712dc8e63224ff37a893615edc49d7e0809f2663ffca5b51a1b8f6b7d9"
INFO[2021-05-31T11:35:30.312973723Z] Container to stop "a02772fcef07c90a33dd85eb05aab5a282370d8c89f9ac2c6c3daed985b12d49" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:30.312884617Z] StopContainer for "7240611c0a2661f07cde3d3d0022e1115e85064397a7f4f8165e11952930c309" returns successfully
INFO[2021-05-31T11:35:30.313097631Z] TearDown network for sandbox "dceb9a712dc8e63224ff37a893615edc49d7e0809f2663ffca5b51a1b8f6b7d9" successfully
INFO[2021-05-31T11:35:30.313118532Z] StopPodSandbox for "dceb9a712dc8e63224ff37a893615edc49d7e0809f2663ffca5b51a1b8f6b7d9" returns successfully
INFO[2021-05-31T11:35:30.312992624Z] StopPodSandbox for "b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48"
INFO[2021-05-31T11:35:30.313217838Z] Container to stop "4ad62ff9625721b50946986e0f546a40cf40cfc693582eceb1f1c3cb1f7e3084" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:30.312866716Z] StopPodSandbox for "407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749"
INFO[2021-05-31T11:35:30.312907019Z] StopPodSandbox for "41c5bc1acf3f41342d5b140009a18a4435782b38e177a1153c42bceacfba1ed7"
INFO[2021-05-31T11:35:30.313369948Z] Container to stop "abd46405e7365b01a15eb71e3fd97595811086fa89d804ff89baa5925e5c09c0" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:30.313372348Z] Container to stop "fa0fcd4feacc9f4f74f8c39e3f512cd029e47e8e86b6062e9e5f767b2beb2f65" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:30.313623864Z] TearDown network for sandbox "41c5bc1acf3f41342d5b140009a18a4435782b38e177a1153c42bceacfba1ed7" successfully
INFO[2021-05-31T11:35:30.313643165Z] StopPodSandbox for "41c5bc1acf3f41342d5b140009a18a4435782b38e177a1153c42bceacfba1ed7" returns successfully
INFO[2021-05-31T11:35:30.313001925Z] StopPodSandbox for "db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97"
INFO[2021-05-31T11:35:30.312920720Z] StopPodSandbox for "077488a0b3aadb9b64c2bbbf2bd74eb939d500ac9598ebafa08b93640df69686"
INFO[2021-05-31T11:35:30.313723070Z] Container to stop "168cf0a156e6deea9792a794248fc62bc60712ff820878712bc9d2468b27be0b" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:30.313752472Z] Container to stop "0b52af0a7848f8b2c7d022e3848d93b93921e3d2bf1a655aa6dfb3d26618587b" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:30.314100993Z] TearDown network for sandbox "077488a0b3aadb9b64c2bbbf2bd74eb939d500ac9598ebafa08b93640df69686" successfully
INFO[2021-05-31T11:35:30.314121695Z] StopPodSandbox for "077488a0b3aadb9b64c2bbbf2bd74eb939d500ac9598ebafa08b93640df69686" returns successfully
INFO[2021-05-31T11:35:30.313386549Z] StopPodSandbox for "08dcbc707213a202f1bfc9a47bbbd5742726a38390e96e5cf52cddccd05fc1dc"
INFO[2021-05-31T11:35:30.314189799Z] shim disconnected                             id=ad9d7f314ca4d3b3ed19c04e52b4ab9efacc24659668765f3ab9e13644c9a4ba
INFO[2021-05-31T11:35:30.314207700Z] Container to stop "7240611c0a2661f07cde3d3d0022e1115e85064397a7f4f8165e11952930c309" must be in running or unknown state, current state "CONTAINER_EXITED"
ERRO[2021-05-31T11:35:30.314240602Z] copy shim log                                 error="read /proc/self/fd/116: file already closed"
INFO[2021-05-31T11:35:30.314334508Z] TearDown network for sandbox "08dcbc707213a202f1bfc9a47bbbd5742726a38390e96e5cf52cddccd05fc1dc" successfully
INFO[2021-05-31T11:35:30.314381711Z] StopPodSandbox for "08dcbc707213a202f1bfc9a47bbbd5742726a38390e96e5cf52cddccd05fc1dc" returns successfully
INFO[2021-05-31T11:35:30.314425014Z] TaskExit event &TaskExit{ContainerID:3236f91258244df5401386921d30f79b459fb42e2e573df2f9b26ac6646b12cd,ID:3236f91258244df5401386921d30f79b459fb42e2e573df2f9b26ac6646b12cd,Pid:201740,ExitStatus:137,ExitedAt:2021-05-31 11:35:29.024980687 +0000 UTC,XXX_unrecognized:[],}
ERRO[2021-05-31T11:35:30.321607263Z] StopPodSandbox for "6414e3c3a9d87f08fbb4b5fa1dc2f6bb2dbb0aba79218c139560cb06f0c4b88e" failed  error="failed to destroy network for sandbox \"6414e3c3a9d87f08fbb4b5fa1dc2f6bb2dbb0aba79218c139560cb06f0c4b88e\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:30.321871  236858 remote_runtime.go:143] StopPodSandbox "6414e3c3a9d87f08fbb4b5fa1dc2f6bb2dbb0aba79218c139560cb06f0c4b88e" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "6414e3c3a9d87f08fbb4b5fa1dc2f6bb2dbb0aba79218c139560cb06f0c4b88e": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:30.321951  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "6414e3c3a9d87f08fbb4b5fa1dc2f6bb2dbb0aba79218c139560cb06f0c4b88e"}
E0531 11:35:30.321993  236858 kubelet_pods.go:1246] Failed killing the pod "csi-provisioner-848fd784dc-jnbmw": failed to "KillPodSandbox" for "1127820b-ea86-42df-bd62-cfbac36e5543" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"6414e3c3a9d87f08fbb4b5fa1dc2f6bb2dbb0aba79218c139560cb06f0c4b88e\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
W0531 11:35:30.363692  236858 pod_container_deletor.go:79] Container "ad9d7f314ca4d3b3ed19c04e52b4ab9efacc24659668765f3ab9e13644c9a4ba" not found in pod's containers
W0531 11:35:30.366808  236858 pod_container_deletor.go:79] Container "db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97" not found in pod's containers
ERRO[2021-05-31T11:35:30.375100212Z] StopPodSandbox for "db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97" failed  error="failed to destroy network for sandbox \"db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:30.375262  236858 remote_runtime.go:143] StopPodSandbox "db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:30.375306  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97"}
E0531 11:35:30.375341  236858 kubelet_pods.go:1246] Failed killing the pod "csi-snapshotter-5f6d68fffd-h4m2j": failed to "KillPodSandbox" for "cb6cfdea-2375-4a9f-9c7b-b0060b958196" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
INFO[2021-05-31T11:35:30.377766479Z] shim disconnected                             id=3236f91258244df5401386921d30f79b459fb42e2e573df2f9b26ac6646b12cd
ERRO[2021-05-31T11:35:30.377825983Z] copy shim log                                 error="read /proc/self/fd/38: file already closed"
INFO[2021-05-31T11:35:30.378052697Z] TaskExit event &TaskExit{ContainerID:0b1a2e7b98b8a529a9bfc4d32182299d790a5c9c4f0c3df1b71122a7c6cf3042,ID:0b1a2e7b98b8a529a9bfc4d32182299d790a5c9c4f0c3df1b71122a7c6cf3042,Pid:201785,ExitStatus:137,ExitedAt:2021-05-31 11:35:29.145472131 +0000 UTC,XXX_unrecognized:[],}
ERRO[2021-05-31T11:35:30.378170605Z] StopPodSandbox for "ad9d7f314ca4d3b3ed19c04e52b4ab9efacc24659668765f3ab9e13644c9a4ba" failed  error="failed to destroy network for sandbox \"ad9d7f314ca4d3b3ed19c04e52b4ab9efacc24659668765f3ab9e13644c9a4ba\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:30.378363  236858 remote_runtime.go:143] StopPodSandbox "ad9d7f314ca4d3b3ed19c04e52b4ab9efacc24659668765f3ab9e13644c9a4ba" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "ad9d7f314ca4d3b3ed19c04e52b4ab9efacc24659668765f3ab9e13644c9a4ba": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:30.378396  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "ad9d7f314ca4d3b3ed19c04e52b4ab9efacc24659668765f3ab9e13644c9a4ba"}
E0531 11:35:30.378439  236858 kubelet_pods.go:1246] Failed killing the pod "csi-provisioner-848fd784dc-wbfpz": failed to "KillPodSandbox" for "39e0aff0-8422-4308-af72-51d8000ce3bd" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"ad9d7f314ca4d3b3ed19c04e52b4ab9efacc24659668765f3ab9e13644c9a4ba\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
W0531 11:35:30.379711  236858 pod_container_deletor.go:79] Container "6414e3c3a9d87f08fbb4b5fa1dc2f6bb2dbb0aba79218c139560cb06f0c4b88e" not found in pod's containers
ERRO[2021-05-31T11:35:30.380823571Z] StopPodSandbox for "407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749" failed  error="failed to destroy network for sandbox \"407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:30.380952  236858 remote_runtime.go:143] StopPodSandbox "407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:30.381017  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749"}
W0531 11:35:30.381037  236858 pod_container_deletor.go:79] Container "b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48" not found in pod's containers
E0531 11:35:30.381080  236858 kubelet_pods.go:1246] Failed killing the pod "csi-resizer-67c44f4c59-hr4gp": failed to "KillPodSandbox" for "eaa5431c-29cf-432a-a1cf-2302986663ef" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
W0531 11:35:30.382300  236858 pod_container_deletor.go:79] Container "08dcbc707213a202f1bfc9a47bbbd5742726a38390e96e5cf52cddccd05fc1dc" not found in pod's containers
ERRO[2021-05-31T11:35:30.385901489Z] StopPodSandbox for "b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48" failed  error="failed to destroy network for sandbox \"b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:30.386058  236858 remote_runtime.go:143] StopPodSandbox "b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:30.386103  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48"}
E0531 11:35:30.386132  236858 kubelet_pods.go:1246] Failed killing the pod "csi-resizer-67c44f4c59-l9cr2": failed to "KillPodSandbox" for "5e7593ef-b495-4dc2-bc62-ca9a52d15b76" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
W0531 11:35:30.391883  236858 pod_container_deletor.go:79] Container "407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749" not found in pod's containers
INFO[2021-05-31T11:35:30.394446624Z] Kill container "e276d005b51326c3bc7a88995e40611a3df12be1bc7653ac8707df4341615085"
INFO[2021-05-31T11:35:30.419506392Z] Finish piping stderr of container "e276d005b51326c3bc7a88995e40611a3df12be1bc7653ac8707df4341615085"
INFO[2021-05-31T11:35:30.419531294Z] Finish piping stdout of container "e276d005b51326c3bc7a88995e40611a3df12be1bc7653ac8707df4341615085"
INFO[2021-05-31T11:35:30.431097918Z] shim disconnected                             id=0b1a2e7b98b8a529a9bfc4d32182299d790a5c9c4f0c3df1b71122a7c6cf3042
INFO[2021-05-31T11:35:30.431112119Z] TaskExit event &TaskExit{ContainerID:e1899ad16d337576f1ffd41beb2c464450daf90a50d2138263b0485cfdef8860,ID:e1899ad16d337576f1ffd41beb2c464450daf90a50d2138263b0485cfdef8860,Pid:194750,ExitStatus:137,ExitedAt:2021-05-31 11:35:29.188201206 +0000 UTC,XXX_unrecognized:[],}
ERRO[2021-05-31T11:35:30.431158322Z] copy shim log                                 error="read /proc/self/fd/14: file already closed"
ERRO[2021-05-31T11:35:30.433671379Z] StopPodSandbox for "3236f91258244df5401386921d30f79b459fb42e2e573df2f9b26ac6646b12cd" failed  error="failed to destroy network for sandbox \"3236f91258244df5401386921d30f79b459fb42e2e573df2f9b26ac6646b12cd\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:30.433906  236858 remote_runtime.go:143] StopPodSandbox "3236f91258244df5401386921d30f79b459fb42e2e573df2f9b26ac6646b12cd" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "3236f91258244df5401386921d30f79b459fb42e2e573df2f9b26ac6646b12cd": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:30.433962  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "3236f91258244df5401386921d30f79b459fb42e2e573df2f9b26ac6646b12cd"}
E0531 11:35:30.433998  236858 kubelet_pods.go:1246] Failed killing the pod "csi-attacher-85976fc6f-zzjxl": failed to "KillPodSandbox" for "5e6c7961-e03e-4af2-9370-a2072e7fe014" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"3236f91258244df5401386921d30f79b459fb42e2e573df2f9b26ac6646b12cd\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
ERRO[2021-05-31T11:35:30.481190554Z] StopPodSandbox for "0b1a2e7b98b8a529a9bfc4d32182299d790a5c9c4f0c3df1b71122a7c6cf3042" failed  error="failed to destroy network for sandbox \"0b1a2e7b98b8a529a9bfc4d32182299d790a5c9c4f0c3df1b71122a7c6cf3042\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
INFO[2021-05-31T11:35:30.481394367Z] shim disconnected                             id=e1899ad16d337576f1ffd41beb2c464450daf90a50d2138263b0485cfdef8860
ERRO[2021-05-31T11:35:30.481455171Z] copy shim log                                 error="read /proc/self/fd/161: file already closed"
E0531 11:35:30.481471  236858 remote_runtime.go:143] StopPodSandbox "0b1a2e7b98b8a529a9bfc4d32182299d790a5c9c4f0c3df1b71122a7c6cf3042" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "0b1a2e7b98b8a529a9bfc4d32182299d790a5c9c4f0c3df1b71122a7c6cf3042": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:30.481522  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "0b1a2e7b98b8a529a9bfc4d32182299d790a5c9c4f0c3df1b71122a7c6cf3042"}
E0531 11:35:30.481564  236858 kubelet_pods.go:1246] Failed killing the pod "csi-attacher-85976fc6f-4z9sj": failed to "KillPodSandbox" for "235f7ff1-cf1f-40b1-9022-909ab5a2b3b7" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"0b1a2e7b98b8a529a9bfc4d32182299d790a5c9c4f0c3df1b71122a7c6cf3042\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
INFO[2021-05-31T11:35:30.481681685Z] TaskExit event &TaskExit{ContainerID:bf8bb075f1119e600352ea3d1be12b899016c5cb5706bc7b2d406d77095f3c8b,ID:bf8bb075f1119e600352ea3d1be12b899016c5cb5706bc7b2d406d77095f3c8b,Pid:198577,ExitStatus:137,ExitedAt:2021-05-31 11:35:29.237589598 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:30.481738189Z] TearDown network for sandbox "e1899ad16d337576f1ffd41beb2c464450daf90a50d2138263b0485cfdef8860" successfully
INFO[2021-05-31T11:35:30.481763290Z] StopPodSandbox for "e1899ad16d337576f1ffd41beb2c464450daf90a50d2138263b0485cfdef8860" returns successfully
INFO[2021-05-31T11:35:30.527764970Z] shim disconnected                             id=bf8bb075f1119e600352ea3d1be12b899016c5cb5706bc7b2d406d77095f3c8b
ERRO[2021-05-31T11:35:30.527841075Z] copy shim log                                 error="read /proc/self/fd/137: file already closed"
INFO[2021-05-31T11:35:30.528056488Z] TaskExit event &TaskExit{ContainerID:a7bfead0eec2fc630fa99ca6e28e1d893d7c28a47d96fec8385487934a6ff18c,ID:a7bfead0eec2fc630fa99ca6e28e1d893d7c28a47d96fec8385487934a6ff18c,Pid:199681,ExitStatus:137,ExitedAt:2021-05-31 11:35:29.47914282 +0000 UTC,XXX_unrecognized:[],}
ERRO[2021-05-31T11:35:30.581279420Z] StopPodSandbox for "bf8bb075f1119e600352ea3d1be12b899016c5cb5706bc7b2d406d77095f3c8b" failed  error="failed to destroy network for sandbox \"bf8bb075f1119e600352ea3d1be12b899016c5cb5706bc7b2d406d77095f3c8b\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:30.581508  236858 remote_runtime.go:143] StopPodSandbox "bf8bb075f1119e600352ea3d1be12b899016c5cb5706bc7b2d406d77095f3c8b" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "bf8bb075f1119e600352ea3d1be12b899016c5cb5706bc7b2d406d77095f3c8b": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:30.581555  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "bf8bb075f1119e600352ea3d1be12b899016c5cb5706bc7b2d406d77095f3c8b"}
E0531 11:35:30.581587  236858 kubelet_pods.go:1246] Failed killing the pod "instance-manager-r-8aaba421": failed to "KillPodSandbox" for "aa6618b8-92ae-4fdc-af00-a7f83ad41060" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"bf8bb075f1119e600352ea3d1be12b899016c5cb5706bc7b2d406d77095f3c8b\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
INFO[2021-05-31T11:35:30.584481521Z] shim disconnected                             id=a7bfead0eec2fc630fa99ca6e28e1d893d7c28a47d96fec8385487934a6ff18c
ERRO[2021-05-31T11:35:30.584550325Z] copy shim log                                 error="read /proc/self/fd/107: file already closed"
INFO[2021-05-31T11:35:30.584762638Z] TaskExit event &TaskExit{ContainerID:dc7672cfde39f812fc8f29c3906cf998d197e62862efb74a37742c0abb5a8684,ID:dc7672cfde39f812fc8f29c3906cf998d197e62862efb74a37742c0abb5a8684,Pid:202142,ExitStatus:137,ExitedAt:2021-05-31 11:35:29.51460124 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:30.584824142Z] TearDown network for sandbox "a7bfead0eec2fc630fa99ca6e28e1d893d7c28a47d96fec8385487934a6ff18c" successfully
INFO[2021-05-31T11:35:30.584853244Z] StopPodSandbox for "a7bfead0eec2fc630fa99ca6e28e1d893d7c28a47d96fec8385487934a6ff18c" returns successfully
INFO[2021-05-31T11:35:30.632530029Z] shim disconnected                             id=dc7672cfde39f812fc8f29c3906cf998d197e62862efb74a37742c0abb5a8684
ERRO[2021-05-31T11:35:30.632616234Z] copy shim log                                 error="read /proc/self/fd/149: file already closed"
INFO[2021-05-31T11:35:30.632802146Z] TaskExit event &TaskExit{ContainerID:c7131a99d594d9de61450820e4bb74080d1916a4623a7561d9981d6127986bd7,ID:c7131a99d594d9de61450820e4bb74080d1916a4623a7561d9981d6127986bd7,Pid:197482,ExitStatus:137,ExitedAt:2021-05-31 11:35:29.565436323 +0000 UTC,XXX_unrecognized:[],}
ERRO[2021-05-31T11:35:30.685145923Z] StopPodSandbox for "dc7672cfde39f812fc8f29c3906cf998d197e62862efb74a37742c0abb5a8684" failed  error="failed to destroy network for sandbox \"dc7672cfde39f812fc8f29c3906cf998d197e62862efb74a37742c0abb5a8684\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:30.685372  236858 remote_runtime.go:143] StopPodSandbox "dc7672cfde39f812fc8f29c3906cf998d197e62862efb74a37742c0abb5a8684" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "dc7672cfde39f812fc8f29c3906cf998d197e62862efb74a37742c0abb5a8684": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:30.685423  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "dc7672cfde39f812fc8f29c3906cf998d197e62862efb74a37742c0abb5a8684"}
E0531 11:35:30.685455  236858 kubelet_pods.go:1246] Failed killing the pod "csi-resizer-67c44f4c59-95hnp": failed to "KillPodSandbox" for "1048bad5-1bc6-4c0e-85c8-84a93d174a79" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"dc7672cfde39f812fc8f29c3906cf998d197e62862efb74a37742c0abb5a8684\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
INFO[2021-05-31T11:35:30.689127972Z] shim disconnected                             id=c7131a99d594d9de61450820e4bb74080d1916a4623a7561d9981d6127986bd7
ERRO[2021-05-31T11:35:30.689219078Z] copy shim log                                 error="read /proc/self/fd/140: file already closed"
INFO[2021-05-31T11:35:30.689366487Z] TaskExit event &TaskExit{ContainerID:b97deb11b14184ded631c24ad2c966dd806651354269091802c1570090874899,ID:b97deb11b14184ded631c24ad2c966dd806651354269091802c1570090874899,Pid:201912,ExitStatus:137,ExitedAt:2021-05-31 11:35:29.612654379 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:30.735935103Z] shim disconnected                             id=b97deb11b14184ded631c24ad2c966dd806651354269091802c1570090874899
ERRO[2021-05-31T11:35:30.736010107Z] copy shim log                                 error="read /proc/self/fd/131: file already closed"
INFO[2021-05-31T11:35:30.736253023Z] TaskExit event &TaskExit{ContainerID:600dccf452c5ce4361371721a73d39a537acd01f2f2d733d078eaad99e250d1d,ID:600dccf452c5ce4361371721a73d39a537acd01f2f2d733d078eaad99e250d1d,Pid:202838,ExitStatus:137,ExitedAt:2021-05-31 11:35:29.688207609 +0000 UTC,XXX_unrecognized:[],}
ERRO[2021-05-31T11:35:30.740188469Z] StopPodSandbox for "c7131a99d594d9de61450820e4bb74080d1916a4623a7561d9981d6127986bd7" failed  error="failed to destroy network for sandbox \"c7131a99d594d9de61450820e4bb74080d1916a4623a7561d9981d6127986bd7\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:30.740408  236858 remote_runtime.go:143] StopPodSandbox "c7131a99d594d9de61450820e4bb74080d1916a4623a7561d9981d6127986bd7" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "c7131a99d594d9de61450820e4bb74080d1916a4623a7561d9981d6127986bd7": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:30.740454  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "c7131a99d594d9de61450820e4bb74080d1916a4623a7561d9981d6127986bd7"}
E0531 11:35:30.740486  236858 kubelet_pods.go:1246] Failed killing the pod "longhorn-driver-deployer-6dbb7667fb-4cxv2": failed to "KillPodSandbox" for "db4c7ed7-3763-4aaf-8d08-e00dbe793c95" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"c7131a99d594d9de61450820e4bb74080d1916a4623a7561d9981d6127986bd7\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
ERRO[2021-05-31T11:35:30.783957609Z] StopPodSandbox for "b97deb11b14184ded631c24ad2c966dd806651354269091802c1570090874899" failed  error="failed to destroy network for sandbox \"b97deb11b14184ded631c24ad2c966dd806651354269091802c1570090874899\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:30.784217  236858 remote_runtime.go:143] StopPodSandbox "b97deb11b14184ded631c24ad2c966dd806651354269091802c1570090874899" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "b97deb11b14184ded631c24ad2c966dd806651354269091802c1570090874899": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:30.784267  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "b97deb11b14184ded631c24ad2c966dd806651354269091802c1570090874899"}
E0531 11:35:30.784299  236858 kubelet_pods.go:1246] Failed killing the pod "csi-provisioner-848fd784dc-6d9b5": failed to "KillPodSandbox" for "e493a540-7004-45ee-a5be-baaccedbd0c4" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"b97deb11b14184ded631c24ad2c966dd806651354269091802c1570090874899\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
INFO[2021-05-31T11:35:30.793704319Z] shim disconnected                             id=600dccf452c5ce4361371721a73d39a537acd01f2f2d733d078eaad99e250d1d
ERRO[2021-05-31T11:35:30.793780724Z] copy shim log                                 error="read /proc/self/fd/74: file already closed"
INFO[2021-05-31T11:35:30.793938734Z] TaskExit event &TaskExit{ContainerID:6c8528f9b0babcb539f8aacea4371feb25a773eee2bac8a1c1a41249ec727d53,ID:6c8528f9b0babcb539f8aacea4371feb25a773eee2bac8a1c1a41249ec727d53,Pid:196528,ExitStatus:137,ExitedAt:2021-05-31 11:35:29.728419726 +0000 UTC,XXX_unrecognized:[],}
ERRO[2021-05-31T11:35:30.840031220Z] StopPodSandbox for "600dccf452c5ce4361371721a73d39a537acd01f2f2d733d078eaad99e250d1d" failed  error="failed to destroy network for sandbox \"600dccf452c5ce4361371721a73d39a537acd01f2f2d733d078eaad99e250d1d\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:30.840314  236858 remote_runtime.go:143] StopPodSandbox "600dccf452c5ce4361371721a73d39a537acd01f2f2d733d078eaad99e250d1d" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "600dccf452c5ce4361371721a73d39a537acd01f2f2d733d078eaad99e250d1d": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:30.840365  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "600dccf452c5ce4361371721a73d39a537acd01f2f2d733d078eaad99e250d1d"}
E0531 11:35:30.840398  236858 kubelet_pods.go:1246] Failed killing the pod "csi-snapshotter-5f6d68fffd-jmkvd": failed to "KillPodSandbox" for "fa3bcd38-b929-46d6-867c-f4d3748f6236" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"600dccf452c5ce4361371721a73d39a537acd01f2f2d733d078eaad99e250d1d\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
INFO[2021-05-31T11:35:30.842978404Z] shim disconnected                             id=6c8528f9b0babcb539f8aacea4371feb25a773eee2bac8a1c1a41249ec727d53
ERRO[2021-05-31T11:35:30.843051909Z] copy shim log                                 error="read /proc/self/fd/86: file already closed"
INFO[2021-05-31T11:35:30.843260922Z] TaskExit event &TaskExit{ContainerID:69fec1b6339031dc3233048afa88191db808156968a754e109c19c2189901b5a,ID:69fec1b6339031dc3233048afa88191db808156968a754e109c19c2189901b5a,Pid:198379,ExitStatus:137,ExitedAt:2021-05-31 11:35:29.79237393 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:30.890097854Z] shim disconnected                             id=69fec1b6339031dc3233048afa88191db808156968a754e109c19c2189901b5a
ERRO[2021-05-31T11:35:30.890178659Z] copy shim log                                 error="read /proc/self/fd/83: file already closed"
ERRO[2021-05-31T11:35:30.890401973Z] StopPodSandbox for "6c8528f9b0babcb539f8aacea4371feb25a773eee2bac8a1c1a41249ec727d53" failed  error="failed to destroy network for sandbox \"6c8528f9b0babcb539f8aacea4371feb25a773eee2bac8a1c1a41249ec727d53\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
INFO[2021-05-31T11:35:30.890402473Z] TaskExit event &TaskExit{ContainerID:4572d3396b52957ae5e832ede8f6041fc706889fd258e2783a9075cb3700a075,ID:4572d3396b52957ae5e832ede8f6041fc706889fd258e2783a9075cb3700a075,Pid:202836,ExitStatus:137,ExitedAt:2021-05-31 11:35:29.906318464 +0000 UTC,XXX_unrecognized:[],}
E0531 11:35:30.890652  236858 remote_runtime.go:143] StopPodSandbox "6c8528f9b0babcb539f8aacea4371feb25a773eee2bac8a1c1a41249ec727d53" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "6c8528f9b0babcb539f8aacea4371feb25a773eee2bac8a1c1a41249ec727d53": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:30.890700  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "6c8528f9b0babcb539f8aacea4371feb25a773eee2bac8a1c1a41249ec727d53"}
E0531 11:35:30.890733  236858 kubelet_pods.go:1246] Failed killing the pod "rke2-metrics-server-559dc47bd8-5ltnt": failed to "KillPodSandbox" for "fc39632c-6527-4438-9215-756a57de4f33" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"6c8528f9b0babcb539f8aacea4371feb25a773eee2bac8a1c1a41249ec727d53\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
INFO[2021-05-31T11:35:30.936069732Z] shim disconnected                             id=4572d3396b52957ae5e832ede8f6041fc706889fd258e2783a9075cb3700a075
ERRO[2021-05-31T11:35:30.936155138Z] copy shim log                                 error="read /proc/self/fd/56: file already closed"
INFO[2021-05-31T11:35:30.936333549Z] TaskExit event &TaskExit{ContainerID:de39add8ab0c0df611e1cb366f26227313dda568ebe82871a1f5ca387c49d8f1,ID:de39add8ab0c0df611e1cb366f26227313dda568ebe82871a1f5ca387c49d8f1,Pid:201815,ExitStatus:137,ExitedAt:2021-05-31 11:35:29.958563335 +0000 UTC,XXX_unrecognized:[],}
ERRO[2021-05-31T11:35:30.936603666Z] StopPodSandbox for "69fec1b6339031dc3233048afa88191db808156968a754e109c19c2189901b5a" failed  error="failed to destroy network for sandbox \"69fec1b6339031dc3233048afa88191db808156968a754e109c19c2189901b5a\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:30.936956  236858 remote_runtime.go:143] StopPodSandbox "69fec1b6339031dc3233048afa88191db808156968a754e109c19c2189901b5a" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "69fec1b6339031dc3233048afa88191db808156968a754e109c19c2189901b5a": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:30.937004  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "69fec1b6339031dc3233048afa88191db808156968a754e109c19c2189901b5a"}
E0531 11:35:30.937038  236858 kubelet_pods.go:1246] Failed killing the pod "instance-manager-e-bfcf7752": failed to "KillPodSandbox" for "bf474a29-b6d0-437f-bd82-1c33cfd356af" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"69fec1b6339031dc3233048afa88191db808156968a754e109c19c2189901b5a\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
INFO[0010] Waiting for deletion of etcd static pod
ERRO[2021-05-31T11:35:30.983345692Z] StopPodSandbox for "4572d3396b52957ae5e832ede8f6041fc706889fd258e2783a9075cb3700a075" failed  error="failed to destroy network for sandbox \"4572d3396b52957ae5e832ede8f6041fc706889fd258e2783a9075cb3700a075\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:30.983568  236858 remote_runtime.go:143] StopPodSandbox "4572d3396b52957ae5e832ede8f6041fc706889fd258e2783a9075cb3700a075" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "4572d3396b52957ae5e832ede8f6041fc706889fd258e2783a9075cb3700a075": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:30.983614  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "4572d3396b52957ae5e832ede8f6041fc706889fd258e2783a9075cb3700a075"}
E0531 11:35:30.983645  236858 kubelet_pods.go:1246] Failed killing the pod "csi-snapshotter-5f6d68fffd-8t8p2": failed to "KillPodSandbox" for "febda349-c8a4-4939-85e5-d7721278fa0a" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"4572d3396b52957ae5e832ede8f6041fc706889fd258e2783a9075cb3700a075\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
INFO[2021-05-31T11:35:30.996097690Z] shim disconnected                             id=de39add8ab0c0df611e1cb366f26227313dda568ebe82871a1f5ca387c49d8f1
ERRO[2021-05-31T11:35:30.996172395Z] copy shim log                                 error="read /proc/self/fd/155: file already closed"
INFO[2021-05-31T11:35:30.996431811Z] TaskExit event &TaskExit{ContainerID:f13274e2c989d8f7f0f0f64c9489416b8c21e08d6250734a25fbf236ce6f5f46,ID:f13274e2c989d8f7f0f0f64c9489416b8c21e08d6250734a25fbf236ce6f5f46,Pid:197448,ExitStatus:137,ExitedAt:2021-05-31 11:35:30.010321875 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:31.045353574Z] shim disconnected                             id=f13274e2c989d8f7f0f0f64c9489416b8c21e08d6250734a25fbf236ce6f5f46
ERRO[2021-05-31T11:35:31.045427579Z] copy shim log                                 error="read /proc/self/fd/170: file already closed"
INFO[2021-05-31T11:35:31.045460581Z] TaskExit event &TaskExit{ContainerID:b8b3446d73a12d2dd76f254b3e3c8b7275e188b1cac00105f3a9a4a071a4d10a,ID:b8b3446d73a12d2dd76f254b3e3c8b7275e188b1cac00105f3a9a4a071a4d10a,Pid:197432,ExitStatus:137,ExitedAt:2021-05-31 11:35:30.048587071 +0000 UTC,XXX_unrecognized:[],}
ERRO[2021-05-31T11:35:31.050568101Z] StopPodSandbox for "de39add8ab0c0df611e1cb366f26227313dda568ebe82871a1f5ca387c49d8f1" failed  error="failed to destroy network for sandbox \"de39add8ab0c0df611e1cb366f26227313dda568ebe82871a1f5ca387c49d8f1\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:31.050790  236858 remote_runtime.go:143] StopPodSandbox "de39add8ab0c0df611e1cb366f26227313dda568ebe82871a1f5ca387c49d8f1" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "de39add8ab0c0df611e1cb366f26227313dda568ebe82871a1f5ca387c49d8f1": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:31.050837  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "de39add8ab0c0df611e1cb366f26227313dda568ebe82871a1f5ca387c49d8f1"}
E0531 11:35:31.050869  236858 kubelet_pods.go:1246] Failed killing the pod "csi-attacher-85976fc6f-jvjvn": failed to "KillPodSandbox" for "1ce3185f-dce7-44d1-b79d-5858e1794a03" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"de39add8ab0c0df611e1cb366f26227313dda568ebe82871a1f5ca387c49d8f1\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
ERRO[2021-05-31T11:35:31.097738854Z] StopPodSandbox for "f13274e2c989d8f7f0f0f64c9489416b8c21e08d6250734a25fbf236ce6f5f46" failed  error="failed to destroy network for sandbox \"f13274e2c989d8f7f0f0f64c9489416b8c21e08d6250734a25fbf236ce6f5f46\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:31.097981  236858 remote_runtime.go:143] StopPodSandbox "f13274e2c989d8f7f0f0f64c9489416b8c21e08d6250734a25fbf236ce6f5f46" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "f13274e2c989d8f7f0f0f64c9489416b8c21e08d6250734a25fbf236ce6f5f46": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:31.098032  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "f13274e2c989d8f7f0f0f64c9489416b8c21e08d6250734a25fbf236ce6f5f46"}
E0531 11:35:31.098082  236858 kubelet_pods.go:1246] Failed killing the pod "longhorn-manager-47khv": failed to "KillPodSandbox" for "6c413fd7-39c4-4cf0-9b24-14ab39b66811" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"f13274e2c989d8f7f0f0f64c9489416b8c21e08d6250734a25fbf236ce6f5f46\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
INFO[2021-05-31T11:35:31.099678875Z] shim disconnected                             id=b8b3446d73a12d2dd76f254b3e3c8b7275e188b1cac00105f3a9a4a071a4d10a
ERRO[2021-05-31T11:35:31.099752480Z] copy shim log                                 error="read /proc/self/fd/128: file already closed"
INFO[2021-05-31T11:35:31.099965693Z] TaskExit event &TaskExit{ContainerID:2cd5b03f21adbac5af8ee9067f717bebf6a63563aaf7d30367c4905d64cc423f,ID:2cd5b03f21adbac5af8ee9067f717bebf6a63563aaf7d30367c4905d64cc423f,Pid:202791,ExitStatus:137,ExitedAt:2021-05-31 11:35:30.118094922 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:31.145811663Z] shim disconnected                             id=2cd5b03f21adbac5af8ee9067f717bebf6a63563aaf7d30367c4905d64cc423f
ERRO[2021-05-31T11:35:31.145895169Z] copy shim log                                 error="read /proc/self/fd/32: file already closed"
INFO[2021-05-31T11:35:31.146022377Z] TaskExit event &TaskExit{ContainerID:e276d005b51326c3bc7a88995e40611a3df12be1bc7653ac8707df4341615085,ID:e276d005b51326c3bc7a88995e40611a3df12be1bc7653ac8707df4341615085,Pid:195921,ExitStatus:137,ExitedAt:2021-05-31 11:35:30.421468415 +0000 UTC,XXX_unrecognized:[],}
ERRO[2021-05-31T11:35:31.147638178Z] StopPodSandbox for "b8b3446d73a12d2dd76f254b3e3c8b7275e188b1cac00105f3a9a4a071a4d10a" failed  error="failed to destroy network for sandbox \"b8b3446d73a12d2dd76f254b3e3c8b7275e188b1cac00105f3a9a4a071a4d10a\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:31.147863  236858 remote_runtime.go:143] StopPodSandbox "b8b3446d73a12d2dd76f254b3e3c8b7275e188b1cac00105f3a9a4a071a4d10a" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "b8b3446d73a12d2dd76f254b3e3c8b7275e188b1cac00105f3a9a4a071a4d10a": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:31.147913  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "b8b3446d73a12d2dd76f254b3e3c8b7275e188b1cac00105f3a9a4a071a4d10a"}
E0531 11:35:31.147946  236858 kubelet_pods.go:1246] Failed killing the pod "longhorn-ui-6fd7c49b94-sz94c": failed to "KillPodSandbox" for "a88c79af-f4fa-443b-9f24-1861fcf44d38" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"b8b3446d73a12d2dd76f254b3e3c8b7275e188b1cac00105f3a9a4a071a4d10a\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
INFO[2021-05-31T11:35:31.189351489Z] shim disconnected                             id=e276d005b51326c3bc7a88995e40611a3df12be1bc7653ac8707df4341615085
ERRO[2021-05-31T11:35:31.189427794Z] copy shim log                                 error="read /proc/self/fd/164: file already closed"
INFO[2021-05-31T11:35:31.194254696Z] StopContainer for "e276d005b51326c3bc7a88995e40611a3df12be1bc7653ac8707df4341615085" returns successfully
INFO[2021-05-31T11:35:31.194677223Z] StopPodSandbox for "9b79f57f2efe26b48b58d3077be38cec15456bd081d3f5f82d61048f2d06d814"
INFO[2021-05-31T11:35:31.194747627Z] Container to stop "4691d432dcfc36587d5ad25837428c9c2369a3c6eef9971c90625a0981bb9b3a" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:31.194780429Z] Container to stop "572935c64bb6ed9ee4e570548fe267be2faed9cfc9521fbc278b885ffdb4eb5f" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:31.194794330Z] Container to stop "28ab98f2bc95b24782c5ecfe15e14eeab2d3da7d73d42a43a61ab39b9043fc5d" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:31.194806131Z] Container to stop "e276d005b51326c3bc7a88995e40611a3df12be1bc7653ac8707df4341615085" must be in running or unknown state, current state "CONTAINER_EXITED"
ERRO[2021-05-31T11:35:31.203662485Z] StopPodSandbox for "2cd5b03f21adbac5af8ee9067f717bebf6a63563aaf7d30367c4905d64cc423f" failed  error="failed to destroy network for sandbox \"2cd5b03f21adbac5af8ee9067f717bebf6a63563aaf7d30367c4905d64cc423f\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:31.203859  236858 remote_runtime.go:143] StopPodSandbox "2cd5b03f21adbac5af8ee9067f717bebf6a63563aaf7d30367c4905d64cc423f" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "2cd5b03f21adbac5af8ee9067f717bebf6a63563aaf7d30367c4905d64cc423f": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:31.203904  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "2cd5b03f21adbac5af8ee9067f717bebf6a63563aaf7d30367c4905d64cc423f"}
E0531 11:35:31.203937  236858 kubelet_pods.go:1246] Failed killing the pod "longhorn-csi-plugin-p9sx7": failed to "KillPodSandbox" for "0599b534-b12d-4553-92d8-0a0217c1aaa5" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"2cd5b03f21adbac5af8ee9067f717bebf6a63563aaf7d30367c4905d64cc423f\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
INFO[2021-05-31T11:35:31.244516843Z] TaskExit event &TaskExit{ContainerID:9b79f57f2efe26b48b58d3077be38cec15456bd081d3f5f82d61048f2d06d814,ID:9b79f57f2efe26b48b58d3077be38cec15456bd081d3f5f82d61048f2d06d814,Pid:194802,ExitStatus:137,ExitedAt:2021-05-31 11:35:31.244287829 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:31.291861207Z] shim disconnected                             id=9b79f57f2efe26b48b58d3077be38cec15456bd081d3f5f82d61048f2d06d814
ERRO[2021-05-31T11:35:31.291962513Z] copy shim log                                 error="read /proc/self/fd/95: file already closed"
INFO[2021-05-31T11:35:31.292092021Z] TearDown network for sandbox "9b79f57f2efe26b48b58d3077be38cec15456bd081d3f5f82d61048f2d06d814" successfully
INFO[2021-05-31T11:35:31.292120823Z] StopPodSandbox for "9b79f57f2efe26b48b58d3077be38cec15456bd081d3f5f82d61048f2d06d814" returns successfully
W0531 11:35:31.399562  236858 pod_container_deletor.go:79] Container "69fec1b6339031dc3233048afa88191db808156968a754e109c19c2189901b5a" not found in pod's containers
W0531 11:35:31.401170  236858 pod_container_deletor.go:79] Container "b8b3446d73a12d2dd76f254b3e3c8b7275e188b1cac00105f3a9a4a071a4d10a" not found in pod's containers
W0531 11:35:31.402313  236858 pod_container_deletor.go:79] Container "bf8bb075f1119e600352ea3d1be12b899016c5cb5706bc7b2d406d77095f3c8b" not found in pod's containers
W0531 11:35:31.404161  236858 pod_container_deletor.go:79] Container "9b79f57f2efe26b48b58d3077be38cec15456bd081d3f5f82d61048f2d06d814" not found in pod's containers
W0531 11:35:31.405375  236858 pod_container_deletor.go:79] Container "c7131a99d594d9de61450820e4bb74080d1916a4623a7561d9981d6127986bd7" not found in pod's containers
W0531 11:35:31.406504  236858 pod_container_deletor.go:79] Container "a7bfead0eec2fc630fa99ca6e28e1d893d7c28a47d96fec8385487934a6ff18c" not found in pod's containers
W0531 11:35:31.407466  236858 pod_container_deletor.go:79] Container "4572d3396b52957ae5e832ede8f6041fc706889fd258e2783a9075cb3700a075" not found in pod's containers
W0531 11:35:31.408383  236858 pod_container_deletor.go:79] Container "dc7672cfde39f812fc8f29c3906cf998d197e62862efb74a37742c0abb5a8684" not found in pod's containers
W0531 11:35:31.409464  236858 pod_container_deletor.go:79] Container "0b1a2e7b98b8a529a9bfc4d32182299d790a5c9c4f0c3df1b71122a7c6cf3042" not found in pod's containers
W0531 11:35:31.410494  236858 pod_container_deletor.go:79] Container "6c8528f9b0babcb539f8aacea4371feb25a773eee2bac8a1c1a41249ec727d53" not found in pod's containers
W0531 11:35:31.411752  236858 pod_container_deletor.go:79] Container "600dccf452c5ce4361371721a73d39a537acd01f2f2d733d078eaad99e250d1d" not found in pod's containers
W0531 11:35:31.412729  236858 pod_container_deletor.go:79] Container "de39add8ab0c0df611e1cb366f26227313dda568ebe82871a1f5ca387c49d8f1" not found in pod's containers
W0531 11:35:31.414163  236858 pod_container_deletor.go:79] Container "e1899ad16d337576f1ffd41beb2c464450daf90a50d2138263b0485cfdef8860" not found in pod's containers
W0531 11:35:31.415485  236858 pod_container_deletor.go:79] Container "2cd5b03f21adbac5af8ee9067f717bebf6a63563aaf7d30367c4905d64cc423f" not found in pod's containers
W0531 11:35:31.416472  236858 pod_container_deletor.go:79] Container "3236f91258244df5401386921d30f79b459fb42e2e573df2f9b26ac6646b12cd" not found in pod's containers
W0531 11:35:31.417531  236858 pod_container_deletor.go:79] Container "f13274e2c989d8f7f0f0f64c9489416b8c21e08d6250734a25fbf236ce6f5f46" not found in pod's containers
W0531 11:35:31.418504  236858 pod_container_deletor.go:79] Container "b97deb11b14184ded631c24ad2c966dd806651354269091802c1570090874899" not found in pod's containers
INFO[2021-05-31T11:35:33.872722884Z] Finish piping stdout of container "b10aa220c3246c0b8317a22fb79d8d1676ac88dd135d6521880b0e0756544f0d"
INFO[2021-05-31T11:35:33.872752086Z] Finish piping stderr of container "b10aa220c3246c0b8317a22fb79d8d1676ac88dd135d6521880b0e0756544f0d"
INFO[2021-05-31T11:35:33.874823915Z] TaskExit event &TaskExit{ContainerID:b10aa220c3246c0b8317a22fb79d8d1676ac88dd135d6521880b0e0756544f0d,ID:b10aa220c3246c0b8317a22fb79d8d1676ac88dd135d6521880b0e0756544f0d,Pid:196824,ExitStatus:0,ExitedAt:2021-05-31 11:35:33.874615702 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:33.918394943Z] shim disconnected                             id=b10aa220c3246c0b8317a22fb79d8d1676ac88dd135d6521880b0e0756544f0d
ERRO[2021-05-31T11:35:33.918471148Z] copy shim log                                 error="read /proc/self/fd/119: file already closed"
INFO[2021-05-31T11:35:33.923342353Z] StopContainer for "b10aa220c3246c0b8317a22fb79d8d1676ac88dd135d6521880b0e0756544f0d" returns successfully
INFO[2021-05-31T11:35:33.923739278Z] StopPodSandbox for "109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7"
INFO[2021-05-31T11:35:33.923807182Z] Container to stop "b10aa220c3246c0b8317a22fb79d8d1676ac88dd135d6521880b0e0756544f0d" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:33.987359961Z] TaskExit event &TaskExit{ContainerID:109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7,ID:109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7,Pid:196294,ExitStatus:137,ExitedAt:2021-05-31 11:35:33.987086344 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:34.036989168Z] shim disconnected                             id=109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7
ERRO[2021-05-31T11:35:34.037146178Z] copy shim log                                 error="read /proc/self/fd/20: file already closed"
ERRO[2021-05-31T11:35:34.083546582Z] StopPodSandbox for "109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7" failed  error="failed to destroy network for sandbox \"109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:34.083859  236858 remote_runtime.go:143] StopPodSandbox "109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:34.083913  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7"}
E0531 11:35:34.083953  236858 kubelet_pods.go:1246] Failed killing the pod "rke2-coredns-rke2-coredns-675b5596bc-m2hw2": failed to "KillPodSandbox" for "c8901107-ef6b-4cce-9a6a-840211ce986f" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
INFO[2021-05-31T11:35:34.310437687Z] StopContainer for "b10aa220c3246c0b8317a22fb79d8d1676ac88dd135d6521880b0e0756544f0d" with timeout 30 (s)
INFO[2021-05-31T11:35:34.310492691Z] Container to stop "b10aa220c3246c0b8317a22fb79d8d1676ac88dd135d6521880b0e0756544f0d" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:34.310534593Z] StopContainer for "b10aa220c3246c0b8317a22fb79d8d1676ac88dd135d6521880b0e0756544f0d" returns successfully
INFO[2021-05-31T11:35:34.311249938Z] StopPodSandbox for "109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7"
INFO[2021-05-31T11:35:34.311308342Z] Container to stop "b10aa220c3246c0b8317a22fb79d8d1676ac88dd135d6521880b0e0756544f0d" must be in running or unknown state, current state "CONTAINER_EXITED"
ERRO[2021-05-31T11:35:34.357202015Z] StopPodSandbox for "109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7" failed  error="failed to destroy network for sandbox \"109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:34.357400  236858 remote_runtime.go:143] StopPodSandbox "109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:34.357449  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7"}
E0531 11:35:34.357482  236858 kubelet_pods.go:1246] Failed killing the pod "rke2-coredns-rke2-coredns-675b5596bc-m2hw2": failed to "KillPodSandbox" for "c8901107-ef6b-4cce-9a6a-840211ce986f" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
W0531 11:35:34.427674  236858 pod_container_deletor.go:79] Container "109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7" not found in pod's containers
INFO[0020] Waiting for deletion of kube-controller-manager static pod
INFO[0030] Waiting for deletion of kube-controller-manager static pod
INFO[2021-05-31T11:35:58.363960376Z] Kill container "4d7b1966552f191aeba7d827cf4fb02a2811525ec67105111ce3c43f34e97853"
INFO[2021-05-31T11:35:58.389650985Z] Finish piping stderr of container "4d7b1966552f191aeba7d827cf4fb02a2811525ec67105111ce3c43f34e97853"
INFO[2021-05-31T11:35:58.389687487Z] Finish piping stdout of container "4d7b1966552f191aeba7d827cf4fb02a2811525ec67105111ce3c43f34e97853"
INFO[2021-05-31T11:35:58.392209545Z] TaskExit event &TaskExit{ContainerID:4d7b1966552f191aeba7d827cf4fb02a2811525ec67105111ce3c43f34e97853,ID:4d7b1966552f191aeba7d827cf4fb02a2811525ec67105111ce3c43f34e97853,Pid:201292,ExitStatus:137,ExitedAt:2021-05-31 11:35:58.391959429 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:58.443299743Z] shim disconnected                             id=4d7b1966552f191aeba7d827cf4fb02a2811525ec67105111ce3c43f34e97853
ERRO[2021-05-31T11:35:58.443398950Z] copy shim log                                 error="read /proc/self/fd/65: file already closed"
INFO[2021-05-31T11:35:58.447916432Z] StopContainer for "4d7b1966552f191aeba7d827cf4fb02a2811525ec67105111ce3c43f34e97853" returns successfully
INFO[2021-05-31T11:35:58.448331958Z] StopPodSandbox for "15dbe828817c7a1a794ed2ecdf34fd73569e56a0d9ac25908fe8e3ef60bd0804"
INFO[2021-05-31T11:35:58.448408663Z] Container to stop "4d7b1966552f191aeba7d827cf4fb02a2811525ec67105111ce3c43f34e97853" must be in running or unknown state, current state "CONTAINER_EXITED"
INFO[2021-05-31T11:35:58.493566190Z] TaskExit event &TaskExit{ContainerID:15dbe828817c7a1a794ed2ecdf34fd73569e56a0d9ac25908fe8e3ef60bd0804,ID:15dbe828817c7a1a794ed2ecdf34fd73569e56a0d9ac25908fe8e3ef60bd0804,Pid:198470,ExitStatus:137,ExitedAt:2021-05-31 11:35:58.493352777 +0000 UTC,XXX_unrecognized:[],}
INFO[2021-05-31T11:35:58.540084203Z] shim disconnected                             id=15dbe828817c7a1a794ed2ecdf34fd73569e56a0d9ac25908fe8e3ef60bd0804
ERRO[2021-05-31T11:35:58.540151207Z] copy shim log                                 error="read /proc/self/fd/23: file already closed"
ERRO[2021-05-31T11:35:58.597841419Z] StopPodSandbox for "15dbe828817c7a1a794ed2ecdf34fd73569e56a0d9ac25908fe8e3ef60bd0804" failed  error="failed to destroy network for sandbox \"15dbe828817c7a1a794ed2ecdf34fd73569e56a0d9ac25908fe8e3ef60bd0804\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:35:58.598139  236858 remote_runtime.go:143] StopPodSandbox "15dbe828817c7a1a794ed2ecdf34fd73569e56a0d9ac25908fe8e3ef60bd0804" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "15dbe828817c7a1a794ed2ecdf34fd73569e56a0d9ac25908fe8e3ef60bd0804": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:35:58.598190  236858 kuberuntime_manager.go:923] Failed to stop sandbox {"containerd" "15dbe828817c7a1a794ed2ecdf34fd73569e56a0d9ac25908fe8e3ef60bd0804"}
E0531 11:35:58.598229  236858 kubelet_pods.go:1246] Failed killing the pod "engine-image-ei-817782be-v8b2b": failed to "KillPodSandbox" for "6ae9645d-8498-49a3-b37e-79f6598cfc76" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \"15dbe828817c7a1a794ed2ecdf34fd73569e56a0d9ac25908fe8e3ef60bd0804\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
W0531 11:35:59.490505  236858 pod_container_deletor.go:79] Container "15dbe828817c7a1a794ed2ecdf34fd73569e56a0d9ac25908fe8e3ef60bd0804" not found in pod's containers
INFO[0040] Waiting for deletion of kube-controller-manager static pod
INFO[0050] Waiting for deletion of kube-apiserver static pod
INFO[0060] Waiting for deletion of kube-controller-manager static pod
I0531 11:36:26.271013  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 43d3cefdad766490e38623465108038771ee54b1053b921a83fea3da0d0943db
INFO[2021-05-31T11:36:26.272771729Z] RemoveContainer for "43d3cefdad766490e38623465108038771ee54b1053b921a83fea3da0d0943db"
INFO[2021-05-31T11:36:26.290603645Z] RemoveContainer for "43d3cefdad766490e38623465108038771ee54b1053b921a83fea3da0d0943db" returns successfully
I0531 11:36:26.290852  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: a02772fcef07c90a33dd85eb05aab5a282370d8c89f9ac2c6c3daed985b12d49
INFO[2021-05-31T11:36:26.291802320Z] RemoveContainer for "a02772fcef07c90a33dd85eb05aab5a282370d8c89f9ac2c6c3daed985b12d49"
INFO[2021-05-31T11:36:26.303567657Z] RemoveContainer for "a02772fcef07c90a33dd85eb05aab5a282370d8c89f9ac2c6c3daed985b12d49" returns successfully
I0531 11:36:26.303750  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 168cf0a156e6deea9792a794248fc62bc60712ff820878712bc9d2468b27be0b
INFO[2021-05-31T11:36:26.304729929Z] RemoveContainer for "168cf0a156e6deea9792a794248fc62bc60712ff820878712bc9d2468b27be0b"
INFO[2021-05-31T11:36:26.322369934Z] RemoveContainer for "168cf0a156e6deea9792a794248fc62bc60712ff820878712bc9d2468b27be0b" returns successfully
I0531 11:36:26.322554  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 484b5f26c648ae1e8a14b6e321b85df898543dda324d589c9654845a25202858
INFO[2021-05-31T11:36:26.323431100Z] RemoveContainer for "484b5f26c648ae1e8a14b6e321b85df898543dda324d589c9654845a25202858"
INFO[2021-05-31T11:36:26.336556722Z] RemoveContainer for "484b5f26c648ae1e8a14b6e321b85df898543dda324d589c9654845a25202858" returns successfully
I0531 11:36:26.336729  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: b339e15cd86e8095e17408d7bd37abfafc346f460feeeda471b01f84d79570fe
INFO[2021-05-31T11:36:26.337630789Z] RemoveContainer for "b339e15cd86e8095e17408d7bd37abfafc346f460feeeda471b01f84d79570fe"
INFO[2021-05-31T11:36:26.354764062Z] RemoveContainer for "b339e15cd86e8095e17408d7bd37abfafc346f460feeeda471b01f84d79570fe" returns successfully
I0531 11:36:26.355077  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: a4907d95317ec33b9e18aefc2cd35948223a0af6fc5ac5df2671fa1b8ac4f40b
INFO[2021-05-31T11:36:26.355976738Z] RemoveContainer for "a4907d95317ec33b9e18aefc2cd35948223a0af6fc5ac5df2671fa1b8ac4f40b"
INFO[2021-05-31T11:36:26.369127861Z] RemoveContainer for "a4907d95317ec33b9e18aefc2cd35948223a0af6fc5ac5df2671fa1b8ac4f40b" returns successfully
I0531 11:36:26.369373  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: e276d005b51326c3bc7a88995e40611a3df12be1bc7653ac8707df4341615085
INFO[2021-05-31T11:36:26.370203828Z] RemoveContainer for "e276d005b51326c3bc7a88995e40611a3df12be1bc7653ac8707df4341615085"
INFO[2021-05-31T11:36:26.383410855Z] RemoveContainer for "e276d005b51326c3bc7a88995e40611a3df12be1bc7653ac8707df4341615085" returns successfully
I0531 11:36:26.383603  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 35b48bbfd52bbaab2073ae7f2cb49279b0ef94f6586ff81304aac891cb6b3c3d
INFO[2021-05-31T11:36:26.384666334Z] RemoveContainer for "35b48bbfd52bbaab2073ae7f2cb49279b0ef94f6586ff81304aac891cb6b3c3d"
INFO[2021-05-31T11:36:26.396149153Z] RemoveContainer for "35b48bbfd52bbaab2073ae7f2cb49279b0ef94f6586ff81304aac891cb6b3c3d" returns successfully
I0531 11:36:26.396396  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 41e474f3f2995d5eff262daf7a83ea16086d8b58ab04c806f6e54abecf0d3125
INFO[2021-05-31T11:36:26.397369529Z] RemoveContainer for "41e474f3f2995d5eff262daf7a83ea16086d8b58ab04c806f6e54abecf0d3125"
INFO[2021-05-31T11:36:26.408314614Z] RemoveContainer for "41e474f3f2995d5eff262daf7a83ea16086d8b58ab04c806f6e54abecf0d3125" returns successfully
I0531 11:36:26.408502  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 5196b8846626eaad2bfa309478ff1f5ab5731ad78c141b1b77cdbea8a1e57eaa
INFO[2021-05-31T11:36:26.409423284Z] RemoveContainer for "5196b8846626eaad2bfa309478ff1f5ab5731ad78c141b1b77cdbea8a1e57eaa"
INFO[2021-05-31T11:36:26.421623248Z] RemoveContainer for "5196b8846626eaad2bfa309478ff1f5ab5731ad78c141b1b77cdbea8a1e57eaa" returns successfully
I0531 11:36:26.421822  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: fcb78381d2248c96acfb5f1d4f0da188d3fd7db0dd2e1cafbf26f67959908459
INFO[2021-05-31T11:36:26.422768219Z] RemoveContainer for "fcb78381d2248c96acfb5f1d4f0da188d3fd7db0dd2e1cafbf26f67959908459"
INFO[2021-05-31T11:36:26.437816861Z] RemoveContainer for "fcb78381d2248c96acfb5f1d4f0da188d3fd7db0dd2e1cafbf26f67959908459" returns successfully
I0531 11:36:26.437994  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: dfb4378a6838ba4057b39ce35ea286ecd7877d1ef2ebed56407656a9ec2df1d6
INFO[2021-05-31T11:36:26.438796423Z] RemoveContainer for "dfb4378a6838ba4057b39ce35ea286ecd7877d1ef2ebed56407656a9ec2df1d6"
INFO[2021-05-31T11:36:26.450730870Z] RemoveContainer for "dfb4378a6838ba4057b39ce35ea286ecd7877d1ef2ebed56407656a9ec2df1d6" returns successfully
I0531 11:36:26.450933  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 28ab98f2bc95b24782c5ecfe15e14eeab2d3da7d73d42a43a61ab39b9043fc5d
INFO[2021-05-31T11:36:26.452005550Z] RemoveContainer for "28ab98f2bc95b24782c5ecfe15e14eeab2d3da7d73d42a43a61ab39b9043fc5d"
INFO[2021-05-31T11:36:26.466646466Z] RemoveContainer for "28ab98f2bc95b24782c5ecfe15e14eeab2d3da7d73d42a43a61ab39b9043fc5d" returns successfully
I0531 11:36:26.466922  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: abd46405e7365b01a15eb71e3fd97595811086fa89d804ff89baa5925e5c09c0
INFO[2021-05-31T11:36:26.467979550Z] RemoveContainer for "abd46405e7365b01a15eb71e3fd97595811086fa89d804ff89baa5925e5c09c0"
INFO[2021-05-31T11:36:26.481995427Z] RemoveContainer for "abd46405e7365b01a15eb71e3fd97595811086fa89d804ff89baa5925e5c09c0" returns successfully
I0531 11:36:26.482182  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 1831ba963ffe1241448d5e737f863d49570f85e67491e7ea04e1d205b9060b58
INFO[2021-05-31T11:36:26.483058194Z] RemoveContainer for "1831ba963ffe1241448d5e737f863d49570f85e67491e7ea04e1d205b9060b58"
INFO[2021-05-31T11:36:26.494129287Z] RemoveContainer for "1831ba963ffe1241448d5e737f863d49570f85e67491e7ea04e1d205b9060b58" returns successfully
I0531 11:36:26.494380  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 0b52af0a7848f8b2c7d022e3848d93b93921e3d2bf1a655aa6dfb3d26618587b
INFO[2021-05-31T11:36:26.497369690Z] RemoveContainer for "0b52af0a7848f8b2c7d022e3848d93b93921e3d2bf1a655aa6dfb3d26618587b"
INFO[2021-05-31T11:36:26.510036183Z] RemoveContainer for "0b52af0a7848f8b2c7d022e3848d93b93921e3d2bf1a655aa6dfb3d26618587b" returns successfully
I0531 11:36:26.510222  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 7240611c0a2661f07cde3d3d0022e1115e85064397a7f4f8165e11952930c309
INFO[2021-05-31T11:36:26.511058247Z] RemoveContainer for "7240611c0a2661f07cde3d3d0022e1115e85064397a7f4f8165e11952930c309"
INFO[2021-05-31T11:36:26.535742492Z] RemoveContainer for "7240611c0a2661f07cde3d3d0022e1115e85064397a7f4f8165e11952930c309" returns successfully
I0531 11:36:26.535998  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: ff4f7aa2c9a53c3df3cf67ed2f5b76ea9f7dff200b97dfa898f92a3c1b763d3a
INFO[2021-05-31T11:36:26.537074976Z] RemoveContainer for "ff4f7aa2c9a53c3df3cf67ed2f5b76ea9f7dff200b97dfa898f92a3c1b763d3a"
INFO[2021-05-31T11:36:26.556376084Z] RemoveContainer for "ff4f7aa2c9a53c3df3cf67ed2f5b76ea9f7dff200b97dfa898f92a3c1b763d3a" returns successfully
I0531 11:36:26.556514  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 4ad62ff9625721b50946986e0f546a40cf40cfc693582eceb1f1c3cb1f7e3084
INFO[2021-05-31T11:36:26.557411649Z] RemoveContainer for "4ad62ff9625721b50946986e0f546a40cf40cfc693582eceb1f1c3cb1f7e3084"
INFO[2021-05-31T11:36:26.571961960Z] RemoveContainer for "4ad62ff9625721b50946986e0f546a40cf40cfc693582eceb1f1c3cb1f7e3084" returns successfully
I0531 11:36:26.572140  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: fa0fcd4feacc9f4f74f8c39e3f512cd029e47e8e86b6062e9e5f767b2beb2f65
INFO[2021-05-31T11:36:26.573023726Z] RemoveContainer for "fa0fcd4feacc9f4f74f8c39e3f512cd029e47e8e86b6062e9e5f767b2beb2f65"
INFO[2021-05-31T11:36:26.588785113Z] RemoveContainer for "fa0fcd4feacc9f4f74f8c39e3f512cd029e47e8e86b6062e9e5f767b2beb2f65" returns successfully
I0531 11:36:26.588937  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 307f709c72d167bf70b186a526cb562fa4837c790be984f433ee132536b5822c
INFO[2021-05-31T11:36:26.589677969Z] RemoveContainer for "307f709c72d167bf70b186a526cb562fa4837c790be984f433ee132536b5822c"
INFO[2021-05-31T11:36:26.603674945Z] RemoveContainer for "307f709c72d167bf70b186a526cb562fa4837c790be984f433ee132536b5822c" returns successfully
I0531 11:36:26.603846  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 55373d72ab04ce7cd951686d8e8152307aa0b340bb1f58b6d670f868fe7a12bc
INFO[2021-05-31T11:36:26.604698409Z] RemoveContainer for "55373d72ab04ce7cd951686d8e8152307aa0b340bb1f58b6d670f868fe7a12bc"
INFO[2021-05-31T11:36:26.617423406Z] RemoveContainer for "55373d72ab04ce7cd951686d8e8152307aa0b340bb1f58b6d670f868fe7a12bc" returns successfully
I0531 11:36:26.617581  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 572935c64bb6ed9ee4e570548fe267be2faed9cfc9521fbc278b885ffdb4eb5f
INFO[2021-05-31T11:36:26.618499073Z] RemoveContainer for "572935c64bb6ed9ee4e570548fe267be2faed9cfc9521fbc278b885ffdb4eb5f"
INFO[2021-05-31T11:36:26.630669835Z] RemoveContainer for "572935c64bb6ed9ee4e570548fe267be2faed9cfc9521fbc278b885ffdb4eb5f" returns successfully
I0531 11:36:26.630837  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: bef614a8c136075240c887d23cc202965eb8c7b4218501b2459eab892021e302
INFO[2021-05-31T11:36:26.634595181Z] RemoveContainer for "bef614a8c136075240c887d23cc202965eb8c7b4218501b2459eab892021e302"
INFO[2021-05-31T11:36:26.658462075Z] RemoveContainer for "bef614a8c136075240c887d23cc202965eb8c7b4218501b2459eab892021e302" returns successfully
I0531 11:36:26.658843  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 473e345c2545aed47f219e5b646672215d4c5011fe0eba43bf1c72e8b36506f8
INFO[2021-05-31T11:36:26.659812560Z] RemoveContainer for "473e345c2545aed47f219e5b646672215d4c5011fe0eba43bf1c72e8b36506f8"
INFO[2021-05-31T11:36:26.672397948Z] RemoveContainer for "473e345c2545aed47f219e5b646672215d4c5011fe0eba43bf1c72e8b36506f8" returns successfully
I0531 11:36:26.672679  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 7f99f0de5096ddac1c6658063e307ba13a680a76e96685a76f8370728a7f159f
INFO[2021-05-31T11:36:26.673706629Z] RemoveContainer for "7f99f0de5096ddac1c6658063e307ba13a680a76e96685a76f8370728a7f159f"
INFO[2021-05-31T11:36:26.689663828Z] RemoveContainer for "7f99f0de5096ddac1c6658063e307ba13a680a76e96685a76f8370728a7f159f" returns successfully
I0531 11:36:26.689852  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: a95a719aa4b0d17316076cfe39f3fe7adebc8b99db1e5ab07f70b6df30f4618d
INFO[2021-05-31T11:36:26.690875904Z] RemoveContainer for "a95a719aa4b0d17316076cfe39f3fe7adebc8b99db1e5ab07f70b6df30f4618d"
INFO[2021-05-31T11:36:26.712621066Z] RemoveContainer for "a95a719aa4b0d17316076cfe39f3fe7adebc8b99db1e5ab07f70b6df30f4618d" returns successfully
I0531 11:36:26.712903  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 693a32131387a0d8303d6df80693d3de6a9019e9a66edd3d203187d88edc06f2
INFO[2021-05-31T11:36:26.713860043Z] RemoveContainer for "693a32131387a0d8303d6df80693d3de6a9019e9a66edd3d203187d88edc06f2"
INFO[2021-05-31T11:36:26.890634010Z] RemoveContainer for "693a32131387a0d8303d6df80693d3de6a9019e9a66edd3d203187d88edc06f2" returns successfully
I0531 11:36:26.890906  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 00175952fea534a928e53fbc04af5e0fa86ce39e5c0926453dd74a5560f1bfc0
INFO[2021-05-31T11:36:26.891902490Z] RemoveContainer for "00175952fea534a928e53fbc04af5e0fa86ce39e5c0926453dd74a5560f1bfc0"
INFO[2021-05-31T11:36:26.907015236Z] RemoveContainer for "00175952fea534a928e53fbc04af5e0fa86ce39e5c0926453dd74a5560f1bfc0" returns successfully
I0531 11:36:26.907253  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: cd037ddca1cf79d703a16c6046deaf4d639247b736e9a29b07688ce931e886dd
INFO[2021-05-31T11:36:26.908118005Z] RemoveContainer for "cd037ddca1cf79d703a16c6046deaf4d639247b736e9a29b07688ce931e886dd"
INFO[2021-05-31T11:36:26.921214825Z] RemoveContainer for "cd037ddca1cf79d703a16c6046deaf4d639247b736e9a29b07688ce931e886dd" returns successfully
I0531 11:36:26.921383  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: fcddf6b37679ac40690c76cf8dea0f4b54d37511a8c4b3df1ed69ed02abd8484
INFO[2021-05-31T11:36:26.922370097Z] RemoveContainer for "fcddf6b37679ac40690c76cf8dea0f4b54d37511a8c4b3df1ed69ed02abd8484"
INFO[2021-05-31T11:36:26.938517708Z] RemoveContainer for "fcddf6b37679ac40690c76cf8dea0f4b54d37511a8c4b3df1ed69ed02abd8484" returns successfully
I0531 11:36:26.938809  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: e0552d3ed6e4ae17d4a054e8575542d685cb0193b96aa8db0d4a6ae4ca8db652
INFO[2021-05-31T11:36:26.939730484Z] RemoveContainer for "e0552d3ed6e4ae17d4a054e8575542d685cb0193b96aa8db0d4a6ae4ca8db652"
INFO[2021-05-31T11:36:26.953959575Z] RemoveContainer for "e0552d3ed6e4ae17d4a054e8575542d685cb0193b96aa8db0d4a6ae4ca8db652" returns successfully
I0531 11:36:26.954239  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: ebcc58d0a3b98ba775f22ac16b396e58e625e98f0995ff601020a5116a31a3c1
INFO[2021-05-31T11:36:26.955829392Z] RemoveContainer for "ebcc58d0a3b98ba775f22ac16b396e58e625e98f0995ff601020a5116a31a3c1"
INFO[2021-05-31T11:36:26.970272196Z] RemoveContainer for "ebcc58d0a3b98ba775f22ac16b396e58e625e98f0995ff601020a5116a31a3c1" returns successfully
I0531 11:36:26.970583  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 4691d432dcfc36587d5ad25837428c9c2369a3c6eef9971c90625a0981bb9b3a
INFO[2021-05-31T11:36:26.971455670Z] RemoveContainer for "4691d432dcfc36587d5ad25837428c9c2369a3c6eef9971c90625a0981bb9b3a"
INFO[2021-05-31T11:36:26.984376479Z] RemoveContainer for "4691d432dcfc36587d5ad25837428c9c2369a3c6eef9971c90625a0981bb9b3a" returns successfully
I0531 11:36:26.984661  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 607731d9394e26bd44a3b83c714203e7b87695398f0b26977ac661154f2f1a11
INFO[2021-05-31T11:36:26.985972979Z] RemoveContainer for "607731d9394e26bd44a3b83c714203e7b87695398f0b26977ac661154f2f1a11"
INFO[2021-05-31T11:36:26.999916952Z] RemoveContainer for "607731d9394e26bd44a3b83c714203e7b87695398f0b26977ac661154f2f1a11" returns successfully
I0531 11:36:27.000196  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: a45842958375fbb490dcbb1b02df255224280d3f6c7a77794256dd0ab043493a
INFO[2021-05-31T11:36:27.001135828Z] RemoveContainer for "a45842958375fbb490dcbb1b02df255224280d3f6c7a77794256dd0ab043493a"
INFO[2021-05-31T11:36:27.034966446Z] RemoveContainer for "a45842958375fbb490dcbb1b02df255224280d3f6c7a77794256dd0ab043493a" returns successfully
I0531 11:36:27.035387  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: 4d7b1966552f191aeba7d827cf4fb02a2811525ec67105111ce3c43f34e97853
INFO[2021-05-31T11:36:27.036986773Z] RemoveContainer for "4d7b1966552f191aeba7d827cf4fb02a2811525ec67105111ce3c43f34e97853"
INFO[2021-05-31T11:36:27.060668956Z] RemoveContainer for "4d7b1966552f191aeba7d827cf4fb02a2811525ec67105111ce3c43f34e97853" returns successfully
I0531 11:36:27.060962  236858 scope.go:111] [topologymanager] RemoveContainer - Container ID: b10aa220c3246c0b8317a22fb79d8d1676ac88dd135d6521880b0e0756544f0d
INFO[2021-05-31T11:36:27.062204352Z] RemoveContainer for "b10aa220c3246c0b8317a22fb79d8d1676ac88dd135d6521880b0e0756544f0d"
INFO[2021-05-31T11:36:27.076752762Z] RemoveContainer for "b10aa220c3246c0b8317a22fb79d8d1676ac88dd135d6521880b0e0756544f0d" returns successfully
INFO[2021-05-31T11:36:27.078405566Z] StopPodSandbox for "a7bfead0eec2fc630fa99ca6e28e1d893d7c28a47d96fec8385487934a6ff18c"
INFO[2021-05-31T11:36:27.078523173Z] TearDown network for sandbox "a7bfead0eec2fc630fa99ca6e28e1d893d7c28a47d96fec8385487934a6ff18c" successfully
INFO[2021-05-31T11:36:27.078547075Z] StopPodSandbox for "a7bfead0eec2fc630fa99ca6e28e1d893d7c28a47d96fec8385487934a6ff18c" returns successfully
INFO[2021-05-31T11:36:27.078855394Z] RemovePodSandbox for "a7bfead0eec2fc630fa99ca6e28e1d893d7c28a47d96fec8385487934a6ff18c"
INFO[2021-05-31T11:36:27.093443007Z] RemovePodSandbox "a7bfead0eec2fc630fa99ca6e28e1d893d7c28a47d96fec8385487934a6ff18c" returns successfully
INFO[2021-05-31T11:36:27.093896636Z] StopPodSandbox for "3236f91258244df5401386921d30f79b459fb42e2e573df2f9b26ac6646b12cd"
ERRO[2021-05-31T11:36:27.145123743Z] StopPodSandbox for "3236f91258244df5401386921d30f79b459fb42e2e573df2f9b26ac6646b12cd" failed  error="failed to destroy network for sandbox \"3236f91258244df5401386921d30f79b459fb42e2e573df2f9b26ac6646b12cd\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:27.145482  236858 remote_runtime.go:143] StopPodSandbox "3236f91258244df5401386921d30f79b459fb42e2e573df2f9b26ac6646b12cd" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "3236f91258244df5401386921d30f79b459fb42e2e573df2f9b26ac6646b12cd": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:27.145534  236858 kuberuntime_gc.go:176] Failed to stop sandbox "3236f91258244df5401386921d30f79b459fb42e2e573df2f9b26ac6646b12cd" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "3236f91258244df5401386921d30f79b459fb42e2e573df2f9b26ac6646b12cd": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:27.145762383Z] StopPodSandbox for "ad9d7f314ca4d3b3ed19c04e52b4ab9efacc24659668765f3ab9e13644c9a4ba"
ERRO[2021-05-31T11:36:27.195616304Z] StopPodSandbox for "ad9d7f314ca4d3b3ed19c04e52b4ab9efacc24659668765f3ab9e13644c9a4ba" failed  error="failed to destroy network for sandbox \"ad9d7f314ca4d3b3ed19c04e52b4ab9efacc24659668765f3ab9e13644c9a4ba\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:27.195924  236858 remote_runtime.go:143] StopPodSandbox "ad9d7f314ca4d3b3ed19c04e52b4ab9efacc24659668765f3ab9e13644c9a4ba" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "ad9d7f314ca4d3b3ed19c04e52b4ab9efacc24659668765f3ab9e13644c9a4ba": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:27.195966  236858 kuberuntime_gc.go:176] Failed to stop sandbox "ad9d7f314ca4d3b3ed19c04e52b4ab9efacc24659668765f3ab9e13644c9a4ba" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "ad9d7f314ca4d3b3ed19c04e52b4ab9efacc24659668765f3ab9e13644c9a4ba": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:27.196180239Z] StopPodSandbox for "ac256db2fd8193de21cdfb3d5ea7ef881876a7f20c590a9816d50cc2d494e7eb"
INFO[2021-05-31T11:36:27.196284646Z] TearDown network for sandbox "ac256db2fd8193de21cdfb3d5ea7ef881876a7f20c590a9816d50cc2d494e7eb" successfully
INFO[2021-05-31T11:36:27.196298047Z] StopPodSandbox for "ac256db2fd8193de21cdfb3d5ea7ef881876a7f20c590a9816d50cc2d494e7eb" returns successfully
INFO[2021-05-31T11:36:27.196549662Z] RemovePodSandbox for "ac256db2fd8193de21cdfb3d5ea7ef881876a7f20c590a9816d50cc2d494e7eb"
INFO[2021-05-31T11:36:27.210967965Z] RemovePodSandbox "ac256db2fd8193de21cdfb3d5ea7ef881876a7f20c590a9816d50cc2d494e7eb" returns successfully
INFO[2021-05-31T11:36:27.211493198Z] StopPodSandbox for "bf8bb075f1119e600352ea3d1be12b899016c5cb5706bc7b2d406d77095f3c8b"
ERRO[2021-05-31T11:36:27.257619486Z] StopPodSandbox for "bf8bb075f1119e600352ea3d1be12b899016c5cb5706bc7b2d406d77095f3c8b" failed  error="failed to destroy network for sandbox \"bf8bb075f1119e600352ea3d1be12b899016c5cb5706bc7b2d406d77095f3c8b\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:27.257876  236858 remote_runtime.go:143] StopPodSandbox "bf8bb075f1119e600352ea3d1be12b899016c5cb5706bc7b2d406d77095f3c8b" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "bf8bb075f1119e600352ea3d1be12b899016c5cb5706bc7b2d406d77095f3c8b": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:27.257913  236858 kuberuntime_gc.go:176] Failed to stop sandbox "bf8bb075f1119e600352ea3d1be12b899016c5cb5706bc7b2d406d77095f3c8b" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "bf8bb075f1119e600352ea3d1be12b899016c5cb5706bc7b2d406d77095f3c8b": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:27.258135318Z] StopPodSandbox for "dceb9a712dc8e63224ff37a893615edc49d7e0809f2663ffca5b51a1b8f6b7d9"
INFO[2021-05-31T11:36:27.258263426Z] TearDown network for sandbox "dceb9a712dc8e63224ff37a893615edc49d7e0809f2663ffca5b51a1b8f6b7d9" successfully
INFO[2021-05-31T11:36:27.258280327Z] StopPodSandbox for "dceb9a712dc8e63224ff37a893615edc49d7e0809f2663ffca5b51a1b8f6b7d9" returns successfully
INFO[2021-05-31T11:36:27.258610348Z] RemovePodSandbox for "dceb9a712dc8e63224ff37a893615edc49d7e0809f2663ffca5b51a1b8f6b7d9"
INFO[2021-05-31T11:36:27.274270828Z] RemovePodSandbox "dceb9a712dc8e63224ff37a893615edc49d7e0809f2663ffca5b51a1b8f6b7d9" returns successfully
INFO[2021-05-31T11:36:27.274783260Z] StopPodSandbox for "08dcbc707213a202f1bfc9a47bbbd5742726a38390e96e5cf52cddccd05fc1dc"
INFO[2021-05-31T11:36:27.274889867Z] TearDown network for sandbox "08dcbc707213a202f1bfc9a47bbbd5742726a38390e96e5cf52cddccd05fc1dc" successfully
INFO[2021-05-31T11:36:27.274904168Z] StopPodSandbox for "08dcbc707213a202f1bfc9a47bbbd5742726a38390e96e5cf52cddccd05fc1dc" returns successfully
INFO[2021-05-31T11:36:27.275166484Z] RemovePodSandbox for "08dcbc707213a202f1bfc9a47bbbd5742726a38390e96e5cf52cddccd05fc1dc"
INFO[2021-05-31T11:36:27.289706695Z] RemovePodSandbox "08dcbc707213a202f1bfc9a47bbbd5742726a38390e96e5cf52cddccd05fc1dc" returns successfully
INFO[2021-05-31T11:36:27.289975911Z] StopPodSandbox for "69fec1b6339031dc3233048afa88191db808156968a754e109c19c2189901b5a"
ERRO[2021-05-31T11:36:27.337459984Z] StopPodSandbox for "69fec1b6339031dc3233048afa88191db808156968a754e109c19c2189901b5a" failed  error="failed to destroy network for sandbox \"69fec1b6339031dc3233048afa88191db808156968a754e109c19c2189901b5a\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:27.337694  236858 remote_runtime.go:143] StopPodSandbox "69fec1b6339031dc3233048afa88191db808156968a754e109c19c2189901b5a" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "69fec1b6339031dc3233048afa88191db808156968a754e109c19c2189901b5a": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:27.337737  236858 kuberuntime_gc.go:176] Failed to stop sandbox "69fec1b6339031dc3233048afa88191db808156968a754e109c19c2189901b5a" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "69fec1b6339031dc3233048afa88191db808156968a754e109c19c2189901b5a": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:27.337996518Z] StopPodSandbox for "41c5bc1acf3f41342d5b140009a18a4435782b38e177a1153c42bceacfba1ed7"
INFO[2021-05-31T11:36:27.338117825Z] TearDown network for sandbox "41c5bc1acf3f41342d5b140009a18a4435782b38e177a1153c42bceacfba1ed7" successfully
INFO[2021-05-31T11:36:27.338133226Z] StopPodSandbox for "41c5bc1acf3f41342d5b140009a18a4435782b38e177a1153c42bceacfba1ed7" returns successfully
INFO[2021-05-31T11:36:27.338387542Z] RemovePodSandbox for "41c5bc1acf3f41342d5b140009a18a4435782b38e177a1153c42bceacfba1ed7"
INFO[2021-05-31T11:36:27.365898365Z] RemovePodSandbox "41c5bc1acf3f41342d5b140009a18a4435782b38e177a1153c42bceacfba1ed7" returns successfully
INFO[2021-05-31T11:36:27.366330692Z] StopPodSandbox for "2cd5b03f21adbac5af8ee9067f717bebf6a63563aaf7d30367c4905d64cc423f"
ERRO[2021-05-31T11:36:27.412833403Z] StopPodSandbox for "2cd5b03f21adbac5af8ee9067f717bebf6a63563aaf7d30367c4905d64cc423f" failed  error="failed to destroy network for sandbox \"2cd5b03f21adbac5af8ee9067f717bebf6a63563aaf7d30367c4905d64cc423f\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:27.413066  236858 remote_runtime.go:143] StopPodSandbox "2cd5b03f21adbac5af8ee9067f717bebf6a63563aaf7d30367c4905d64cc423f" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "2cd5b03f21adbac5af8ee9067f717bebf6a63563aaf7d30367c4905d64cc423f": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:27.413100  236858 kuberuntime_gc.go:176] Failed to stop sandbox "2cd5b03f21adbac5af8ee9067f717bebf6a63563aaf7d30367c4905d64cc423f" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "2cd5b03f21adbac5af8ee9067f717bebf6a63563aaf7d30367c4905d64cc423f": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:27.413297432Z] StopPodSandbox for "b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48"
ERRO[2021-05-31T11:36:27.463059848Z] StopPodSandbox for "b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48" failed  error="failed to destroy network for sandbox \"b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:27.463342  236858 remote_runtime.go:143] StopPodSandbox "b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:27.463379  236858 kuberuntime_gc.go:176] Failed to stop sandbox "b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "b523b8a554db426396ae43e1848da1a49bc8c8fd9e6051bfbe7da138f93bbb48": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:27.463588081Z] StopPodSandbox for "109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7"
ERRO[2021-05-31T11:36:27.515075804Z] StopPodSandbox for "109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7" failed  error="failed to destroy network for sandbox \"109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:27.515333  236858 remote_runtime.go:143] StopPodSandbox "109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:27.515377  236858 kuberuntime_gc.go:176] Failed to stop sandbox "109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "109e4981a9e4fd94d15fe8abf19c2d84686d8eec51a87bf0b9353ab7f1a4b5e7": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:27.515576235Z] StopPodSandbox for "5c41f3a7c446e46ed904398c49abf4b412a105ecb6ce3856db4a7ac1beb4c57e"
INFO[2021-05-31T11:36:27.515679242Z] TearDown network for sandbox "5c41f3a7c446e46ed904398c49abf4b412a105ecb6ce3856db4a7ac1beb4c57e" successfully
INFO[2021-05-31T11:36:27.515694043Z] StopPodSandbox for "5c41f3a7c446e46ed904398c49abf4b412a105ecb6ce3856db4a7ac1beb4c57e" returns successfully
INFO[2021-05-31T11:36:27.515972360Z] RemovePodSandbox for "5c41f3a7c446e46ed904398c49abf4b412a105ecb6ce3856db4a7ac1beb4c57e"
INFO[2021-05-31T11:36:27.531498632Z] RemovePodSandbox "5c41f3a7c446e46ed904398c49abf4b412a105ecb6ce3856db4a7ac1beb4c57e" returns successfully
INFO[2021-05-31T11:36:27.531823853Z] StopPodSandbox for "6414e3c3a9d87f08fbb4b5fa1dc2f6bb2dbb0aba79218c139560cb06f0c4b88e"
ERRO[2021-05-31T11:36:27.584638459Z] StopPodSandbox for "6414e3c3a9d87f08fbb4b5fa1dc2f6bb2dbb0aba79218c139560cb06f0c4b88e" failed  error="failed to destroy network for sandbox \"6414e3c3a9d87f08fbb4b5fa1dc2f6bb2dbb0aba79218c139560cb06f0c4b88e\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:27.584820  236858 remote_runtime.go:143] StopPodSandbox "6414e3c3a9d87f08fbb4b5fa1dc2f6bb2dbb0aba79218c139560cb06f0c4b88e" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "6414e3c3a9d87f08fbb4b5fa1dc2f6bb2dbb0aba79218c139560cb06f0c4b88e": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:27.584847  236858 kuberuntime_gc.go:176] Failed to stop sandbox "6414e3c3a9d87f08fbb4b5fa1dc2f6bb2dbb0aba79218c139560cb06f0c4b88e" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "6414e3c3a9d87f08fbb4b5fa1dc2f6bb2dbb0aba79218c139560cb06f0c4b88e": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:27.584991681Z] StopPodSandbox for "b8b3446d73a12d2dd76f254b3e3c8b7275e188b1cac00105f3a9a4a071a4d10a"
ERRO[2021-05-31T11:36:27.632729770Z] StopPodSandbox for "b8b3446d73a12d2dd76f254b3e3c8b7275e188b1cac00105f3a9a4a071a4d10a" failed  error="failed to destroy network for sandbox \"b8b3446d73a12d2dd76f254b3e3c8b7275e188b1cac00105f3a9a4a071a4d10a\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:27.632970  236858 remote_runtime.go:143] StopPodSandbox "b8b3446d73a12d2dd76f254b3e3c8b7275e188b1cac00105f3a9a4a071a4d10a" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "b8b3446d73a12d2dd76f254b3e3c8b7275e188b1cac00105f3a9a4a071a4d10a": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:27.633007  236858 kuberuntime_gc.go:176] Failed to stop sandbox "b8b3446d73a12d2dd76f254b3e3c8b7275e188b1cac00105f3a9a4a071a4d10a" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "b8b3446d73a12d2dd76f254b3e3c8b7275e188b1cac00105f3a9a4a071a4d10a": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:27.633236402Z] StopPodSandbox for "db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97"
ERRO[2021-05-31T11:36:27.678536238Z] StopPodSandbox for "db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97" failed  error="failed to destroy network for sandbox \"db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:27.678802  236858 remote_runtime.go:143] StopPodSandbox "db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:27.678838  236858 kuberuntime_gc.go:176] Failed to stop sandbox "db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "db1a4cc1dda1c63b9f096bd7eeab84acfe925289d91c199e440c687826485d97": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:27.679023768Z] StopPodSandbox for "6c8528f9b0babcb539f8aacea4371feb25a773eee2bac8a1c1a41249ec727d53"
ERRO[2021-05-31T11:36:27.725915604Z] StopPodSandbox for "6c8528f9b0babcb539f8aacea4371feb25a773eee2bac8a1c1a41249ec727d53" failed  error="failed to destroy network for sandbox \"6c8528f9b0babcb539f8aacea4371feb25a773eee2bac8a1c1a41249ec727d53\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:27.726234  236858 remote_runtime.go:143] StopPodSandbox "6c8528f9b0babcb539f8aacea4371feb25a773eee2bac8a1c1a41249ec727d53" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "6c8528f9b0babcb539f8aacea4371feb25a773eee2bac8a1c1a41249ec727d53": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:27.726270  236858 kuberuntime_gc.go:176] Failed to stop sandbox "6c8528f9b0babcb539f8aacea4371feb25a773eee2bac8a1c1a41249ec727d53" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "6c8528f9b0babcb539f8aacea4371feb25a773eee2bac8a1c1a41249ec727d53": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:27.726471539Z] StopPodSandbox for "4572d3396b52957ae5e832ede8f6041fc706889fd258e2783a9075cb3700a075"
ERRO[2021-05-31T11:36:27.779869382Z] StopPodSandbox for "4572d3396b52957ae5e832ede8f6041fc706889fd258e2783a9075cb3700a075" failed  error="failed to destroy network for sandbox \"4572d3396b52957ae5e832ede8f6041fc706889fd258e2783a9075cb3700a075\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:27.780145  236858 remote_runtime.go:143] StopPodSandbox "4572d3396b52957ae5e832ede8f6041fc706889fd258e2783a9075cb3700a075" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "4572d3396b52957ae5e832ede8f6041fc706889fd258e2783a9075cb3700a075": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:27.780180  236858 kuberuntime_gc.go:176] Failed to stop sandbox "4572d3396b52957ae5e832ede8f6041fc706889fd258e2783a9075cb3700a075" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "4572d3396b52957ae5e832ede8f6041fc706889fd258e2783a9075cb3700a075": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:27.780408615Z] StopPodSandbox for "407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749"
ERRO[2021-05-31T11:36:27.830520453Z] StopPodSandbox for "407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749" failed  error="failed to destroy network for sandbox \"407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:27.830750  236858 remote_runtime.go:143] StopPodSandbox "407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:27.830783  236858 kuberuntime_gc.go:176] Failed to stop sandbox "407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "407aa55effb55d461eda2fc1fa92edc53e6e4d125c905f91c9f870eed34a5749": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:27.830981882Z] StopPodSandbox for "c7131a99d594d9de61450820e4bb74080d1916a4623a7561d9981d6127986bd7"
ERRO[2021-05-31T11:36:27.876111007Z] StopPodSandbox for "c7131a99d594d9de61450820e4bb74080d1916a4623a7561d9981d6127986bd7" failed  error="failed to destroy network for sandbox \"c7131a99d594d9de61450820e4bb74080d1916a4623a7561d9981d6127986bd7\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:27.876356  236858 remote_runtime.go:143] StopPodSandbox "c7131a99d594d9de61450820e4bb74080d1916a4623a7561d9981d6127986bd7" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "c7131a99d594d9de61450820e4bb74080d1916a4623a7561d9981d6127986bd7": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:27.876444  236858 kuberuntime_gc.go:176] Failed to stop sandbox "c7131a99d594d9de61450820e4bb74080d1916a4623a7561d9981d6127986bd7" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "c7131a99d594d9de61450820e4bb74080d1916a4623a7561d9981d6127986bd7": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:27.876705644Z] StopPodSandbox for "600dccf452c5ce4361371721a73d39a537acd01f2f2d733d078eaad99e250d1d"
ERRO[2021-05-31T11:36:27.922836432Z] StopPodSandbox for "600dccf452c5ce4361371721a73d39a537acd01f2f2d733d078eaad99e250d1d" failed  error="failed to destroy network for sandbox \"600dccf452c5ce4361371721a73d39a537acd01f2f2d733d078eaad99e250d1d\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:27.923097  236858 remote_runtime.go:143] StopPodSandbox "600dccf452c5ce4361371721a73d39a537acd01f2f2d733d078eaad99e250d1d" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "600dccf452c5ce4361371721a73d39a537acd01f2f2d733d078eaad99e250d1d": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:27.923138  236858 kuberuntime_gc.go:176] Failed to stop sandbox "600dccf452c5ce4361371721a73d39a537acd01f2f2d733d078eaad99e250d1d" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "600dccf452c5ce4361371721a73d39a537acd01f2f2d733d078eaad99e250d1d": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:27.923490373Z] StopPodSandbox for "b97deb11b14184ded631c24ad2c966dd806651354269091802c1570090874899"
ERRO[2021-05-31T11:36:27.970229399Z] StopPodSandbox for "b97deb11b14184ded631c24ad2c966dd806651354269091802c1570090874899" failed  error="failed to destroy network for sandbox \"b97deb11b14184ded631c24ad2c966dd806651354269091802c1570090874899\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:27.970470  236858 remote_runtime.go:143] StopPodSandbox "b97deb11b14184ded631c24ad2c966dd806651354269091802c1570090874899" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "b97deb11b14184ded631c24ad2c966dd806651354269091802c1570090874899": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:27.970507  236858 kuberuntime_gc.go:176] Failed to stop sandbox "b97deb11b14184ded631c24ad2c966dd806651354269091802c1570090874899" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "b97deb11b14184ded631c24ad2c966dd806651354269091802c1570090874899": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:27.970719830Z] StopPodSandbox for "15dbe828817c7a1a794ed2ecdf34fd73569e56a0d9ac25908fe8e3ef60bd0804"
ERRO[2021-05-31T11:36:28.017435555Z] StopPodSandbox for "15dbe828817c7a1a794ed2ecdf34fd73569e56a0d9ac25908fe8e3ef60bd0804" failed  error="failed to destroy network for sandbox \"15dbe828817c7a1a794ed2ecdf34fd73569e56a0d9ac25908fe8e3ef60bd0804\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:28.017725  236858 remote_runtime.go:143] StopPodSandbox "15dbe828817c7a1a794ed2ecdf34fd73569e56a0d9ac25908fe8e3ef60bd0804" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "15dbe828817c7a1a794ed2ecdf34fd73569e56a0d9ac25908fe8e3ef60bd0804": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:28.017760  236858 kuberuntime_gc.go:176] Failed to stop sandbox "15dbe828817c7a1a794ed2ecdf34fd73569e56a0d9ac25908fe8e3ef60bd0804" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "15dbe828817c7a1a794ed2ecdf34fd73569e56a0d9ac25908fe8e3ef60bd0804": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:28.017964188Z] StopPodSandbox for "f13274e2c989d8f7f0f0f64c9489416b8c21e08d6250734a25fbf236ce6f5f46"
ERRO[2021-05-31T11:36:28.062587181Z] StopPodSandbox for "f13274e2c989d8f7f0f0f64c9489416b8c21e08d6250734a25fbf236ce6f5f46" failed  error="failed to destroy network for sandbox \"f13274e2c989d8f7f0f0f64c9489416b8c21e08d6250734a25fbf236ce6f5f46\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:28.062825  236858 remote_runtime.go:143] StopPodSandbox "f13274e2c989d8f7f0f0f64c9489416b8c21e08d6250734a25fbf236ce6f5f46" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "f13274e2c989d8f7f0f0f64c9489416b8c21e08d6250734a25fbf236ce6f5f46": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:28.062857  236858 kuberuntime_gc.go:176] Failed to stop sandbox "f13274e2c989d8f7f0f0f64c9489416b8c21e08d6250734a25fbf236ce6f5f46" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "f13274e2c989d8f7f0f0f64c9489416b8c21e08d6250734a25fbf236ce6f5f46": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:28.063057811Z] StopPodSandbox for "6680d04a2929b55bbddbcef4e4aec959f97f0c167738a5f9a03c64409b1d2984"
INFO[2021-05-31T11:36:28.063164818Z] TearDown network for sandbox "6680d04a2929b55bbddbcef4e4aec959f97f0c167738a5f9a03c64409b1d2984" successfully
INFO[2021-05-31T11:36:28.063178718Z] StopPodSandbox for "6680d04a2929b55bbddbcef4e4aec959f97f0c167738a5f9a03c64409b1d2984" returns successfully
INFO[2021-05-31T11:36:28.063454736Z] RemovePodSandbox for "6680d04a2929b55bbddbcef4e4aec959f97f0c167738a5f9a03c64409b1d2984"
INFO[2021-05-31T11:36:28.076863275Z] RemovePodSandbox "6680d04a2929b55bbddbcef4e4aec959f97f0c167738a5f9a03c64409b1d2984" returns successfully
INFO[2021-05-31T11:36:28.077226598Z] StopPodSandbox for "b0e456e1626c305d909135080b576f7b65daecbf608c3ae1f47265be6068797d"
ERRO[2021-05-31T11:36:28.131665606Z] StopPodSandbox for "b0e456e1626c305d909135080b576f7b65daecbf608c3ae1f47265be6068797d" failed  error="failed to destroy network for sandbox \"b0e456e1626c305d909135080b576f7b65daecbf608c3ae1f47265be6068797d\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:28.131918  236858 remote_runtime.go:143] StopPodSandbox "b0e456e1626c305d909135080b576f7b65daecbf608c3ae1f47265be6068797d" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "b0e456e1626c305d909135080b576f7b65daecbf608c3ae1f47265be6068797d": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:28.131957  236858 kuberuntime_gc.go:176] Failed to stop sandbox "b0e456e1626c305d909135080b576f7b65daecbf608c3ae1f47265be6068797d" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "b0e456e1626c305d909135080b576f7b65daecbf608c3ae1f47265be6068797d": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:28.132168138Z] StopPodSandbox for "077488a0b3aadb9b64c2bbbf2bd74eb939d500ac9598ebafa08b93640df69686"
INFO[2021-05-31T11:36:28.132278144Z] TearDown network for sandbox "077488a0b3aadb9b64c2bbbf2bd74eb939d500ac9598ebafa08b93640df69686" successfully
INFO[2021-05-31T11:36:28.132292745Z] StopPodSandbox for "077488a0b3aadb9b64c2bbbf2bd74eb939d500ac9598ebafa08b93640df69686" returns successfully
INFO[2021-05-31T11:36:28.132590964Z] RemovePodSandbox for "077488a0b3aadb9b64c2bbbf2bd74eb939d500ac9598ebafa08b93640df69686"
INFO[2021-05-31T11:36:28.153380666Z] RemovePodSandbox "077488a0b3aadb9b64c2bbbf2bd74eb939d500ac9598ebafa08b93640df69686" returns successfully
INFO[2021-05-31T11:36:28.153690385Z] StopPodSandbox for "0b1a2e7b98b8a529a9bfc4d32182299d790a5c9c4f0c3df1b71122a7c6cf3042"
ERRO[2021-05-31T11:36:28.198998121Z] StopPodSandbox for "0b1a2e7b98b8a529a9bfc4d32182299d790a5c9c4f0c3df1b71122a7c6cf3042" failed  error="failed to destroy network for sandbox \"0b1a2e7b98b8a529a9bfc4d32182299d790a5c9c4f0c3df1b71122a7c6cf3042\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:28.199248  236858 remote_runtime.go:143] StopPodSandbox "0b1a2e7b98b8a529a9bfc4d32182299d790a5c9c4f0c3df1b71122a7c6cf3042" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "0b1a2e7b98b8a529a9bfc4d32182299d790a5c9c4f0c3df1b71122a7c6cf3042": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:28.199283  236858 kuberuntime_gc.go:176] Failed to stop sandbox "0b1a2e7b98b8a529a9bfc4d32182299d790a5c9c4f0c3df1b71122a7c6cf3042" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "0b1a2e7b98b8a529a9bfc4d32182299d790a5c9c4f0c3df1b71122a7c6cf3042": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:28.199458350Z] StopPodSandbox for "9b79f57f2efe26b48b58d3077be38cec15456bd081d3f5f82d61048f2d06d814"
INFO[2021-05-31T11:36:28.199583858Z] TearDown network for sandbox "9b79f57f2efe26b48b58d3077be38cec15456bd081d3f5f82d61048f2d06d814" successfully
INFO[2021-05-31T11:36:28.199598759Z] StopPodSandbox for "9b79f57f2efe26b48b58d3077be38cec15456bd081d3f5f82d61048f2d06d814" returns successfully
INFO[2021-05-31T11:36:28.199849675Z] RemovePodSandbox for "9b79f57f2efe26b48b58d3077be38cec15456bd081d3f5f82d61048f2d06d814"
INFO[2021-05-31T11:36:28.216264702Z] RemovePodSandbox "9b79f57f2efe26b48b58d3077be38cec15456bd081d3f5f82d61048f2d06d814" returns successfully
INFO[2021-05-31T11:36:28.216596423Z] StopPodSandbox for "dc7672cfde39f812fc8f29c3906cf998d197e62862efb74a37742c0abb5a8684"
ERRO[2021-05-31T11:36:28.269642344Z] StopPodSandbox for "dc7672cfde39f812fc8f29c3906cf998d197e62862efb74a37742c0abb5a8684" failed  error="failed to destroy network for sandbox \"dc7672cfde39f812fc8f29c3906cf998d197e62862efb74a37742c0abb5a8684\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:28.270006  236858 remote_runtime.go:143] StopPodSandbox "dc7672cfde39f812fc8f29c3906cf998d197e62862efb74a37742c0abb5a8684" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "dc7672cfde39f812fc8f29c3906cf998d197e62862efb74a37742c0abb5a8684": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:28.270125  236858 kuberuntime_gc.go:176] Failed to stop sandbox "dc7672cfde39f812fc8f29c3906cf998d197e62862efb74a37742c0abb5a8684" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "dc7672cfde39f812fc8f29c3906cf998d197e62862efb74a37742c0abb5a8684": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[2021-05-31T11:36:28.270454595Z] StopPodSandbox for "e1899ad16d337576f1ffd41beb2c464450daf90a50d2138263b0485cfdef8860"
INFO[2021-05-31T11:36:28.270562102Z] TearDown network for sandbox "e1899ad16d337576f1ffd41beb2c464450daf90a50d2138263b0485cfdef8860" successfully
INFO[2021-05-31T11:36:28.270577003Z] StopPodSandbox for "e1899ad16d337576f1ffd41beb2c464450daf90a50d2138263b0485cfdef8860" returns successfully
INFO[2021-05-31T11:36:28.270928325Z] RemovePodSandbox for "e1899ad16d337576f1ffd41beb2c464450daf90a50d2138263b0485cfdef8860"
INFO[2021-05-31T11:36:28.285423532Z] RemovePodSandbox "e1899ad16d337576f1ffd41beb2c464450daf90a50d2138263b0485cfdef8860" returns successfully
INFO[2021-05-31T11:36:28.285765654Z] StopPodSandbox for "de39add8ab0c0df611e1cb366f26227313dda568ebe82871a1f5ca387c49d8f1"
ERRO[2021-05-31T11:36:28.335640776Z] StopPodSandbox for "de39add8ab0c0df611e1cb366f26227313dda568ebe82871a1f5ca387c49d8f1" failed  error="failed to destroy network for sandbox \"de39add8ab0c0df611e1cb366f26227313dda568ebe82871a1f5ca387c49d8f1\": error getting ClusterInformation: Get \"https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default\": dial tcp 10.43.0.1:443: connect: connection refused"
E0531 11:36:28.335852  236858 remote_runtime.go:143] StopPodSandbox "de39add8ab0c0df611e1cb366f26227313dda568ebe82871a1f5ca387c49d8f1" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "de39add8ab0c0df611e1cb366f26227313dda568ebe82871a1f5ca387c49d8f1": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
E0531 11:36:28.335883  236858 kuberuntime_gc.go:176] Failed to stop sandbox "de39add8ab0c0df611e1cb366f26227313dda568ebe82871a1f5ca387c49d8f1" before removing: rpc error: code = Unknown desc = failed to destroy network for sandbox "de39add8ab0c0df611e1cb366f26227313dda568ebe82871a1f5ca387c49d8f1": error getting ClusterInformation: Get "https://[10.43.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp 10.43.0.1:443: connect: connection refused
INFO[0070] Static pod cleanup completed successfully
INFO[0070] Starting rke2 v1.20.7+rke2r2 (ea6514775f816b0b4972f832de7d98eba8dff55c)
INFO[0070] Managed etcd cluster bootstrap already complete and initialized
INFO[0070] Running kube-apiserver --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=https://kubernetes.default.svc.cluster.local,rke2 --authorization-mode=Node,RBAC --bind-address=0.0.0.0 --cert-dir=/var/lib/rancher/rke2/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/rke2/server/tls/client-ca.crt --enable-admission-plugins=NodeRestriction,PodSecurityPolicy --encryption-provider-config=/var/lib/rancher/rke2/server/cred/encryption-config.json --etcd-cafile=/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --etcd-certfile=/var/lib/rancher/rke2/server/tls/etcd/client.crt --etcd-keyfile=/var/lib/rancher/rke2/server/tls/etcd/client.key --etcd-servers=https://127.0.0.1:2379 --feature-gates=TTLAfterFinished=true --insecure-port=0 --kubelet-certificate-authority=/var/lib/rancher/rke2/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/rke2/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/rke2/server/tls/client-kube-apiserver.key --profiling=false --proxy-client-cert-file=/var/lib/rancher/rke2/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/rke2/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/rke2/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6443 --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/var/lib/rancher/rke2/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/rke2/server/tls/service.key --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/rke2/server/tls/serving-kube-apiserver.crt --tls-private-key-file=/var/lib/rancher/rke2/server/tls/serving-kube-apiserver.key
INFO[0070] Running kube-scheduler --address=127.0.0.1 --bind-address=127.0.0.1 --kubeconfig=/var/lib/rancher/rke2/server/cred/scheduler.kubeconfig --port=10251 --profiling=false --secure-port=0
INFO[0070] Running kube-controller-manager --address=127.0.0.1 --allocate-node-cidrs=true --bind-address=127.0.0.1 --cluster-cidr=10.42.0.0/16 --cluster-signing-kube-apiserver-client-cert-file=/var/lib/rancher/rke2/server/tls/client-ca.crt --cluster-signing-kube-apiserver-client-key-file=/var/lib/rancher/rke2/server/tls/client-ca.key --cluster-signing-kubelet-client-cert-file=/var/lib/rancher/rke2/server/tls/client-ca.crt --cluster-signing-kubelet-client-key-file=/var/lib/rancher/rke2/server/tls/client-ca.key --cluster-signing-kubelet-serving-cert-file=/var/lib/rancher/rke2/server/tls/server-ca.crt --cluster-signing-kubelet-serving-key-file=/var/lib/rancher/rke2/server/tls/server-ca.key --cluster-signing-legacy-unknown-cert-file=/var/lib/rancher/rke2/server/tls/client-ca.crt --cluster-signing-legacy-unknown-key-file=/var/lib/rancher/rke2/server/tls/client-ca.key --configure-cloud-routes=false --controllers=*,-service,-route,-cloud-node-lifecycle --feature-gates=TTLAfterFinished=true --kubeconfig=/var/lib/rancher/rke2/server/cred/controller.kubeconfig --node-cidr-mask-size=22 --pod-eviction-timeout=10s --port=10252 --profiling=false --root-ca-file=/var/lib/rancher/rke2/server/tls/server-ca.crt --secure-port=0 --service-account-private-key-file=/var/lib/rancher/rke2/server/tls/service.key --use-service-account-credentials=true
INFO[0070] Node token is available at /var/lib/rancher/rke2/server/token
INFO[0070] To join node to cluster: rke2 agent -s https://10.0.1.6:9345 -t ${NODE_TOKEN}
INFO[0070] Waiting for cloudcontroller rbac role to be created
INFO[0070] Wrote kubeconfig /etc/rancher/rke2/rke2.yaml
INFO[0070] Run: rke2 kubectl
INFO[0070] Cluster-Http-Server 2021/05/31 11:36:31 http: TLS handshake error from 127.0.0.1:58430: remote error: tls: bad certificate
INFO[0070] Cluster-Http-Server 2021/05/31 11:36:31 http: TLS handshake error from 127.0.0.1:58436: remote error: tls: bad certificate
INFO[0070] certificate CN=server0 signed by CN=rke2-server-ca@1622459627: notBefore=2021-05-31 11:13:47 +0000 UTC notAfter=2022-05-31 11:36:31 +0000 UTC
INFO[0070] certificate CN=system:node:server0,O=system:nodes signed by CN=rke2-client-ca@1622459627: notBefore=2021-05-31 11:13:47 +0000 UTC notAfter=2022-05-31 11:36:31 +0000 UTC
INFO[0070] Module overlay was already loaded
INFO[0070] Module nf_conntrack was already loaded
INFO[0070] Module br_netfilter was already loaded
INFO[0070] Module iptable_nat was already loaded
INFO[0070] Runtime image index.docker.io/rancher/rke2-runtime:v1.20.7-rke2r2 bin and charts directories already exist; skipping extract
INFO[0070] Updated HelmChart /var/lib/rancher/rke2/server/manifests/rke2-cilium.yaml to set cluster configuration values
INFO[0070] Updated HelmChart /var/lib/rancher/rke2/server/manifests/rke2-coredns.yaml to set cluster configuration values
INFO[0070] Updated HelmChart /var/lib/rancher/rke2/server/manifests/rke2-ingress-nginx.yaml to set cluster configuration values
INFO[0070] Updated HelmChart /var/lib/rancher/rke2/server/manifests/rke2-kube-proxy.yaml to set cluster configuration values
INFO[0070] Updated HelmChart /var/lib/rancher/rke2/server/manifests/rke2-metrics-server.yaml to set cluster configuration values
INFO[0070] Updated HelmChart /var/lib/rancher/rke2/server/manifests/rancher-vsphere-cpi.yaml to set cluster configuration values
INFO[0070] Updated HelmChart /var/lib/rancher/rke2/server/manifests/rancher-vsphere-csi.yaml to set cluster configuration values
INFO[0070] Updated HelmChart /var/lib/rancher/rke2/server/manifests/rke2-canal.yaml to set cluster configuration values
INFO[0070] Using private registry config file at /etc/rancher/rke2/registries.yaml
WARN[0070] SELinux is enabled on this host, but rke2 has not been started with --selinux - containerd SELinux support is disabled
INFO[0070] Logging containerd to /var/lib/rancher/rke2/agent/containerd/containerd.log
INFO[0070] Running containerd -c /var/lib/rancher/rke2/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/rke2/agent/containerd
INFO[0071] Containerd is now running
INFO[0071] Connecting to proxy                           url="wss://127.0.0.1:9345/v1-rke2/connect"
INFO[0071] Handling backend connection request [server0]
INFO[0071] Running kubelet --address=0.0.0.0 --alsologtostderr=false --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=cgroupfs --client-ca-file=/var/lib/rancher/rke2/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --container-runtime=remote --containerd=/run/k3s/containerd/containerd.sock --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --healthz-bind-address=127.0.0.1 --hostname-override=server0 --kubeconfig=/var/lib/rancher/rke2/agent/kubelet.kubeconfig --kubelet-cgroups=/rke2 --log-file-max-size=50 --log-file=/var/lib/rancher/rke2/agent/logs/kubelet.log --logtostderr=false --max-pods=500 --node-labels= --pod-manifest-path=/var/lib/rancher/rke2/agent/pod-manifests --read-only-port=0 --resolv-conf=/etc/resolv.conf --serialize-image-pulls=false --stderrthreshold=FATAL --tls-cert-file=/var/lib/rancher/rke2/agent/serving-kubelet.crt --tls-private-key-file=/var/lib/rancher/rke2/agent/serving-kubelet.key
Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --file-check-frequency has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --sync-frequency has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --address has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --anonymous-auth has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --authentication-token-webhook has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --authorization-mode has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --client-ca-file has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --cloud-provider has been deprecated, will be removed in 1.23, in favor of removing cloud provider code from Kubelet.
Flag --cluster-dns has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --cluster-domain has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --containerd has been deprecated, This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.
Flag --eviction-hard has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --eviction-minimum-reclaim has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --healthz-bind-address has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --kubelet-cgroups has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --max-pods has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --pod-manifest-path has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --read-only-port has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --resolv-conf has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --serialize-image-pulls has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --tls-cert-file has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --tls-private-key-file has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
INFO[0075] Waiting for cloudcontroller rbac role to be created
WARN[0076] Unable to watch for tunnel endpoints: Get "https://127.0.0.1:6443/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&resourceVersion=0&watch=true": dial tcp 127.0.0.1:6443: connect: connection refused
{"level":"warn","ts":"2021-05-31T11:36:41.001Z","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"passthrough:///https://127.0.0.1:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: connection error: desc = \"transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused\""}
INFO[0080] Failed to test data store connection: context deadline exceeded
INFO[0080] Waiting for cloudcontroller rbac role to be created
WARN[0081] Unable to watch for tunnel endpoints: Get "https://127.0.0.1:6443/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&resourceVersion=0&watch=true": dial tcp 127.0.0.1:6443: connect: connection refused
{"level":"warn","ts":"2021-05-31T11:36:45.997Z","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"passthrough:///https://127.0.0.1:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: connection error: desc = \"transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused\""}
INFO[0085] Waiting for cloudcontroller rbac role to be created
INFO[0085] etcd data store connection OK
INFO[0085] Saving current etcd snapshot set to rke2-etcd-snapshots ConfigMap
INFO[0085] Waiting for API server to become available
INFO[0085] Etcd is running, restart without --cluster-reset flag now. Backup and delete ${datadir}/server/db on each peer etcd server and rejoin the nodes
[INFO] [2021-05-31T11:36:50+0000]: Etcd snapshot completed.
[INFO] [2021-05-31T11:36:50+0000]: Copying backup files...
cp: cannot stat '/var/lib/longhorn/longhorn-disk.cfg': No such file or directory
cp: cannot stat '/mnt/restore/cluster0/server0/longhorn_disk/longhorn-disk.cfg': No such file or directory
[INFO] [2021-05-31T11:36:50+0000]: Copy backup files completed.
[INFO] [2021-05-31T11:36:50+0000]: Starting RKE2 Server...
[INFO] [2021-05-31T11:36:53+0000]: Started RKE2 Server successfully.
brandond commented 3 years ago

The logs indicate a successful cluster reset- some errors along the way are to be expected, as things are terminated to accommodate the operation. You say some things are missing from the cluster afterwards? What specifically is missing?

Also, it looks like you're using some scripts to do this restore, as opposed to directly running the restore commands. These scripts appear to be related to Longhorn somehow? Where did those scripts come from? Can you provide more information on what they're attempting to do? The actual rke2 command they're running does not appear to be included in the output.

brandond commented 3 years ago

If the command that is being run is what's given up above then I'm not surprised things are missing - there is no filename to restore from provided, so all that's done is a cluster-reset without restoring anything.

rke2 server --cluster-reset --cluster-reset-restore-path=

adeepsri commented 3 years ago

@brandond script , runs a service check if rke2 is up, if longhorn backup is disabled and then then run etcd restore

########
# Restore ETCD Snapshots
########
function restore_etcd_snapshot(){
  info "Restoring ETCD for Hostname ${ETCD_RESTORE_PATH} started..."

  # disable backup
  disable_backup

  # stop rke2-server before etcd restore
  stop_rke2_server

  # shellcheck disable=SC2012
  latest_snapshot=$(ls -t1 "${RESTORE_DIR}/${ETCD_RESTORE_PATH}/${HOSTNAME}/snapshots"|  head -n 1)

  RESTORE_FILE=${RESTORE_DIR}/${ETCD_RESTORE_PATH}/${HOSTNAME}/snapshots/${latest_snapshot}
  info "Restore ETCD with snapshot ${latest_snapshot} started..."

  rke2 server --cluster-reset --cluster-reset-restore-path="${RESTORE_FILE}" &

  RKE2_PID=$!
  echo "${RKE2_PID}"

  completed=0
  maxtry=100
  try=0
  while (( try < maxtry ));
  do
    if ps -p $RKE2_PID > /dev/null
    then
      try=$((try+1))
      sleep 10
    else
      info "Etcd snapshot completed."
      completed=1
      break
    fi
  done

  if [ ${completed} -eq 0 ]; then
    kill -9  ${RKE2_PID} || true
    info "Stop snapshot restore."
  fi

  # copy_backup_files
  start_rke2_server
}

please find the script where we are executing etcd restore. After ETCD snapshot restore completes, none of the data available in previous cluster gets restored.

brandond commented 3 years ago

As I mentioned above, is is normal to see errors from the kubelet and containerd while the static pods are being cleaned up.

The only way I can get the behavior you're observing is if I leave the --cluster-reset-restore-path= flag value empty; when I include a valid path I eventually see:

INFO[0080] Static pod cleanup completed successfully
INFO[0080] Starting rke2 v1.20.7+rke2r2 (ea6514775f816b0b4972f832de7d98eba8dff55c)
INFO[0080] Managed etcd cluster bootstrap already complete and initialized
INFO[0080] Pre-restore etcd database moved to /var/lib/rancher/rke2/server/db/etcd-old-1622574351
{"level":"info","msg":"restoring snapshot","path":"/root/on-demand-centos01.lan.khaus-1622574106","wal-dir":"/var/lib/rancher/rke2/server/db/etcd/member/wal","data-dir":"/var/lib/rancher/rke2/server/db/etcd","snap-dir":"/var/lib/rancher/rke2/server/db/etcd/member/snap"}
{"level":"info","msg":"added member","cluster-id":"5fcba85a61b79aef","local-member-id":"0","added-peer-id":"d19b13366d24ff3e","added-peer-peer-urls":["https://10.0.1.137:2380"]}
{"level":"info","msg":"restored snapshot","path":"/root/on-demand-centos01.lan.khaus-1622574106","wal-dir":"/var/lib/rancher/rke2/server/db/etcd/member/wal","data-dir":"/var/lib/rancher/rke2/server/db/etcd","snap-dir":"/var/lib/rancher/rke2/server/db/etcd/member/snap"}

Can you modify your script to include set -x somewhere at the top, so that all of the commands and arguments run by the script are captured in your log output? I suspect there's a syntax error somewhere that's preventing RESTORE_FILE from being set properly.

adeepsri commented 3 years ago

@brandond I have validated and found out issue was with nfs server storing etcd. when etcd snapshot gets created it creates a part file and later deletes it and create corresponding file. This results in empty file passed in the rke2 restore command causing the issue.

I have fix the script and didnot see any issue after that.