Calico tries to reassign IP addresses already in use

[TheKangaroo]

Calico tries to reassign IP addresses already in use

Expected Behavior

Calico should assign a unused IP address to a new created pod

Current Behavior

Calico trise several IP addresses, already in use by other pods, leading to multiple errors like

Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "9f90580e9dd409a5ab621ef44c1637972ec6fb5d55db732c1943647ce1cf9e00" network for pod "nginx-ingress-controller-6664bc7855-8hxwv": networkPlugin cni failed to set up pod "nginx-ingress-controller-6664bc7855-8hxwv_eai-selfservice-dev" network: error adding host side routes for interface: calicfa03c2c5b1, error: route (Ifindex: 436, Dst: 100.80.1.62/32, Scope: 253) already exists for an interface other than 'calicfa03c2c5b1'

with changing IP addresses. At some point the container starts with a free IP address assigned.

Steps to Reproduce (for bugs)

I'm not sure on how to reproduce this, but I'll provide as many informations as I can: The events on a pods looks like this:

  Warning  FailedCreatePodSandBox  43s                kubelet, k8s-worker-01.example.com  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "9f90580e9dd409a5ab621ef44c1637972ec6fb5d55db732c1943647ce1cf9e00" network for pod "nginx-ingress-controller-6664bc7855-8hxwv": networkPlugin cni failed to set up pod "nginx-ingress-controller-6664bc7855-8hxwv_eai-selfservice-dev" network: error adding host side routes for interface: calicfa03c2c5b1, error: route (Ifindex: 436, Dst: 100.80.1.62/32, Scope: 253) already exists for an interface other than 'calicfa03c2c5b1'
  Warning  FailedCreatePodSandBox  37s                kubelet, k8s-worker-01.example.com  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "b9ecd16415de567bdad4523f42ee9e0a1b7ebf609eed03a61e88f117b2b5f2d1" network for pod "nginx-ingress-controller-6664bc7855-8hxwv": networkPlugin cni failed to set up pod "nginx-ingress-controller-6664bc7855-8hxwv_eai-selfservice-dev" network: error adding host side routes for interface: calicfa03c2c5b1, error: route (Ifindex: 437, Dst: 100.80.1.63/32, Scope: 253) already exists for an interface other than 'calicfa03c2c5b1'
  Warning  FailedCreatePodSandBox  26s                kubelet, k8s-worker-01.example.com  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "667a2b553cf6598f7fc0da75dfb9d5bd18163c87348db3a03e42be98d9a0762c" network for pod "nginx-ingress-controller-6664bc7855-8hxwv": networkPlugin cni failed to set up pod "nginx-ingress-controller-6664bc7855-8hxwv_eai-selfservice-dev" network: error adding host side routes for interface: calicfa03c2c5b1, error: route (Ifindex: 439, Dst: 100.80.1.65/32, Scope: 253) already exists for an interface other than 'calicfa03c2c5b1'

You can see here, that it tries to assign the IP addresses 100.80.1.62, .63 and .64, which are all in use. Next it startet the pod successfull with 100.80.1.67.

Next I checked one of the exisiting pods, but everything looks good as far as I can tell:

k get pod -owide --all-namespaces|grep 100.80.1.63
eai-dev               default-http-backend-6fd4978fb-2hkx4                           1/1     Running                      0          37h     100.80.1.63      k8s-worker-01.example.com    <none>           <none>

And to be sure I check the ipamhandle of this pod too:

k get ipamhandle k8s-pod-network.5a1d961664df5e922c225bc06e69d0fc60aedf6eaa96670ecd5febc7cb160262 -oyaml
apiVersion: crd.projectcalico.org/v1
kind: IPAMHandle
metadata:
  annotations:
    projectcalico.org/metadata: '{"creationTimestamp":null}'
  creationTimestamp: "2020-08-25T23:45:08Z"
  generation: 1
  name: k8s-pod-network.5a1d961664df5e922c225bc06e69d0fc60aedf6eaa96670ecd5febc7cb160262
  resourceVersion: "971980875"
  selfLink: /apis/crd.projectcalico.org/v1/ipamhandles/k8s-pod-network.5a1d961664df5e922c225bc06e69d0fc60aedf6eaa96670ecd5febc7cb160262
  uid: 467c0bca-4a32-4179-b214-3e286cadb783
spec:
  block:
    100.80.1.0/26: 1
  handleID: k8s-pod-network.5a1d961664df5e922c225bc06e69d0fc60aedf6eaa96670ecd5febc7cb160262

(even though I'm not sure whether this helps.)

I checked the ipamblock of the node, which is huge and shortened here, but you can see, that the IP addresses are not listet under unallocated:

apiVersion: crd.projectcalico.org/v1
kind: IPAMBlock
metadata:
  annotations:
    projectcalico.org/metadata: '{"creationTimestamp":null}'
  creationTimestamp: "2020-07-12T11:09:10Z"
  generation: 155
  name: 100-80-1-0-26
  resourceVersion: "971980877"
  selfLink: /apis/crd.projectcalico.org/v1/ipamblocks/100-80-1-0-26
  uid: 1c670855-c430-11ea-b730-00505684b2d0
spec:
  affinity: host:k8s-worker-00.example.com
  allocations:
  - null
  - null
  - 15
  - 11
  - 29
  - 40
  - 51
  - 12
  - 13
  - 14
  - null
  - 0
  - null
  - 1
  - 2
  - null
  - null
  - null
  - null
  - 3
  - 16
  - 4
  - 5
  - 6
  - 7
  - 8
  - 9
  - 17
  - 10
  - 18
  - 19
  - 20
  - 21
  - 22
  - 23
  - 24
  - 25
  - 26
  - 27
  - 28
  - 30
  - 31
  - 32
  - 33
  - 34
  - 35
  - 36
  - 37
  - 38
  - 39
  - 41
  - 42
  - 43
  - 44
  - 45
  - 46
  - 47
  - 48
  - 49
  - 50
  - 52
  - 53
  - 54
  - 55
  attributes:
  - handle_id: "k8s-pod-network.dd372304a3d85e8900e0b940ce4d98936543fd403312ee151f6f9a178d08636d\r\neth0"
    secondary:
      namespace: rook-ceph-stage-primary
      node: k8s-worker-00.example.com
      pod: rook-ceph-stage-primary-backup-pdm7z
  - handle_id: "k8s-pod-network.518cee30324d9bee2c1a80a78fb41055ea5609298833e0a042ab9097d51be239\r\neth0"
    secondary:
      namespace: cxa-stage
      node: k8s-worker-00.example.com
      pod: release-name-zookeeper-1
[...]
  - handle_id: k8s-pod-network.44333bb9ad1fa32a354121431cb75c4e4b8996587793434eeb0c493ea6e4893d
    secondary:
      namespace: eai-dev
      node: k8s-worker-01.example.com
      pod: comp00000006-66c689ff6f-x9472
  - handle_id: k8s-pod-network.5a1d961664df5e922c225bc06e69d0fc60aedf6eaa96670ecd5febc7cb160262
    secondary:
      namespace: eai-dev
      node: k8s-worker-01.example.com
      pod: default-http-backend-6fd4978fb-2hkx4
  cidr: 100.80.1.0/26
  deleted: false
  strictAffinity: false
  unallocated:
  - 0
  - 1
  - 10
  - 12
  - 15
  - 16
  - 17
  - 18

One strange thing I noticed here, is that under attributes it lists another node (k8s-worker-00.example.com)which has been deleted and is no longer part of the cluster. I'm not sure if this is a problem. Maybe this happend because calico reused the CIDR range for this node. I checked the node ipamhandle too, but this looks good to me too:

apiVersion: crd.projectcalico.org/v1
kind: IPAMHandle
metadata:
  annotations:
    projectcalico.org/metadata: '{"creationTimestamp":null}'
  creationTimestamp: "2020-08-24T23:28:51Z"
  generation: 1
  name: ipip-tunnel-addr-k8s-worker-01.example.com
  resourceVersion: "970070259"
  selfLink: /apis/crd.projectcalico.org/v1/ipamhandles/ipip-tunnel-addr-k8s-worker-01.example.com
  uid: d80ea18c-4490-446a-b079-ef286a17cfe0
spec:
  block:
    100.80.1.64/26: 1
  handleID: ipip-tunnel-addr-k8s-worker-01.example.com

And last here are the calico-node logs for this particular pod, but it seems to only log the successful assignment of the final IP address:

k logs -n kube-system calico-node-8l5pp |grep nginx-ingress-controller-6664bc7855-8hxwv
2020-08-26 13:22:57.551 [INFO][22004] calc_graph.go 392: Local endpoint updated id=WorkloadEndpoint(node=k8s-worker-01.example.com, orchestrator=k8s, workload=eai-selfservice-dev/nginx-ingress-controller-6664bc7855-8hxwv, name=eth0)
2020-08-26 13:22:57.555 [INFO][22004] int_dataplane.go 834: Received *proto.WorkloadEndpointUpdate update from calculation graph msg=id:<orchestrator_id:"k8s" workload_id:"eai-selfservice-dev/nginx-ingress-controller-6664bc7855-8hxwv" endpoint_id:"eth0" > endpoint:<state:"active" name:"calicfa03c2c5b1" profile_ids:"kns.eai-selfservice-dev" profile_ids:"ksa.eai-selfservice-dev.eai-selfservice-dev-nginx-ingress-serviceaccount" ipv4_nets:"100.80.1.67/32" tiers:<name:"default" ingress_policies:"eai-selfservice-dev/knp.default.allow-ingress-access" ingress_policies:"eai-selfservice-dev/knp.default.allow-mssql" ingress_policies:"eai-selfservice-dev/knp.default.allow-own-namepace" ingress_policies:"eai-selfservice-dev/knp.default.allow-prometheus" ingress_policies:"eai-selfservice-dev/knp.default.allow-prometheus-springboot" ingress_policies:"eai-selfservice-dev/knp.default.default-deny-do-not-delete" egress_policies:"eai-selfservice-dev/knp.default.allow-active-directory" egress_policies:"eai-selfservice-dev/knp.default.allow-dns-access" egress_policies:"eai-selfservice-dev/knp.default.allow-mssql" egress_policies:"eai-selfservice-dev/knp.default.allow-nginx-apiserver-access" egress_policies:"eai-selfservice-dev/knp.default.allow-own-namepace" egress_policies:"eai-selfservice-dev/knp.default.allow-pki-cert" egress_policies:"eai-selfservice-dev/knp.default.allow-smtp" egress_policies:"eai-selfservice-dev/knp.default.allow-tibco-ems" egress_policies:"eai-selfservice-dev/knp.default.allow-tls" egress_policies:"eai-selfservice-dev/knp.default.default-deny-do-not-delete" > >
2020-08-26 13:22:57.555 [INFO][22004] endpoint_mgr.go 545: Updating per-endpoint chains. id=proto.WorkloadEndpointID{OrchestratorId:"k8s", WorkloadId:"eai-selfservice-dev/nginx-ingress-controller-6664bc7855-8hxwv", EndpointId:"eth0"}
2020-08-26 13:22:57.556 [INFO][22004] endpoint_mgr.go 572: Updating endpoint routes. id=proto.WorkloadEndpointID{OrchestratorId:"k8s", WorkloadId:"eai-selfservice-dev/nginx-ingress-controller-6664bc7855-8hxwv", EndpointId:"eth0"}
2020-08-26 13:22:57.556 [INFO][22004] endpoint_mgr.go 438: Re-evaluated workload endpoint status adminUp=true failed=false known=true operUp=true status="up" workloadEndpointID=proto.WorkloadEndpointID{OrchestratorId:"k8s", WorkloadId:"eai-selfservice-dev/nginx-ingress-controller-6664bc7855-8hxwv", EndpointId:"eth0"}
2020-08-26 13:22:57.557 [INFO][22004] status_combiner.go 58: Storing endpoint status update ipVersion=0x4 status="up" workload=proto.WorkloadEndpointID{OrchestratorId:"k8s", WorkloadId:"eai-selfservice-dev/nginx-ingress-controller-6664bc7855-8hxwv", EndpointId:"eth0"}
2020-08-26 13:22:57.635 [INFO][22004] status_combiner.go 81: Endpoint up for at least one IP version id=proto.WorkloadEndpointID{OrchestratorId:"k8s", WorkloadId:"eai-selfservice-dev/nginx-ingress-controller-6664bc7855-8hxwv", EndpointId:"eth0"} ipVersion=0x4 status="up"
2020-08-26 13:22:57.635 [INFO][22004] status_combiner.go 98: Reporting combined status. id=proto.WorkloadEndpointID{OrchestratorId:"k8s", WorkloadId:"eai-selfservice-dev/nginx-ingress-controller-6664bc7855-8hxwv", EndpointId:"eth0"} status="up"

Context

We did redeploy calico before, because we needed to change the pod IP CIDR. For this reason we completely deleted and reapplied calico with all CRs and CRDs. All the pods on this node has been started after the pod IP CIDR change.

If I missed to provide some important information please let me know.

Your Environment

• Calico version v3.11.3
• Orchestrator version (e.g. kubernetes, mesos, rkt): k8s 1.17.6
• Operating System and version: Flatcar Container Linux 2345.3.1

[caseydavenport]

We did redeploy calico before, because we needed to change the pod IP CIDR. For this reason we completely deleted and reapplied calico with all CRs and CRDs. All the pods on this node has been started after the pod IP CIDR change.

I can't say for sure if this is the problem here, but if you delete all the Calico CRs and CRDs, then you really need to restart every pod in the cluster, otherwise the IPAM data will be out-of-sync with the actual state of the cluster, and it can result in IP addresses being used more than once, which could result in behavior like this.

When re-deploying Calico, you should make sure that the CRs and CRDs are left in-place and not deleted. Otherwise, you'll need to tear down every pod in the cluster, clear our Calico's IPAM state by deleting the CRs, and then re-launch the pods.

[TheKangaroo]

Actually we already drained, deleted, redeployed and rejoined every node in the cluster after reinstalling calico, so this should be fine.

@fasaxc assisted me in troubleshooting this issue in calico slack (https://calicousers.slack.com/archives/C0BCA117T/p1598449811033700) but I could no longer reproduce this issue. Maybe it is best to close this issue and I'll come back with more information when I see this happen again.

[TheKangaroo]

Hi just a quick update. We definitely saw this issue again, but I couldn't reproduce it after a colleague saw this on one of his pods.

[TheKangaroo]

I discussed some new evidence in slack (https://calicousers.slack.com/archives/C0BCA117T/p1603264717140400) with @fasaxc and @caseydavenport and thought I would put it in this ticket and reopen it as well. I increased the loglevel of calico controller to debug and removed a node from the cluster. It seems like calico is not able to delete the ipamhandles but thinks they are already gone so it contiunes to delete the node and leves stale ipamhandles in the cluster. For me it looks like this happens because of a comparison between ipamhandle names like k8s-pod-network.<id>\r\neth0 and k8s-pod-network.<id>:

2020-10-23 14:46:26.490 [DEBUG][1] k8s.go 564: Performing 'Get' for ClusterInformation(default)
2020-10-23 14:46:26.490 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=ClusterInformation(default) Resource="ClusterInformations" Revision=""
2020-10-23 14:46:26.490 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=ClusterInformation(default) Name="default" Namespace="" Resource="ClusterInformations" Revision=""
2020-10-23 14:46:26.495 [DEBUG][1] client.go 282: Cluster GUID value already set ClusterGUID="3b54b2588a634e44bff43a77a1829349"
2020-10-23 14:46:26.495 [DEBUG][1] client.go 292: DatastoreReady value already set DatastoreReady=(*bool)(0xc0002ec777)
2020-10-23 14:46:31.360 [INFO][1] ipam.go 45: Synchronizing IPAM data
2020-10-23 14:46:31.364 [DEBUG][1] watcher.go 122: Kubernetes event converted and sent to backend watcher resource="Node"
2020-10-23 14:46:31.364 [DEBUG][1] watchercache.go 100: Reading event from results channel ListRoot="/calico/resources/v3/projectcalico.org/nodes" RC=(<-chan api.WatchEvent)(0xc000822300)
2020-10-23 14:46:31.364 [DEBUG][1] watchercache.go 387: Datastore entry deleted, sending syncer update Key="Node(k8s-worker-23.example.com)" ListRoot="/calico/resources/v3/projectcalico.org/nodes"
2020-10-23 14:46:31.364 [DEBUG][1] watchersyncer.go 241: Sending syncer updates (if any to send) NumUpdates=1
2020-10-23 14:46:31.364 [DEBUG][1] node_syncer.go 57: Node controller syncer received updates: []api.Update{api.Update{KVPair:model.KVPair{Key:model.ResourceKey{Name:"k8s-worker-23.example.com", Namespace:"", Kind:"Node"}, Value:interface {}(nil), Revision:"", UID:(*types.UID)(nil), TTL:0}, UpdateType:0x3}}
2020-10-23 14:46:31.364 [DEBUG][1] node_syncer.go 116: Unmapping k8s node -> calico node. k8s-worker-23.example.com -> k8s-worker-23.example.com
2020-10-23 14:46:31.365 [DEBUG][1] k8s.go 579: Performing 'List' for {IPVersion:0} model.BlockListOptions
2020-10-23 14:46:31.365 [DEBUG][1] customresource.go 242: List Custom K8s Resource ListInterface=IPAMBlock Resource="IPAMBlocks"
2020-10-23 14:46:31.407 [DEBUG][1] ipam.go 107: Calico nodes found in IPAM: map[k8s-master-01.example.com:[{0xc000903e80 map[namespace:kube-system node:k8s-master-01.example.com pod:expire-check-master-5p8mj]} {0xc000027290 map[node:k8s-master-01.example.com type:ipipTunnelAddress]} {0xc000026ff0 map[namespace:kube-system node:k8s-master-01.example.com pod:coredns-8688c5f956-x2hv4]} {0xc000027060 map[namespace:kube-system node:k8s-master-01.example.com pod:default-http-backend-59478d4c57-r272n]} {0xc0000270d0 map[namespace:kube-system node:k8s-master-01.example.com pod:expire-check-master-lszqt]} {0xc000027140 map[namespace:kube-system node:k8s-master-01.example.com pod:coredns-8688c5f956-7m9rb]} {0xc0000271b0 map[namespace:kube-system node:k8s-master-01.example.com pod:coredns-8688c5f956-qmvqz]} {0xc000027220 map[namespace:kube-system node:k8s-master-01.example.com pod:calico-kube-controllers-648f4868b8-h65k4]} {0xc0000272e0 map[namespace:kube-system node:k8s-master-01.example.com pod:expire-check-master-vgx4b]}] k8s-master-02.example.com:[{0xc0006d1fb0 map[namespace:kube-system node:k8s-master-02.example.com pod:expire-check-master-9t294]} {0xc00058e020 map[namespace:kube-system node:k8s-master-02.example.com pod:expire-check-master-9t294]} {0xc0006d1f40 map[namespace:kube-system node:k8s-master-02.example.com pod:expire-check-master-9t294]} {0xc00058e090 map[namespace:kube-system node:k8s-master-02.example.com pod:expire-check-master-9t294]} {0xc00058e100 map[namespace:kube-system node:k8s-master-02.example.com pod:expire-check-master-9t294]} {0xc00058e170 map[namespace:kube-system node:k8s-master-02.example.com pod:expire-check-master-9t294]} {0xc00058e1e0 map[namespace:kube-system node:k8s-master-02.example.com pod:expire-check-master-9t294]} {0xc0005e6910 map[node:k8s-master-02.example.com type:ipipTunnelAddress]}] k8s-worker-00.example.com:[{0xc00058eec0 map[namespace:rook-ceph-dev-primary node:k8s-worker-00.example.com pod:rook-ceph-mon-a-6c595656f8-rkr7n]} {0xc00058ef30 map[namespace:rook-ceph-dev-primary node:k8s-worker-00.example.com pod:rook-discover-qxxlm]} {0xc00058efa0 map[namespace:rook-ceph-dev-primary node:k8s-worker-00.example.com pod:rook-ceph-mon-e-cddcbb978-t9qlw]} {0xc00058f010 map[namespace:kube-system node:k8s-worker-00.example.com pod:kube-iptables-tailer-g4l4r]} {0xc00058f080 map[namespace:rook-ceph-dev-primary node:k8s-worker-00.example.com pod:rook-ceph-dev-primary-backup-pbqkk]} {0xc00058f0f0 map[namespace:kube-system node:k8s-worker-00.example.com pod:expire-check-worker-bb288]} {0xc00058f160 map[namespace:velero node:k8s-worker-00.example.com pod:restic-k5rk7]} {0xc0005e6700 map[node:k8s-worker-00.example.com type:ipipTunnelAddress]}] k8s-worker-01.example.com:[{0xc000903f50 map[namespace:nuclio-infra node:k8s-worker-01.example.com pod:thanos-querier-5d5b8f467d-s8mlh]} {0xc000903fc0 map[namespace:lxos-infra node:k8s-worker-01.example.com pod:nginx-ingress-controller-597d797d89-c9d5p]} {0xc000026080 map[namespace:monitoring-infra node:k8s-worker-01.example.com pod:nginx-ingress-controller-77589b585-d9vbh]} {0xc000026250 map[namespace:gitlab-infra node:k8s-worker-01.example.com pod:nginx-ingress-controller-9c448878c-2nhrb]} {0xc000174640 map[node:k8s-worker-01.example.com type:ipipTunnelAddress]} {0xc0000273b0 map[namespace:gitlab-infra node:k8s-worker-01.example.com pod:gitlab-dev-chronograf-f559f8679-l6nn8]} {0xc000027420 map[namespace:kube-system node:k8s-worker-01.example.com pod:dashboard-metrics-scraper-664b58f47c-p6fs7]} {0xc000027f10 map[namespace:rook-ceph-dev-primary node:k8s-worker-01.example.com pod:rook-ceph-dev-primary-backup-4d4qk]} {0xc000027f80 map[namespace:clair-infra node:k8s-worker-01.example.com pod:nginx-ingress-controller-5ccfdfd797-66fv2]} {0xc000027ff0 map[namespace:velero node:k8s-worker-01.example.com pod:restic-jbrmb]} {0xc000174060 map[namespace:esdata-infra node:k8s-worker-01.example.com pod:nginx-ingress-controller-75b5c6c5fc-qbxmb]} {0xc000174160 map[namespace:rene-dev node:k8s-worker-01.example.com pod:grafana-558dcbff8-c6f6t]} {0xc0001741d0 map[namespace:cert-manager node:k8s-worker-01.example.com pod:cert-manager-9dd6685b9-pfjwg]} {0xc000027490 map[namespace:esdata-dev node:k8s-worker-01.example.com pod:default-http-backend-5f65599f44-jzlxw]} {0xc000027500 map[namespace:gitlab-infra node:k8s-worker-01.example.com pod:thanos-store-gateway-0]} {0xc000027570 map[namespace:couchbase-dev node:k8s-worker-01.example.com pod:default-http-backend-5f65599f44-29rl8]} {0xc0000275e0 map[namespace:esdata-infra node:k8s-worker-01.example.com pod:alertmanager-infra-alertmanager-1]} {0xc000027650 map[namespace:kube-system node:k8s-worker-01.example.com pod:expire-check-worker-gx86x]} {0xc0000276c0 map[namespace:catalog node:k8s-worker-01.example.com pod:catalog-catalog-controller-manager-75d98fbc4d-8swjt]} {0xc000027730 map[namespace:aws-service-broker node:k8s-worker-01.example.com pod:aws-servicebroker-fc58f95bb-6zqrk]} {0xc0000277a0 map[namespace:clair-infra node:k8s-worker-01.example.com pod:kube-state-metrics-765fdbd9c-cnnfr]} {0xc000027810 map[namespace:nuclio-infra node:k8s-worker-01.example.com pod:thanos-store-gateway-0]} {0xc000027880 map[namespace:catalog node:k8s-worker-01.example.com pod:catalog-catalog-apiserver-865497c556-jlklm]} {0xc0000278f0 map[namespace:clair-dev node:k8s-worker-01.example.com pod:default-http-backend-5f65599f44-s274l]} {0xc000027960 map[namespace:cert-manager node:k8s-worker-01.example.com pod:cert-manager-cainjector-8545fdf87c-5jxgt]} {0xc0000279d0 map[namespace:lxos-dev node:k8s-worker-01.example.com pod:default-http-backend-5f65599f44-qvtnc]} {0xc000027a40 map[namespace:couchbase-infra node:k8s-worker-01.example.com pod:default-http-backend-5f65599f44-jzsf4]} {0xc000027ab0 map[namespace:esdata-dev node:k8s-worker-01.example.com pod:nginx-ingress-controller-5499bc654c-jfh86]} {0xc000027b20 map[namespace:cert-manager node:k8s-worker-01.example.com pod:cert-manager-webhook-8c5db9fb6-sns9b]} {0xc000027b90 map[namespace:couchbase-dev node:k8s-worker-01.example.com pod:nginx-ingress-controller-7885d48697-b25h8]} {0xc000027c00 map[namespace:clair-dev node:k8s-worker-01.example.com pod:nginx-ingress-controller-866dd75bb9-qr5bx]} {0xc000027c70 map[namespace:gitlab-infra node:k8s-worker-01.example.com pod:kube-state-metrics-7fc464478b-k5p6h]} {0xc000027ce0 map[namespace:esdata-infra node:k8s-worker-01.example.com pod:thanos-store-gateway-0]} {0xc000027d50 map[namespace:kube-system node:k8s-worker-01.example.com pod:kube-iptables-tailer-58lbv]} {0xc000027dc0 map[namespace:couchbase-infra node:k8s-worker-01.example.com pod:nginx-ingress-controller-8684cd764f-nqjmp]} {0xc000027e30 map[namespace:rook-ceph-dev-primary node:k8s-worker-01.example.com pod:rook-discover-2vwkv]} {0xc000027ea0 map[namespace:lxos-infra node:k8s-worker-01.example.com pod:nginx-ingress-controller-597d797d89-tj9mz]} {0xc000174250 map[namespace:kube-system node:k8s-worker-01.example.com pod:dashboard-metrics-scraper-664b58f47c-c7q75]} {0xc0001747e0 map[namespace:catalog node:k8s-worker-01.example.com pod:catalog-catalog-controller-manager-959946d4d-zspqb]} {0xc000174770 map[namespace:catalog node:k8s-worker-01.example.com pod:catalog-catalog-webhook-76987b56bd-k62mk]} {0xc0001742c0 map[namespace:prometheus node:k8s-worker-01.example.com pod:master-grafana-7d65f7df9b-grgtf]} {0xc000174330 map[namespace:gitlab-infra node:k8s-worker-01.example.com pod:alertmanager-infra-alertmanager-0]} {0xc0001743a0 map[namespace:kube-system node:k8s-worker-01.example.com pod:sealed-secrets-controller-56c5d66bcc-9dqz5]} {0xc000174410 map[namespace:kube-system node:k8s-worker-01.example.com pod:metrics-server-78df7f98c4-v5ljj]} {0xc000174480 map[namespace:monitoring node:k8s-worker-01.example.com pod:mon-prometheus-alertmanager-65568b4666-gxqnw]} {0xc0001744f0 map[namespace:gitlab-infra node:k8s-worker-01.example.com pod:gitlab-dev-chronograf-f559f8679-smbzc]} {0xc000174560 map[namespace:kube-system node:k8s-worker-01.example.com pod:kubernetes-dashboard-fcc659c67-q9t7l]} {0xc0001745d0 map[namespace:monitoring-infra node:k8s-worker-01.example.com pod:nginx-ingress-controller-77589b585-vlh4g]} {0xc000174690 map[namespace:velero node:k8s-worker-01.example.com pod:restic-kz5ck]} {0xc000174700 map[namespace:kube-system node:k8s-worker-01.example.com pod:coredns-66459df666-zprnq]} {0xc0001748c0 map[namespace:lxos-dev node:k8s-worker-01.example.com pod:nginx-ingress-controller-8dcf8dc56-47sr6]} {0xc000174930 map[namespace:kube-system node:k8s-worker-01.example.com pod:calico-kube-controllers-798b6d864-68qs2]} {0xc000174850 map[namespace:rook-ceph-dev-primary node:k8s-worker-01.example.com pod:rook-ceph-operator-6c656d4995-qs9zt]}] k8s-worker-02.example.com:[{0xc000026330 map[namespace:monitoring node:k8s-worker-02.example.com pod:nginx-ingress-controller-859d5cf8f-dvlwn]} {0xc0000263a0 map[namespace:velero node:k8s-worker-02.example.com pod:velero-dev-676556f788-hs6cz]} {0xc000026440 map[namespace:rene-dev node:k8s-worker-02.example.com pod:prometheus-kube-state-metrics-86b5b94b74-vtld6]} {0xc0000264c0 map[namespace:velero node:k8s-worker-02.example.com pod:restic-v988n]} {0xc0007b18d0 map[namespace:velero node:k8s-worker-02.example.com pod:restic-swklx]} {0xc0007b06b0 map[namespace:max-test-alerting node:k8s-worker-02.example.com pod:test-alert-84778555f5-22x42]} {0xc0007b0e40 map[namespace:kube-system node:k8s-worker-02.example.com pod:kubernetes-dashboard-fcc659c67-bwdsd]} {0xc0007b12b0 map[namespace:prometheus node:k8s-worker-02.example.com pod:master-grafana-7d65f7df9b-c4cht]} {0xc0007b1410 map[namespace:prometheus node:k8s-worker-02.example.com pod:master-kube-state-metrics-7f4f8dfd55-tg6qh]} {0xc0007b1480 map[namespace:elastic-infra node:k8s-worker-02.example.com pod:alertmanager-infra-alertmanager-1]} {0xc0007b14f0 map[namespace:prometheus node:k8s-worker-02.example.com pod:master-prometheus-operator-operator-cff4b9c67-k2nrz]} {0xc0007b1560 map[namespace:elastic-dev node:k8s-worker-02.example.com pod:filebeat-syslog-5fd9947874-fktcx]} {0xc0007b0230 map[namespace:kube-system node:k8s-worker-02.example.com pod:tiller-deploy-786bc89dd8-449w2]} {0xc0007b15d0 map[namespace:couchbase-dev node:k8s-worker-02.example.com pod:default-http-backend-5f65599f44-9z9tk]} {0xc0007b02a0 map[namespace:rene-dev node:k8s-worker-02.example.com pod:default-http-backend-5f65599f44-2mt5d]} {0xc0007b0320 map[namespace:gitlab-infra node:k8s-worker-02.example.com pod:alertmanager-infra-alertmanager-0]} {0xc0007b0400 map[namespace:monitoring node:k8s-worker-02.example.com pod:mon-prometheus-alertmanager-65568b4666-n2xd9]} {0xc0007b0480 map[namespace:nuclio-infra node:k8s-worker-02.example.com pod:default-http-backend-5f65599f44-h9gdv]} {0xc0007b04f0 map[namespace:lxos-infra node:k8s-worker-02.example.com pod:thanos-compactor-0]} {0xc0007b0560 map[namespace:elastic-dev node:k8s-worker-02.example.com pod:nginx-ingress-controller-7c8c64fd89-vlmwx]} {0xc0007b05d0 map[namespace:monitoring node:k8s-worker-02.example.com pod:default-http-backend-5f65599f44-rmfdj]} {0xc0007b0640 map[namespace:nuclio-dev node:k8s-worker-02.example.com pod:nginx-ingress-controller-754b5bd7f5-xxwt7]} {0xc0007b0720 map[namespace:couchbase-infra node:k8s-worker-02.example.com pod:thanos-querier-bdccc947d-7zdln]} {0xc0007b0790 map[namespace:monitoring node:k8s-worker-02.example.com pod:mon-prometheus-pushgateway-7987d7d4b6-zkb8m]} {0xc0007b1640 map[namespace:catalog node:k8s-worker-02.example.com pod:catalog-catalog-apiserver-865497c556-rfqv6]} {0xc0007b0800 map[namespace:clair-infra node:k8s-worker-02.example.com pod:nginx-ingress-controller-5ccfdfd797-5ktpj]} {0xc0007b0870 map[namespace:prometheus node:k8s-worker-02.example.com pod:prometheus-msteams-7d6fcd79cc-bjxqp]} {0xc0007b08e0 map[namespace:kube-system node:k8s-worker-02.example.com pod:expire-check-worker-q7lfr]} {0xc0007b16b0 map[namespace:esdata-infra node:k8s-worker-02.example.com pod:nginx-ingress-controller-75b5c6c5fc-6r4x7]} {0xc0007b0950 map[namespace:monitoring-infra node:k8s-worker-02.example.com pod:nginx-ingress-controller-77589b585-zh8tt]} {0xc0007b09c0 map[namespace:elastic-system node:k8s-worker-02.example.com pod:elastic-operator-0]} {0xc0007b1720 map[namespace:lxos-infra node:k8s-worker-02.example.com pod:nginx-ingress-controller-597d797d89-gl6sr]} {0xc0007b0a40 map[namespace:couchbase-infra node:k8s-worker-02.example.com pod:nginx-ingress-controller-8684cd764f-4jhqf]} {0xc0007b0ab0 map[namespace:gitlab-dev node:k8s-worker-02.example.com pod:nginx-ingress-controller-85d6f6cd6c-fcbqk]} {0xc0007b0b20 map[namespace:gitlab-infra node:k8s-worker-02.example.com pod:default-http-backend-5f65599f44-8bl54]} {0xc0007b0ba0 map[namespace:kube-system node:k8s-worker-02.example.com pod:dashboard-metrics-scraper-664b58f47c-dflv8]} {0xc0007b0c10 map[namespace:snyk-monitor node:k8s-worker-02.example.com pod:snyk-monitor-6fc8c86d4c-sv25n]} {0xc0007b0c80 map[namespace:gitlab-infra node:k8s-worker-02.example.com pod:gitlab-dev-chronograf-f559f8679-ptkd5]} {0xc0007b1790 map[namespace:couchbase-dev node:k8s-worker-02.example.com pod:nginx-ingress-controller-7885d48697-l59j2]} {0xc0007b0cf0 map[namespace:gitlab-infra node:k8s-worker-02.example.com pod:thanos-compactor-0]} {0xc0007b0d60 map[namespace:kube-system node:k8s-worker-02.example.com pod:kube-iptables-tailer-jsl2x]} {0xc0007b0dd0 map[namespace:couchbase-infra node:k8s-worker-02.example.com pod:alertmanager-infra-alertmanager-0]} {0xc0007b1800 map[namespace:postgres-operator node:k8s-worker-02.example.com pod:postgres-operator-6c9ff7cff5-c7mp6]} {0xc0007b0eb0 map[namespace:rook-ceph-dev-primary node:k8s-worker-02.example.com pod:rook-ceph-mgr-a-9b6c5f86-g8wd6]} {0xc0007b0f20 map[namespace:couchbase-infra node:k8s-worker-02.example.com pod:thanos-compactor-0]} {0xc0007b0fa0 map[namespace:kube-system node:k8s-worker-02.example.com pod:external-dns-5c48cc494b-kgvnx]} {0xc0007b1010 map[namespace:elastic-infra node:k8s-worker-02.example.com pod:nginx-ingress-controller-67f88bb59c-7gg7s]} {0xc0007b1080 map[namespace:kube-system node:k8s-worker-02.example.com pod:prometheus-metrics-adapter-c74fd5b66-ctq89]} {0xc0007b10f0 map[namespace:kube-system node:k8s-worker-02.example.com pod:sealed-secrets-controller-56c5d66bcc-qdf25]} {0xc0007b1160 map[namespace:kube-system node:k8s-worker-02.example.com pod:prometheus-operator-f64dd6c56-czt6k]} {0xc0007b11d0 map[namespace:kube-system node:k8s-worker-02.example.com pod:metrics-server-78df7f98c4-lrg4z]} {0xc0007b1240 map[namespace:rook-ceph-dev-primary node:k8s-worker-02.example.com pod:rook-ceph-dev-primary-backup-lt4b2]} {0xc0007b1320 map[namespace:rook-ceph-dev-primary node:k8s-worker-02.example.com pod:rook-discover-t2q4z]} {0xc0007b1390 map[namespace:velero node:k8s-worker-02.example.com pod:restic-szzmd]} {0xc0007b1870 map[node:k8s-worker-02.example.com type:ipipTunnelAddress]}] k8s-worker-20.example.com:[{0xc0000265c0 map[namespace:kube-system node:k8s-worker-20.example.com pod:nginx-ingress-controller-97bb49f5-hkk76]} {0xc000026650 map[namespace:monitoring-dev node:k8s-worker-20.example.com pod:nginx-ingress-controller-565c5c46c-xsnbq]} {0xc0000266e0 map[namespace:rook-ceph-dev-primary node:k8s-worker-20.example.com pod:rook-discover-qsv27]} {0xc000026750 map[namespace:couchbase-dev node:k8s-worker-20.example.com pod:nginx-ingress-controller-7885d48697-2zmfd]} {0xc0007b1e60 map[namespace:monitoring-infra node:k8s-worker-20.example.com pod:kube-state-metrics-5b587b5c9-gxqpt]} {0xc0003be2a0 map[namespace:kube-system node:k8s-worker-20.example.com pod:kube-iptables-tailer-rhj8p]} {0xc0003be330 map[namespace:rook-ceph-dev-primary node:k8s-worker-20.example.com pod:rook-ceph-operator-6c656d4995-55lbd]} {0xc0003be3a0 map[namespace:elastic-infra node:k8s-worker-20.example.com pod:thanos-querier-dc886459b-bbl4g]} {0xc0003be460 map[namespace:lxos-infra node:k8s-worker-20.example.com pod:alertmanager-infra-alertmanager-0]} {0xc0003be580 map[namespace:esdata-dev node:k8s-worker-20.example.com pod:nginx-ingress-controller-5499bc654c-vd6qw]} {0xc0003be630 map[namespace:rook-ceph-dev-primary node:k8s-worker-20.example.com pod:rook-ceph-tools-79bd8fdfc5-q7xh2]} {0xc0003be6a0 map[namespace:lxos-dev node:k8s-worker-20.example.com pod:hello-kubernetes-759f5dd94-brfqm]} {0xc0007b19f0 map[namespace:elastic-dev node:k8s-worker-20.example.com pod:default-http-backend-5f65599f44-6rzvk]} {0xc0007b1a60 map[namespace:couchbase-infra node:k8s-worker-20.example.com pod:thanos-querier-bdccc947d-4dz7w]} {0xc0007b1bb0 map[namespace:elastic-infra node:k8s-worker-20.example.com pod:thanos-compactor-0]} {0xc0007b1c20 map[namespace:monitoring-infra node:k8s-worker-20.example.com pod:thanos-querier-58fbb5b7d4-j9sz6]} {0xc0007b1ad0 map[namespace:esdata-infra node:k8s-worker-20.example.com pod:nginx-ingress-controller-75b5c6c5fc-5ggm5]} {0xc0007b1b40 map[namespace:gitlab-infra node:k8s-worker-20.example.com pod:kube-state-metrics-7fc464478b-vg9p2]} {0xc0007b1c90 map[namespace:esdata-infra node:k8s-worker-20.example.com pod:alertmanager-infra-alertmanager-0]} {0xc0007b1d10 map[namespace:elastic-infra node:k8s-worker-20.example.com pod:alertmanager-infra-alertmanager-0]} {0xc0007b1d80 map[namespace:kube-system node:k8s-worker-20.example.com pod:expire-check-worker-ghpx2]} {0xc0007b1df0 map[namespace:rook-ceph-dev-primary node:k8s-worker-20.example.com pod:rook-ceph-dev-primary-backup-bdjdf]} {0xc0007b1ed0 map[namespace:elastic-infra node:k8s-worker-20.example.com pod:kube-state-metrics-6dd9ddfcb8-tzzrd]} {0xc0007b1f40 map[namespace:couchbase-dev node:k8s-worker-20.example.com pod:nginx-ingress-controller-7885d48697-2f8hh]} {0xc0007b1fb0 map[namespace:monitoring-infra node:k8s-worker-20.example.com pod:alertmanager-infra-alertmanager-0]} {0xc0003be080 map[namespace:monitoring node:k8s-worker-20.example.com pod:nginx-ingress-controller-859d5cf8f-5k4gr]} {0xc0003be120 map[namespace:velero node:k8s-worker-20.example.com pod:restic-fw4m6]} {0xc0003be1a0 map[namespace:esdata-infra node:k8s-worker-20.example.com pod:thanos-compactor-0]} {0xc0003be230 map[namespace:nuclio-dev node:k8s-worker-20.example.com pod:nginx-ingress-controller-754b5bd7f5-8hqfb]} {0xc0003bea10 map[namespace:nuclio-infra node:k8s-worker-20.example.com pod:thanos-querier-5d5b8f467d-8vp2r]} {0xc0003be750 map[namespace:kube-system node:k8s-worker-20.example.com pod:external-dns-5c48cc494b-g4rnj]} {0xc0003be7f0 map[namespace:elastic-dev node:k8s-worker-20.example.com pod:filebeat-syslog-5fd9947874-tr4g2]} {0xc0003be870 map[namespace:elastic-system node:k8s-worker-20.example.com pod:elastic-operator-0]} {0xc0003be910 map[namespace:prometheus node:k8s-worker-20.example.com pod:master-kube-state-metrics-7f4f8dfd55-fgjpx]} {0xc0003be990 map[namespace:catalog node:k8s-worker-20.example.com pod:catalog-catalog-controller-manager-75d98fbc4d-lbzhj]} {0xc0003bea80 map[namespace:gitlab-dev node:k8s-worker-20.example.com pod:default-http-backend-5f65599f44-jgmnn]} {0xc0003beaf0 map[namespace:rene-dev node:k8s-worker-20.example.com pod:nginx-ingress-controller-7c9b58b755-q8qdz]} {0xc0003beb60 map[node:k8s-worker-20.example.com type:ipipTunnelAddress]} {0xc0003bebd0 map[namespace:velero node:k8s-worker-20.example.com pod:restic-2m6q8]}] k8s-worker-21.example.com:[{0xc000026820 map[namespace:rook-ceph-dev-primary node:k8s-worker-21.example.com pod:rook-ceph-operator-6c656d4995-trd4q]} {0xc000026890 map[namespace:nuclio-dev node:k8s-worker-21.example.com pod:nginx-ingress-controller-754b5bd7f5-sgtm7]} {0xc000026900 map[namespace:velero node:k8s-worker-21.example.com pod:restic-46rnw]} {0xc000026970 map[namespace:lxos-dev node:k8s-worker-21.example.com pod:nginx-ingress-controller-f95cb4d99-mghbj]} {0xc0003bf620 map[namespace:kube-system node:k8s-worker-21.example.com pod:kube-iptables-tailer-fndt6]} {0xc0003becc0 map[namespace:elastic-infra node:k8s-worker-21.example.com pod:nginx-ingress-controller-67f88bb59c-x8tc5]} {0xc0003bed40 map[namespace:elastic-infra node:k8s-worker-21.example.com pod:thanos-querier-dc886459b-pd5xf]} {0xc0003bedb0 map[namespace:elastic-poc-dev node:k8s-worker-21.example.com pod:nginx-ingress-controller-cc5c9777c-scfcw]} {0xc0003bee20 map[namespace:esdata-dev node:k8s-worker-21.example.com pod:nginx-ingress-controller-5499bc654c-999sp]} {0xc0003bee90 map[namespace:gitlab-infra node:k8s-worker-21.example.com pod:default-http-backend-5f65599f44-nm5mj]} {0xc0003bef10 map[namespace:esdata-infra node:k8s-worker-21.example.com pod:thanos-querier-5f64bc4d76-z45zl]} {0xc0003befa0 map[namespace:gitlab-infra node:k8s-worker-21.example.com pod:thanos-querier-78b5dbf4c6-jhxv8]} {0xc0003bf060 map[namespace:kube-system node:k8s-worker-21.example.com pod:kube-ops-view-695447b465-wdsz4]} {0xc0003bf190 map[namespace:clair-infra node:k8s-worker-21.example.com pod:thanos-querier-85f59b5b87-fntw7]} {0xc0003bf2a0 map[namespace:gitlab-infra node:k8s-worker-21.example.com pod:thanos-querier-78b5dbf4c6-fv6n9]} {0xc0003bf310 map[namespace:lxos-infra node:k8s-worker-21.example.com pod:alertmanager-infra-alertmanager-1]} {0xc0003bf380 map[namespace:nuclio-infra node:k8s-worker-21.example.com pod:alertmanager-infra-alertmanager-0]} {0xc0003bf3f0 map[namespace:monitoring-infra node:k8s-worker-21.example.com pod:thanos-compactor-0]} {0xc0003bf460 map[namespace:rene-dev node:k8s-worker-21.example.com pod:nginx-ingress-controller-7c9b58b755-5vrbm]} {0xc0003bf4d0 map[namespace:kube-system node:k8s-worker-21.example.com pod:expire-check-worker-p5ff9]} {0xc0003bf540 map[namespace:monitoring-dev node:k8s-worker-21.example.com pod:nginx-ingress-controller-565c5c46c-4tg5n]} {0xc0003bf5b0 map[namespace:rook-ceph-dev-primary node:k8s-worker-21.example.com pod:rook-discover-cqmdn]} {0xc0003bf6c0 map[namespace:rook-ceph-dev-primary node:k8s-worker-21.example.com pod:rook-ceph-dev-primary-backup-tv248]} {0xc0003bf740 map[namespace:lxos-infra node:k8s-worker-21.example.com pod:testing-6c89dc88f-w7zrf]} {0xc0003bf7c0 map[namespace:nuclio-infra node:k8s-worker-21.example.com pod:thanos-querier-5d5b8f467d-zsrmm]} {0xc0003bf850 map[namespace:velero node:k8s-worker-21.example.com pod:restic-krbsh]} {0xc0003bf8d0 map[namespace:gitlab-infra node:k8s-worker-21.example.com pod:alertmanager-infra-alertmanager-1]} {0xc0003bf9c0 map[namespace:monitoring-infra node:k8s-worker-21.example.com pod:default-http-backend-5f65599f44-hpjsd]} {0xc0003bfa70 map[namespace:gitlab-infra node:k8s-worker-21.example.com pod:thanos-compactor-0]} {0xc0003bfaf0 map[namespace:lxos-infra node:k8s-worker-21.example.com pod:thanos-compactor-0]} {0xc0003bfb90 map[namespace:couchbase-infra node:k8s-worker-21.example.com pod:thanos-compactor-0]} {0xc0003bfc00 map[namespace:elastic-infra node:k8s-worker-21.example.com pod:alertmanager-infra-alertmanager-1]} {0xc0003bfc70 map[namespace:couchbase-infra node:k8s-worker-21.example.com pod:alertmanager-infra-alertmanager-0]} {0xc0003bfcf0 map[namespace:nuclio-infra node:k8s-worker-21.example.com pod:nginx-ingress-controller-85c85d9b85-wssjj]} {0xc0003bfdb0 map[node:k8s-worker-21.example.com type:ipipTunnelAddress]} {0xc0003bfe20 map[namespace:velero node:k8s-worker-21.example.com pod:restic-bws62]}] k8s-worker-22.example.com:[{0xc0005b2630 map[namespace:velero node:k8s-worker-22.example.com pod:restic-srqrv]} {0xc0005b2320 map[namespace:cert-manager node:k8s-worker-22.example.com pod:cert-manager-webhook-8c5db9fb6-pgvhq]} {0xc0005b2390 map[namespace:clair-dev node:k8s-worker-22.example.com pod:default-http-backend-5f65599f44-4xmgt]} {0xc0005b29b0 map[namespace:nuclio-infra node:k8s-worker-22.example.com pod:nginx-ingress-controller-85c85d9b85-fm7cf]} {0xc0005b2400 map[namespace:catalog node:k8s-worker-22.example.com pod:catalog-catalog-controller-manager-75d98fbc4d-mslbq]} {0xc0005b2470 map[namespace:clair-infra node:k8s-worker-22.example.com pod:kube-state-metrics-765fdbd9c-2nsbm]} {0xc0005b24e0 map[namespace:clair-infra node:k8s-worker-22.example.com pod:default-http-backend-5f65599f44-bs9fp]} {0xc0005b2550 map[namespace:clair-infra node:k8s-worker-22.example.com pod:thanos-querier-85f59b5b87-2ctcd]} {0xc0003bff30 map[namespace:clair-infra node:k8s-worker-22.example.com pod:nginx-ingress-controller-5ccfdfd797-8h9gf]} {0xc0003bffa0 map[namespace:couchbase-dev node:k8s-worker-22.example.com pod:default-http-backend-5f65599f44-cm5cf]} {0xc0005b25c0 map[namespace:prometheus node:k8s-worker-22.example.com pod:prometheus-master-prometheus-operator-prometheus-0]} {0xc0005b2010 map[namespace:couchbase-dev node:k8s-worker-22.example.com pod:nginx-ingress-controller-7885d48697-jqgjv]} {0xc0005b2080 map[namespace:couchbase-infra node:k8s-worker-22.example.com pod:thanos-querier-bdccc947d-5zj5w]} {0xc0005b20f0 map[namespace:couchbase-infra node:k8s-worker-22.example.com pod:default-http-backend-5f65599f44-jf6sx]} {0xc0005b2160 map[namespace:elastic-poc-dev node:k8s-worker-22.example.com pod:quickstart-beta1-kb-66c64578bc-mhpbp]} {0xc0005b21d0 map[namespace:clair-infra node:k8s-worker-22.example.com pod:alertmanager-infra-alertmanager-0]} {0xc0005b2240 map[namespace:rook-ceph-dev-primary node:k8s-worker-22.example.com pod:rook-ceph-dev-primary-backup-ntzl4]} {0xc0005b22b0 map[namespace:gitlab-infra node:k8s-worker-22.example.com pod:thanos-querier-78b5dbf4c6-g44px]} {0xc0005b26a0 map[namespace:monitoring-infra node:k8s-worker-22.example.com pod:thanos-querier-58fbb5b7d4-lvgdk]} {0xc0005b2710 map[namespace:kube-system node:k8s-worker-22.example.com pod:kube-iptables-tailer-n8bgr]} {0xc0005b2780 map[namespace:lxos-dev node:k8s-worker-22.example.com pod:hello-kubernetes-759f5dd94-876tl]} {0xc0005b27f0 map[namespace:kube-system node:k8s-worker-22.example.com pod:expire-check-worker-vn8f6]} {0xc0005b2860 map[namespace:rene-dev node:k8s-worker-22.example.com pod:nginx-ingress-controller-7c9b58b755-d8mp7]} {0xc0005b28d0 map[namespace:clair-infra node:k8s-worker-22.example.com pod:alertmanager-infra-alertmanager-0]} {0xc0005b2940 map[namespace:lxos-dev node:k8s-worker-22.example.com pod:nginx-ingress-controller-f95cb4d99-4m6rz]} {0xc0005b2a20 map[namespace:couchbase-infra node:k8s-worker-22.example.com pod:thanos-querier-bdccc947d-hpq4x]} {0xc0005b2a90 map[namespace:elastic-dev node:k8s-worker-22.example.com pod:nginx-ingress-controller-7c8c64fd89-7gj5n]} {0xc0005b2b00 map[namespace:gitlab-dev node:k8s-worker-22.example.com pod:nginx-ingress-controller-85d6f6cd6c-hqx66]} {0xc0005b2b70 map[namespace:clair-dev node:k8s-worker-22.example.com pod:default-http-backend-5f65599f44-6wktr]} {0xc0005b2be0 map[node:k8s-worker-22.example.com type:ipipTunnelAddress]} {0xc0005b2c30 map[namespace:velero node:k8s-worker-22.example.com pod:restic-zxcsx]}] k8s-worker-23.example.com:[{0xc000026a40 map[namespace:kube-system node:k8s-worker-23.example.com pod:expire-check-worker-h7tmb]} {0xc0005b2d00 map[namespace:aws-service-broker node:k8s-worker-23.example.com pod:aws-servicebroker-fc58f95bb-bpb94]} {0xc0005b2d70 map[namespace:cert-manager node:k8s-worker-23.example.com pod:cert-manager-9dd6685b9-cm5g2]} {0xc0005b2de0 map[namespace:cert-manager node:k8s-worker-23.example.com pod:cert-manager-cainjector-8545fdf87c-5hczm]} {0xc0005b2e50 map[namespace:clair-infra node:k8s-worker-23.example.com pod:thanos-querier-85f59b5b87-xwqwb]} {0xc0005b2ec0 map[namespace:clair-dev node:k8s-worker-23.example.com pod:nginx-ingress-controller-866dd75bb9-q7jgm]} {0xc0005b2f30 map[namespace:elastic-dev node:k8s-worker-23.example.com pod:filebeat-syslog-5fd9947874-tb988]} {0xc0005b2fa0 map[namespace:elastic-dev node:k8s-worker-23.example.com pod:nginx-ingress-controller-7c8c64fd89-bmkr4]} {0xc0005b3160 map[namespace:nuclio-infra node:k8s-worker-23.example.com pod:kube-state-metrics-597b7c5f5-kmgtv]} {0xc0005b3010 map[namespace:velero node:k8s-worker-23.example.com pod:restic-qk574]} {0xc0005b3080 map[namespace:lxos-infra node:k8s-worker-23.example.com pod:thanos-querier-558cf4654d-qhnzc]} {0xc0005b30f0 map[namespace:kube-system node:k8s-worker-23.example.com pod:expire-check-worker-8nf9h]} {0xc0005b31d0 map[namespace:kube-system node:k8s-worker-23.example.com pod:tiller-deploy-786bc89dd8-h8lpg]} {0xc0005b3240 map[namespace:nuclio-infra node:k8s-worker-23.example.com pod:default-http-backend-5f65599f44-mrdmz]} {0xc0005b32b0 map[namespace:snyk-monitor node:k8s-worker-23.example.com pod:snyk-monitor-6fc8c86d4c-w6rxl]} {0xc0005b3320 map[namespace:kube-system node:k8s-worker-23.example.com pod:prometheus-metrics-adapter-c74fd5b66-fk9p4]} {0xc0005b3390 map[namespace:monitoring node:k8s-worker-23.example.com pod:mon-prometheus-pushgateway-7987d7d4b6-jdjwr]} {0xc0005b3400 map[namespace:rook-ceph-dev-primary node:k8s-worker-23.example.com pod:rook-discover-x2jcc]} {0xc0005b3470 map[namespace:rook-ceph-dev-primary node:k8s-worker-23.example.com pod:rook-ceph-dev-primary-backup-kp5pt]} {0xc0005b34e0 map[namespace:kube-system node:k8s-worker-23.example.com pod:kube-iptables-tailer-ckml4]} {0xc0005b3550 map[namespace:kube-system node:k8s-worker-23.example.com pod:expire-check-worker-jx7dd]} {0xc0005b35c0 map[namespace:velero node:k8s-worker-23.example.com pod:restic-6fjgw]} {0xc0005b3630 map[node:k8s-worker-23.example.com type:ipipTunnelAddress]}] k8s-worker-24.example.com:[{0xc000026b10 map[namespace:elastic-poc-dev node:k8s-worker-24.example.com pod:nginx-ingress-controller-cc5c9777c-4f26k]} {0xc000026b80 map[namespace:kube-system node:k8s-worker-24.example.com pod:expire-check-worker-9p8nr]} {0xc000026bf0 map[namespace:rook-ceph-dev-primary node:k8s-worker-24.example.com pod:rook-discover-pkzkm]} {0xc000026c60 map[namespace:velero node:k8s-worker-24.example.com pod:restic-nphng]} {0xc0005b36e0 map[namespace:couchbase-dev node:k8s-worker-24.example.com pod:nginx-ingress-controller-7885d48697-z4zpw]} {0xc0005b3750 map[namespace:couchbase-infra node:k8s-worker-24.example.com pod:nginx-ingress-controller-8684cd764f-6n867]} {0xc0005b3ec0 map[namespace:esdata-infra node:k8s-worker-24.example.com pod:thanos-querier-5f64bc4d76-9vv75]} {0xc0005b37c0 map[namespace:elastic-dev node:k8s-worker-24.example.com pod:filebeat-syslog-5fd9947874-lj7bw]} {0xc0005b3830 map[namespace:elastic-infra node:k8s-worker-24.example.com pod:kube-state-metrics-6dd9ddfcb8-4dmlh]} {0xc0005b38a0 map[namespace:elastic-infra node:k8s-worker-24.example.com pod:thanos-querier-dc886459b-qvtzl]} {0xc0005b3910 map[namespace:elastic-poc-dev node:k8s-worker-24.example.com pod:nginx-ingress-controller-cc5c9777c-68wlk]} {0xc0005b3980 map[namespace:elastic-dev node:k8s-worker-24.example.com pod:metricbeat-state-54fc656854-hwb4t]} {0xc0005b39f0 map[namespace:nuclio-infra node:k8s-worker-24.example.com pod:alertmanager-infra-alertmanager-1]} {0xc0005b3a60 map[namespace:monitoring-infra node:k8s-worker-24.example.com pod:alertmanager-infra-alertmanager-1]} {0xc0005b3ad0 map[namespace:velero node:k8s-worker-24.example.com pod:velero-dev-676556f788-hg88t]} {0xc0005b3b40 map[namespace:kube-system node:k8s-worker-24.example.com pod:external-dns-top-zone-d6c8fccc5-mrf8w]} {0xc0005b3bb0 map[namespace:lxos-infra node:k8s-worker-24.example.com pod:nginx-ingress-controller-597d797d89-gxl9w]} {0xc0005b3c20 map[namespace:lxos-infra node:k8s-worker-24.example.com pod:thanos-store-gateway-0]} {0xc0005b3c90 map[namespace:kube-system node:k8s-worker-24.example.com pod:expire-check-worker-z95wx]} {0xc0005b3d00 map[namespace:velero node:k8s-worker-24.example.com pod:restic-b4dxp]} {0xc0005b3d70 map[namespace:lxos-infra node:k8s-worker-24.example.com pod:default-http-backend-5f65599f44-j8hfq]} {0xc0005b3de0 map[namespace:kube-system node:k8s-worker-24.example.com pod:nginx-ingress-controller-97bb49f5-xkgc5]} {0xc0005b3f30 map[namespace:gitlab-infra node:k8s-worker-24.example.com pod:default-http-backend-5f65599f44-r5d7t]} {0xc0005b3e50 map[namespace:rook-ceph-dev-primary node:k8s-worker-24.example.com pod:rook-ceph-dev-primary-backup-vwtql]} {0xc0005b3fa0 map[namespace:clair-infra node:k8s-worker-24.example.com pod:nginx-ingress-controller-5ccfdfd797-trnpw]} {0xc0005c2010 map[namespace:monitoring-infra node:k8s-worker-24.example.com pod:nginx-ingress-controller-77589b585-5h84q]} {0xc0005c2080 map[namespace:rook-ceph-dev-primary node:k8s-worker-24.example.com pod:rook-ceph-mgr-a-9b6c5f86-g7mjc]} {0xc0005c20f0 map[namespace:couchbase-infra node:k8s-worker-24.example.com pod:nginx-ingress-controller-8684cd764f-7n9xq]} {0xc0005c2160 map[node:k8s-worker-24.example.com type:ipipTunnelAddress]} {0xc0005c21b0 map[namespace:kube-system node:k8s-worker-24.example.com pod:till-centos-766b66565c-kjbd6]} {0xc0005c2220 map[namespace:elastic-dev node:k8s-worker-24.example.com pod:filebeat-syslog-b6b9547b6-v7jmb]} {0xc0005c2290 map[namespace:velero node:k8s-worker-24.example.com pod:restic-mlrlb]} {0xc0005c2300 map[namespace:monitoring node:k8s-worker-24.example.com pod:mon-grafana-669bd4b55f-dlq6p]}] k8s-worker-25.example.com:[{0xc000026d30 map[namespace:esdata-infra node:k8s-worker-25.example.com pod:nginx-ingress-controller-75b5c6c5fc-jlrgt]} {0xc000026da0 map[namespace:esdata-dev node:k8s-worker-25.example.com pod:nginx-ingress-controller-5499bc654c-wbs68]} {0xc000026e10 map[namespace:velero node:k8s-worker-25.example.com pod:restic-wmnz8]} {0xc000026e80 map[namespace:rene-dev node:k8s-worker-25.example.com pod:nginx-ingress-controller-7c9b58b755-m9r4w]} {0xc0005c2880 map[namespace:esdata-infra node:k8s-worker-25.example.com pod:nginx-ingress-controller-75b5c6c5fc-tcphr]} {0xc0005c2c70 map[namespace:monitoring-infra node:k8s-worker-25.example.com pod:nginx-ingress-controller-77589b585-6kqkn]} {0xc0005c2ce0 map[namespace:couchbase-infra node:k8s-worker-25.example.com pod:thanos-store-gateway-0]} {0xc0005c2d50 map[namespace:kube-system node:k8s-worker-25.example.com pod:expire-check-worker-tj7ns]} {0xc0005c2dc0 map[namespace:rook-ceph-dev-primary node:k8s-worker-25.example.com pod:rook-ceph-dev-primary-backup-4k7w7]} {0xc0005c2e30 map[namespace:couchbase-infra node:k8s-worker-25.example.com pod:thanos-querier-bdccc947d-7t2mr]} {0xc0005c2ea0 map[namespace:postgres-operator node:k8s-worker-25.example.com pod:postgres-operator-6c9ff7cff5-tjjwp]} {0xc0005c2f10 map[namespace:nuclio-infra node:k8s-worker-25.example.com pod:nginx-ingress-controller-85c85d9b85-rsbtx]} {0xc0005c2420 map[namespace:elastic-infra node:k8s-worker-25.example.com pod:default-http-backend-5f65599f44-nr7b5]} {0xc0005c2490 map[namespace:rene-dev node:k8s-worker-25.example.com pod:prometheus-pushgateway-57b5f95d94-hbnn9]} {0xc0005c2500 map[namespace:elastic-poc-dev node:k8s-worker-25.example.com pod:default-http-backend-5f65599f44-wf5zj]} {0xc0005c2570 map[namespace:elastic-infra node:k8s-worker-25.example.com pod:nginx-ingress-controller-67f88bb59c-mcld9]} {0xc0005c25e0 map[namespace:elastic-infra node:k8s-worker-25.example.com pod:thanos-querier-dc886459b-hhmzf]} {0xc0005c2650 map[namespace:monitoring-infra node:k8s-worker-25.example.com pod:thanos-store-gateway-0]} {0xc0005c26c0 map[namespace:elastic-poc-dev node:k8s-worker-25.example.com pod:quickstart-beta1-kb-66c64578bc-qd872]} {0xc0005c2730 map[namespace:gitlab-infra node:k8s-worker-25.example.com pod:thanos-querier-78b5dbf4c6-bgkpw]} {0xc0005c27a0 map[namespace:kube-system node:k8s-worker-25.example.com pod:kube-iptables-tailer-nlmj6]} {0xc0005c2810 map[namespace:elastic-dev node:k8s-worker-25.example.com pod:metricbeat-state-54fc656854-qfhsr]} {0xc0005c28f0 map[namespace:elastic-poc-dev node:k8s-worker-25.example.com pod:nginx-ingress-controller-cc5c9777c-gwp9f]} {0xc0005c2960 map[namespace:rook-ceph-dev-primary node:k8s-worker-25.example.com pod:rook-discover-zx586]} {0xc0005c29d0 map[namespace:elastic-dev node:k8s-worker-25.example.com pod:default-http-backend-5f65599f44-lchwr]} {0xc0005c2a40 map[namespace:esdata-infra node:k8s-worker-25.example.com pod:thanos-querier-5f64bc4d76-fclxc]} {0xc0005c2ab0 map[namespace:gitlab-dev node:k8s-worker-25.example.com pod:nginx-ingress-controller-85d6f6cd6c-qrhbb]} {0xc0005c2b20 map[namespace:velero node:k8s-worker-25.example.com pod:restic-f766f]} {0xc0005c2b90 map[namespace:elastic-dev node:k8s-worker-25.example.com pod:filebeat-syslog-5fd9947874-tkhvj]} {0xc0005c2c00 map[namespace:elastic-dev node:k8s-worker-25.example.com pod:nginx-ingress-controller-7c8c64fd89-766cg]} {0xc0005c2f80 map[node:k8s-worker-25.example.com type:ipipTunnelAddress]} {0xc0005c2fd0 map[namespace:velero node:k8s-worker-25.example.com pod:restic-btw7n]}] k8s-worker-26.example.com:[{0xc00058fea0 map[namespace:catalog node:k8s-worker-26.example.com pod:toolbox]} {0xc00058f240 map[namespace:rook-ceph-dev-primary node:k8s-worker-26.example.com pod:rook-ceph-dev-primary-backup-6p5rx]} {0xc00058f5d0 map[namespace:kube-system node:k8s-worker-26.example.com pod:kube-iptables-tailer-hlhdk]} {0xc00058f6b0 map[namespace:rook-ceph-dev-primary node:k8s-worker-26.example.com pod:rook-discover-d9wft]} {0xc00058f720 map[namespace:kube-system node:k8s-worker-26.example.com pod:expire-check-worker-6z24z]} {0xc00058fce0 map[namespace:elastic-infra node:k8s-worker-26.example.com pod:nginx-ingress-controller-67f88bb59c-h9qkz]} {0xc00058fd50 map[namespace:couchbase-dev node:k8s-worker-26.example.com pod:nginx-ingress-controller-7885d48697-s54sf]} {0xc00058fdc0 map[namespace:gitlab-infra node:k8s-worker-26.example.com pod:nginx-ingress-controller-9c448878c-k2r9m]} {0xc00058fab0 map[namespace:monitoring-infra node:k8s-worker-26.example.com pod:nginx-ingress-controller-77589b585-2fx4h]} {0xc00058fb20 map[namespace:monitoring-dev node:k8s-worker-26.example.com pod:nginx-ingress-controller-565c5c46c-kbbp6]} {0xc00058fb90 map[namespace:nuclio-infra node:k8s-worker-26.example.com pod:nginx-ingress-controller-85c85d9b85-zcskb]} {0xc00058fc00 map[namespace:monitoring-infra node:k8s-worker-26.example.com pod:thanos-compactor-0]} {0xc00058fc70 map[namespace:prometheus node:k8s-worker-26.example.com pod:alertmanager-master-prometheus-operator-alertmanager-0]} {0xc00058fe30 map[namespace:lxos-infra node:k8s-worker-26.example.com pod:alertmanager-infra-alertmanager-0]} {0xc0007b0010 map[namespace:kube-system node:k8s-worker-26.example.com pod:pdb-controller-5bdbcbd7bc-5w79g]} {0xc00058ff90 map[namespace:lxos-dev node:k8s-worker-26.example.com pod:hello-kubernetes-759f5dd94-fdc2d]} {0xc00058f790 map[namespace:lxos-infra node:k8s-worker-26.example.com pod:default-http-backend-5f65599f44-9fjl2]} {0xc00058f800 map[namespace:elastic-dev node:k8s-worker-26.example.com pod:default-http-backend-5f65599f44-qp6xc]} {0xc00058f870 map[namespace:kube-system node:k8s-worker-26.example.com pod:till-centos-766b66565c-brntw]} {0xc00058f8e0 map[namespace:velero node:k8s-worker-26.example.com pod:velero-dev-676556f788-zdbvx]} {0xc00058f950 map[namespace:esdata-infra node:k8s-worker-26.example.com pod:thanos-store-gateway-0]} {0xc00058f9c0 map[namespace:aws-service-broker node:k8s-worker-26.example.com pod:aws-service-broker-aws-servicebroker-5b4cc95898-6nfzg]} {0xc00058fa30 map[namespace:esdata-infra node:k8s-worker-26.example.com pod:thanos-compactor-0]} {0xc0007b00e0 map[namespace:lxos-dev node:k8s-worker-26.example.com pod:nginx-ingress-controller-8dcf8dc56-ggt5d]} {0xc00058f2b0 map[namespace:monitoring-infra node:k8s-worker-26.example.com pod:default-http-backend-5f65599f44-9bflc]} {0xc00058f320 map[namespace:elastic-infra node:k8s-worker-26.example.com pod:default-http-backend-5f65599f44-tkt97]} {0xc00058f390 map[namespace:monitoring node:k8s-worker-26.example.com pod:default-http-backend-5f65599f44-27bxx]} {0xc00058f400 map[namespace:nuclio-dev node:k8s-worker-26.example.com pod:default-http-backend-5f65599f44-dzp2z]} {0xc00058f470 map[namespace:rene-dev node:k8s-worker-26.example.com pod:default-http-backend-5f65599f44-8xvhw]} {0xc00058f4e0 map[namespace:nuclio-infra node:k8s-worker-26.example.com pod:default-http-backend-5f65599f44-9bhw8]} {0xc00058f550 map[namespace:lxos-infra node:k8s-worker-26.example.com pod:default-http-backend-5f65599f44-8866f]} {0xc00058f640 map[namespace:monitoring-dev node:k8s-worker-26.example.com pod:default-http-backend-5f65599f44-zd8z2]} {0xc00058ff20 map[namespace:velero node:k8s-worker-26.example.com pod:restic-vdbfx]} {0xc0005e6550 map[node:k8s-worker-26.example.com type:ipipTunnelAddress]}] k8s-worker-27.example.com:[{0xc00058e2b0 map[node:k8s-worker-27.example.com type:ipipTunnelAddress]} {0xc00058ed30 map[namespace:elastic-dev node:k8s-worker-27.example.com pod:filebeat-syslog-b6b9547b6-tkg8f]} {0xc00058e8b0 map[namespace:rook-ceph-dev-primary node:k8s-worker-27.example.com pod:rook-ceph-dev-primary-backup-svch6]} {0xc00058e920 map[namespace:kube-system node:k8s-worker-27.example.com pod:kube-iptables-tailer-pbmjm]} {0xc00058e990 map[namespace:kube-system node:k8s-worker-27.example.com pod:expire-check-worker-b8tb9]} {0xc00058ea10 map[namespace:rook-ceph-dev-primary node:k8s-worker-27.example.com pod:rook-discover-b9zgx]} {0xc00058ea80 map[namespace:monitoring-infra node:k8s-worker-27.example.com pod:default-http-backend-5f65599f44-5rgs9]} {0xc00058eaf0 map[namespace:monitoring node:k8s-worker-27.example.com pod:default-http-backend-5f65599f44-kgfpp]} {0xc00058eb60 map[namespace:nuclio-dev node:k8s-worker-27.example.com pod:default-http-backend-5f65599f44-nrb9d]} {0xc00058e300 map[namespace:monitoring-dev node:k8s-worker-27.example.com pod:default-http-backend-5f65599f44-9drvt]} {0xc00058e380 map[namespace:monitoring-infra node:k8s-worker-27.example.com pod:default-http-backend-5f65599f44-2zc2d]} {0xc00058e3f0 map[namespace:monitoring node:k8s-worker-27.example.com pod:default-http-backend-5f65599f44-6cphw]} {0xc00058e460 map[namespace:nuclio-dev node:k8s-worker-27.example.com pod:default-http-backend-5f65599f44-25j66]} {0xc00058e4e0 map[namespace:nuclio-infra node:k8s-worker-27.example.com pod:default-http-backend-5f65599f44-hwzw6]} {0xc00058e550 map[namespace:rene-dev node:k8s-worker-27.example.com pod:default-http-backend-5f65599f44-4f9bl]} {0xc00058e5c0 map[namespace:gitlab-infra node:k8s-worker-27.example.com pod:default-http-backend-5f65599f44-7tmx4]} {0xc00058e630 map[namespace:lxos-infra node:k8s-worker-27.example.com pod:default-http-backend-5f65599f44-tgtdh]} {0xc00058e6c0 map[namespace:elastic-infra node:k8s-worker-27.example.com pod:default-http-backend-5f65599f44-lp8bh]} {0xc00058e730 map[namespace:catalog node:k8s-worker-27.example.com pod:catalog-catalog-controller-manager-75d654cbf8-qs56m]} {0xc00058e7b0 map[namespace:couchbase-infra node:k8s-worker-27.example.com pod:thanos-store-gateway-0]} {0xc00058e840 map[namespace:prometheus node:k8s-worker-27.example.com pod:prometheus-master-prometheus-operator-prometheus-0]} {0xc00058ebd0 map[namespace:gitlab-infra node:k8s-worker-27.example.com pod:alertmanager-infra-alertmanager-0]} {0xc00058ec40 map[namespace:esdata-infra node:k8s-worker-27.example.com pod:alertmanager-infra-alertmanager-1]} {0xc00058ecb0 map[namespace:clair-infra node:k8s-worker-27.example.com pod:thanos-querier-85f59b5b87-b8kl7]} {0xc00058eda0 map[namespace:velero node:k8s-worker-27.example.com pod:restic-bm75w]}] k8s-worker-28.example.com:[{0xc0006d14d0 map[namespace:rook-ceph-dev-primary node:k8s-worker-28.example.com pod:rook-discover-8vc27]} {0xc0006d1770 map[namespace:kube-system node:k8s-worker-28.example.com pod:kube-iptables-tailer-wjgr9]} {0xc0006d18c0 map[namespace:kube-system node:k8s-worker-28.example.com pod:expire-check-worker-qt4qc]} {0xc0006d19a0 map[namespace:rook-ceph-dev-primary node:k8s-worker-28.example.com pod:rook-ceph-dev-primary-backup-5994v]} {0xc0006d1d20 map[namespace:nuclio-dev node:k8s-worker-28.example.com pod:default-http-backend-5f65599f44-c5v58]} {0xc0006d1d90 map[namespace:monitoring node:k8s-worker-28.example.com pod:default-http-backend-5f65599f44-9s9tz]} {0xc0006d1e00 map[namespace:monitoring-infra node:k8s-worker-28.example.com pod:default-http-backend-5f65599f44-qm4ks]} {0xc0006d1cb0 map[namespace:nuclio-infra node:k8s-worker-28.example.com pod:default-http-backend-5f65599f44-957vs]} {0xc0006d1a10 map[namespace:kube-system node:k8s-worker-28.example.com pod:coredns-66459df666-6xmv2]} {0xc0006d1a80 map[namespace:kube-system node:k8s-worker-28.example.com pod:coredns-66459df666-xrplq]} {0xc0006d1e70 map[namespace:velero node:k8s-worker-28.example.com pod:restic-7vklw]} {0xc0006d13f0 map[namespace:kube-system node:k8s-worker-28.example.com pod:coredns-7889b4f656-fkk96]} {0xc0006d1460 map[namespace:kube-system node:k8s-worker-28.example.com pod:coredns-7889b4f656-9zq9v]} {0xc0006d1af0 map[namespace:cert-manager node:k8s-worker-28.example.com pod:certmanager-cert-manager-webhook-7959675d59-92nr9]} {0xc0006d1b60 map[namespace:kube-system node:k8s-worker-28.example.com pod:prometheus-metrics-adapter-c74fd5b66-ng8s9]} {0xc0006d1bd0 map[namespace:elastic-poc-dev node:k8s-worker-28.example.com pod:quickstart-beta1-kb-66c64578bc-ps5qn]} {0xc0006d1c40 map[namespace:monitoring node:k8s-worker-28.example.com pod:mon-prometheus-server-577c57785c-vzl7s]} {0xc0006d1540 map[namespace:external-dns node:k8s-worker-28.example.com pod:top-zone-internal-external-dns-7cbcb75cd4-jvsxp]} {0xc0006d15b0 map[namespace:external-dns node:k8s-worker-28.example.com pod:eai-internal-external-dns-6d5bb7954b-w5mcr]} {0xc0006d1620 map[namespace:external-dns node:k8s-worker-28.example.com pod:default-internal-external-dns-5c88666655-kwxf6]} {0xc0006d1690 map[namespace:external-dns node:k8s-worker-28.example.com pod:default-external-external-dns-6b4dd4bf-2cs5f]} {0xc0006d1700 map[namespace:external-dns node:k8s-worker-28.example.com pod:eai-external-external-dns-685dbd8956-jbx5t]} {0xc0006d17e0 map[namespace:external-dns node:k8s-worker-28.example.com pod:top-zone-external-external-dns-c95c894fb-sbjx9]} {0xc0006d1850 map[namespace:gitlab-infra node:k8s-worker-28.example.com pod:default-http-backend-5f65599f44-4wfh4]} {0xc0006d1930 map[namespace:nuclio-dev node:k8s-worker-28.example.com pod:default-http-backend-5f65599f44-dsmcr]} {0xc0005e6a70 map[node:k8s-worker-28.example.com type:ipipTunnelAddress]}] k8s-worker-50.example.com:[{0xc0005dca70 map[namespace:elastic-dev node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-v7mrj]} {0xc0005dc3e0 map[namespace:nuclio-infra node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-wmclh]} {0xc0005c35e0 map[namespace:kube-system node:k8s-worker-50.example.com pod:expire-check-worker-f46gn]} {0xc0005c37a0 map[namespace:kube-system node:k8s-worker-50.example.com pod:kube-iptables-tailer-hvpsb]} {0xc0005c3b20 map[namespace:velero node:k8s-worker-50.example.com pod:restic-x5gq2]} {0xc0005dc220 map[namespace:monitoring-infra node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-xwhzx]} {0xc0005c3dc0 map[namespace:esdata-dev node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-96rch]} {0xc0005c3e30 map[namespace:monitoring node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-5vb7t]} {0xc0005c3d50 map[namespace:esdata-infra node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-9ds9x]} {0xc0005dc0d0 map[namespace:gitlab-dev node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-jldhd]} {0xc0005dc140 map[namespace:monitoring-dev node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-2fd8t]} {0xc0005c3f80 map[namespace:couchbase-dev node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-m8lcg]} {0xc0005dc1b0 map[namespace:lxos-infra node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-4n9xr]} {0xc0005dc300 map[namespace:nuclio-infra node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-lxppf]} {0xc0005dc370 map[namespace:elastic-poc-dev node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-xbv2n]} {0xc0005c3ff0 map[namespace:monitoring-infra node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-2nnj5]} {0xc0005dc290 map[namespace:nuclio-infra node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-sklxq]} {0xc0005dc060 map[namespace:elastic-infra node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-8fb4r]} {0xc0005c30a0 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-dev-primary-backup-t6295]} {0xc0005c3110 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-discover-c7jjw]} {0xc0005c3ea0 map[namespace:gitlab-infra node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-mkfw7]} {0xc0005c3f10 map[namespace:elastic-dev node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-flm24]} {0xc0005c3180 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-mon-b-6fb7fb76bf-nxnj8]} {0xc0005dc450 map[namespace:monitoring node:k8s-worker-50.example.com pod:mon-prometheus-alertmanager-65568b4666-t8645]} {0xc0005c31f0 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-5-75696f67b6-82sb5]} {0xc0005c3260 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-8-5bfc846db7-9dppx]} {0xc0005c32d0 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-11-5f5f79d5-cxgdb]} {0xc0005c3340 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-14-66f6bbb468-8bn94]} {0xc0005c33b0 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-2-758c4d4745-ngzl7]} {0xc0005c3420 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-17-9fcf66d95-xw97k]} {0xc0005c3490 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-20-565c64d8c7-q86zd]} {0xc0005c3500 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-23-7fbd986ffd-wvtvn]} {0xc0005dc4c0 map[namespace:gitlab-infra node:k8s-worker-50.example.com pod:gitlab-dev-chronograf-f559f8679-jf8bk]} {0xc0005dc530 map[namespace:prometheus node:k8s-worker-50.example.com pod:master-grafana-7d65f7df9b-tkl72]} {0xc0005dc5a0 map[namespace:kube-system node:k8s-worker-50.example.com pod:pdb-controller-5bdbcbd7bc-sp5ln]} {0xc0005dc610 map[namespace:elastic-poc-dev node:k8s-worker-50.example.com pod:nginx-ingress-controller-cc5c9777c-wsrdh]} {0xc0005c3570 map[namespace:prometheus node:k8s-worker-50.example.com pod:alertmanager-master-prometheus-operator-alertmanager-0]} {0xc0005dc680 map[namespace:esdata-dev node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-czwkg]} {0xc0005dc760 map[namespace:couchbase-infra node:k8s-worker-50.example.com pod:thanos-compactor-0]} {0xc0005dc6f0 map[namespace:lxos-dev node:k8s-worker-50.example.com pod:nginx-ingress-controller-f95cb4d99-cdmdl]} {0xc0005c3650 map[namespace:catalog node:k8s-worker-50.example.com pod:catalog-catalog-controller-manager-5f745fff9-vs9jt]} {0xc0005c36c0 map[namespace:catalog node:k8s-worker-50.example.com pod:catalog-catalog-webhook-779b57cb98-qtpfl]} {0xc0005c3730 map[namespace:aws-service-broker node:k8s-worker-50.example.com pod:aws-service-broker-aws-servicebroker-5b8765c8f7-tn5fq]} {0xc0005dc7d0 map[namespace:clair-dev node:k8s-worker-50.example.com pod:nginx-ingress-controller-866dd75bb9-8s79b]} {0xc0005dc840 map[namespace:catalog node:k8s-worker-50.example.com pod:catalog-catalog-controller-manager-75d654cbf8-cdg8s]} {0xc0005dc8b0 map[namespace:gitlab-infra node:k8s-worker-50.example.com pod:thanos-querier-78b5dbf4c6-zpzmv]} {0xc0005dc920 map[namespace:lxos-infra node:k8s-worker-50.example.com pod:thanos-querier-558cf4654d-kdfz4]} {0xc0005dc990 map[namespace:kube-system node:k8s-worker-50.example.com pod:node-labeler-786896fb46-kfh8g]} {0xc0005dca00 map[namespace:kube-system node:k8s-worker-50.example.com pod:prometheus-metrics-adapter-c74fd5b66-jr62t]} {0xc0005c3810 map[namespace:monitoring-infra node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-hsqkg]} {0xc0005dcae0 map[namespace:nuclio-infra node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-9q29s]} {0xc0005dcb50 map[namespace:velero node:k8s-worker-50.example.com pod:restic-4d4kk]} {0xc0005c3880 map[namespace:nuclio-dev node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-x6srg]} {0xc0005c38f0 map[namespace:nuclio-infra node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-sfvvj]} {0xc0005c3960 map[namespace:esdata-dev node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-4b4nz]} {0xc0005c39d0 map[namespace:gitlab-dev node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-grxrj]} {0xc0005c3a40 map[namespace:lxos-infra node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-75xcp]} {0xc0005c3ab0 map[namespace:monitoring node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-s2t65]} {0xc0005c3b90 map[namespace:kube-system node:k8s-worker-50.example.com pod:tiller-deploy-786bc89dd8-ff7x2]} {0xc0005c3c00 map[namespace:monitoring node:k8s-worker-50.example.com pod:mon-prometheus-kube-state-metrics-577dd9b866-nq66w]} {0xc0005c3c70 map[namespace:monitoring node:k8s-worker-50.example.com pod:mon-prometheus-pushgateway-7987d7d4b6-hbnbf]} {0xc0005c3ce0 map[namespace:kube-system node:k8s-worker-50.example.com pod:prometheus-metrics-adapter-c74fd5b66-pdnjn]} {0xc0005dcc20 map[namespace:nuclio-infra node:k8s-worker-50.example.com pod:thanos-querier-5d5b8f467d-jpdvg]} {0xc0005dcc90 map[namespace:lxos-dev node:k8s-worker-50.example.com pod:nginx-ingress-controller-f95cb4d99-vw5th]} {0xc0005dcd00 map[namespace:monitoring node:k8s-worker-50.example.com pod:mon-prometheus-alertmanager-65568b4666-k5sdr]} {0xc0005dcd70 map[namespace:kube-system node:k8s-worker-50.example.com pod:dashboard-metrics-scraper-664b58f47c-bx7sv]} {0xc0005dcde0 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-5-75696f67b6-gbfzs]} {0xc0005dceb0 map[namespace:couchbase-infra node:k8s-worker-50.example.com pod:alertmanager-infra-alertmanager-0]} {0xc0005dcf20 map[namespace:lxos-infra node:k8s-worker-50.example.com pod:nginx-ingress-controller-597d797d89-kzkj6]} {0xc0005dcf90 map[namespace:lxos-dev node:k8s-worker-50.example.com pod:hello-kubernetes-759f5dd94-p5ktk]} {0xc0005dd000 map[namespace:clair-infra node:k8s-worker-50.example.com pod:nginx-ingress-controller-5ccfdfd797-krfxd]} {0xc0005dd070 map[namespace:esdata-dev node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-w5cn2]} {0xc0005dd0e0 map[namespace:couchbase-infra node:k8s-worker-50.example.com pod:thanos-querier-bdccc947d-2m28g]} {0xc0005dd150 map[namespace:couchbase-infra node:k8s-worker-50.example.com pod:thanos-compactor-0]} {0xc0005dd1c0 map[namespace:kube-system node:k8s-worker-50.example.com pod:external-dns-5c48cc494b-lrzhv]} {0xc0005dd230 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-20-565c64d8c7-k4h9d]} {0xc0005dd2a0 map[namespace:couchbase-infra node:k8s-worker-50.example.com pod:nginx-ingress-controller-8684cd764f-frxk7]} {0xc0005dd310 map[namespace:lxos-infra node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-tr9cq]} {0xc0005dd380 map[namespace:monitoring-infra node:k8s-worker-50.example.com pod:thanos-querier-58fbb5b7d4-d6csf]} {0xc0005dd3f0 map[namespace:kube-system node:k8s-worker-50.example.com pod:external-dns-eai-687bb6d8d9-45lhl]} {0xc0005dd460 map[namespace:elastic-infra node:k8s-worker-50.example.com pod:thanos-compactor-0]} {0xc0005dd4d0 map[namespace:lxos-dev node:k8s-worker-50.example.com pod:vscode-6ccdbfc786-cvpnk]} {0xc0005dd540 map[namespace:nuclio-dev node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-ngtg4]} {0xc0005dd5b0 map[namespace:kube-system node:k8s-worker-50.example.com pod:kube-iptables-tailer-2ml7w]} {0xc0005dd620 map[namespace:esdata-infra node:k8s-worker-50.example.com pod:nginx-ingress-controller-75b5c6c5fc-l8pkz]} {0xc0005dd690 map[namespace:elastic-poc-dev node:k8s-worker-50.example.com pod:nginx-ingress-controller-cc5c9777c-5nbx6]} {0xc0005dd700 map[namespace:gitlab-dev node:k8s-worker-50.example.com pod:nginx-ingress-controller-85d6f6cd6c-hs47w]} {0xc0005dd770 map[namespace:nuclio-dev node:k8s-worker-50.example.com pod:nginx-ingress-controller-754b5bd7f5-trkl5]} {0xc0005dd7e0 map[namespace:kube-system node:k8s-worker-50.example.com pod:external-dns-top-zone-7c55c5667d-wfz62]} {0xc0005dd850 map[namespace:kube-system node:k8s-worker-50.example.com pod:external-dns-extern-eai-598c58c48d-b7t42]} {0xc0005dd8c0 map[namespace:lxos-infra node:k8s-worker-50.example.com pod:thanos-querier-558cf4654d-phjxv]} {0xc0005dd930 map[namespace:gitlab-infra node:k8s-worker-50.example.com pod:alertmanager-infra-alertmanager-1]} {0xc0005dd9a0 map[namespace:esdata-infra node:k8s-worker-50.example.com pod:thanos-querier-5f64bc4d76-5xdvm]} {0xc0005dda10 map[namespace:gitlab-infra node:k8s-worker-50.example.com pod:kube-state-metrics-cc9d654f4-g4bct]} {0xc0005dda80 map[namespace:elastic-poc-dev node:k8s-worker-50.example.com pod:nginx-ingress-controller-cc5c9777c-9hgsk]} {0xc0005ddaf0 map[namespace:kube-system node:k8s-worker-50.example.com pod:external-dns-extern-687dd5b476-5mm4s]} {0xc0005ddb60 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-11-5f5f79d5-xznb5]} {0xc0005ddbd0 map[namespace:nuclio-infra node:k8s-worker-50.example.com pod:alertmanager-infra-alertmanager-0]} {0xc0005ddc40 map[namespace:couchbase-infra node:k8s-worker-50.example.com pod:thanos-querier-bdccc947d-qgjw8]} {0xc0005ddcb0 map[namespace:esdata-infra node:k8s-worker-50.example.com pod:alertmanager-infra-alertmanager-0]} {0xc0005ddd20 map[namespace:esdata-infra node:k8s-worker-50.example.com pod:nginx-ingress-controller-75b5c6c5fc-89t28]} {0xc0005ddd90 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-2-758c4d4745-2m98v]} {0xc0005dde00 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-dev-primary-backup-rmmdm]} {0xc0005dde70 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-14-66f6bbb468-p5gt6]} {0xc0005ddee0 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-discover-qcpsv]} {0xc0005ddf50 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-mon-b-6fb7fb76bf-djvms]} {0xc0005ddfc0 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-17-9fcf66d95-5jkmw]} {0xc0005e6030 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-23-7fbd986ffd-hsrd8]} {0xc0005e60a0 map[namespace:rook-ceph-dev-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-8-5bfc846db7-vx4xn]} {0xc0005e6170 map[namespace:couchbase-infra node:k8s-worker-50.example.com pod:default-http-backend-5f65599f44-lwtq8]} {0xc0005e61e0 map[namespace:lxos-dev node:k8s-worker-50.example.com pod:vscode-6ccdbfc786-54gpx]} {0xc0005e6250 map[namespace:esdata-infra node:k8s-worker-50.example.com pod:thanos-querier-5f64bc4d76-4tdhx]} {0xc0005e62c0 map[namespace:lxos-infra node:k8s-worker-50.example.com pod:testing-6c89dc88f-8rgss]} {0xc0005e6330 map[namespace:elastic-infra node:k8s-worker-50.example.com pod:thanos-querier-dc886459b-vg9cv]} {0xc0005e63a0 map[namespace:prometheus node:k8s-worker-50.example.com pod:master-prometheus-operator-operator-cff4b9c67-nn5vf]} {0xc0005e6410 map[namespace:elastic-dev node:k8s-worker-50.example.com pod:filebeat-syslog-645bc9699c-5jfv7]} {0xc0005e6480 map[namespace:elastic-dev node:k8s-worker-50.example.com pod:filebeat-hqjp2]} {0xc0005e6650 map[node:k8s-worker-50.example.com type:ipipTunnelAddress]}] k8s-worker-51.example.com:[{0xc0009b6600 map[namespace:velero node:k8s-worker-51.example.com pod:restic-mn628]} {0xc000174dd0 map[namespace:kube-system node:k8s-worker-51.example.com pod:expire-check-worker-x9kfl]} {0xc000175090 map[namespace:kube-system node:k8s-worker-51.example.com pod:kube-iptables-tailer-6zkmf]} {0xc000175fe0 map[namespace:couchbase-infra node:k8s-worker-51.example.com pod:nginx-ingress-controller-8684cd764f-mmblg]} {0xc000175f70 map[namespace:esdata-infra node:k8s-worker-51.example.com pod:nginx-ingress-controller-75b5c6c5fc-dw7rp]} {0xc000175790 map[namespace:velero node:k8s-worker-51.example.com pod:restic-sznrc]} {0xc000175f00 map[namespace:esdata-infra node:k8s-worker-51.example.com pod:nginx-ingress-controller-75b5c6c5fc-d8bks]} {0xc000175a30 map[namespace:couchbase-dev node:k8s-worker-51.example.com pod:nginx-ingress-controller-7885d48697-45ssq]} {0xc000175aa0 map[namespace:elastic-infra node:k8s-worker-51.example.com pod:nginx-ingress-controller-67f88bb59c-q9jj2]} {0xc0001758e0 map[namespace:esdata-dev node:k8s-worker-51.example.com pod:nginx-ingress-controller-5499bc654c-4j5hr]} {0xc000174a50 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-dev-primary-backup-mlgh7]} {0xc000174ac0 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-discover-6b5g7]} {0xc000174b30 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-tools-57644c8849-rrdfc]} {0xc000174ba0 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-mon-c-65b595c856-fcpxm]} {0xc000175950 map[namespace:elastic-dev node:k8s-worker-51.example.com pod:nginx-ingress-controller-7c8c64fd89-brc5k]} {0xc000174c10 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-18-5b9fd8d676-j5m8z]} {0xc000174c80 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-21-74fb68cd9b-p84cl]} {0xc000174cf0 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-6-85cdc457-2qjrt]} {0xc000174d60 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-3-6ccc57dff6-zp6tg]} {0xc000174e40 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-0-6fb44c7675-4fxdj]} {0xc000174f30 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-9-7d7d959968-8rxt8]} {0xc000174fb0 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-15-58b74cfb89-q872m]} {0xc000175020 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-12-5858886678-bhzf6]} {0xc0001759c0 map[namespace:monitoring node:k8s-worker-51.example.com pod:nginx-ingress-controller-859d5cf8f-r5jqb]} {0xc000175b10 map[namespace:clair-infra node:k8s-worker-51.example.com pod:nginx-ingress-controller-5ccfdfd797-l2dvs]} {0xc000175b80 map[namespace:clair-infra node:k8s-worker-51.example.com pod:thanos-querier-85f59b5b87-wsncq]} {0xc000175bf0 map[namespace:elastic-infra node:k8s-worker-51.example.com pod:default-http-backend-5f65599f44-jwhjg]} {0xc000175c60 map[namespace:couchbase-dev node:k8s-worker-51.example.com pod:nginx-ingress-controller-7885d48697-4zn2x]} {0xc000175cd0 map[namespace:couchbase-infra node:k8s-worker-51.example.com pod:thanos-querier-bdccc947d-7bzwp]} {0xc000175d40 map[namespace:elastic-poc-dev node:k8s-worker-51.example.com pod:nginx-ingress-controller-cc5c9777c-7g96f]} {0xc000175db0 map[namespace:monitoring-dev node:k8s-worker-51.example.com pod:nginx-ingress-controller-565c5c46c-2qg6g]} {0xc000175e20 map[namespace:monitoring node:k8s-worker-51.example.com pod:default-http-backend-5f65599f44-5lhg8]} {0xc000175e90 map[namespace:gitlab-dev node:k8s-worker-51.example.com pod:nginx-ingress-controller-85d6f6cd6c-75qfk]} {0xc0009b6050 map[namespace:nuclio-dev node:k8s-worker-51.example.com pod:nginx-ingress-controller-754b5bd7f5-5qfbd]} {0xc0009b60c0 map[namespace:couchbase-dev node:k8s-worker-51.example.com pod:nginx-ingress-controller-7885d48697-qn7cv]} {0xc0009b6130 map[namespace:gitlab-infra node:k8s-worker-51.example.com pod:nginx-ingress-controller-9c448878c-jc4dv]} {0xc0009b61a0 map[namespace:kube-system node:k8s-worker-51.example.com pod:k8s-toolbox-6958cf74dd-l2mvv]} {0xc0009b6210 map[namespace:couchbase-infra node:k8s-worker-51.example.com pod:thanos-querier-bdccc947d-rxgzs]} {0xc0009b6280 map[namespace:couchbase-infra node:k8s-worker-51.example.com pod:nginx-ingress-controller-8684cd764f-tcf4p]} {0xc0009b62f0 map[namespace:kube-system node:k8s-worker-51.example.com pod:prometheus-metrics-adapter-c74fd5b66-kn8fv]} {0xc0009b6360 map[namespace:elastic-infra node:k8s-worker-51.example.com pod:nginx-ingress-controller-67f88bb59c-6x95w]} {0xc0009b63d0 map[namespace:monitoring-infra node:k8s-worker-51.example.com pod:nginx-ingress-controller-77589b585-p2c59]} {0xc0009b6440 map[namespace:elastic-poc-dev node:k8s-worker-51.example.com pod:nginx-ingress-controller-cc5c9777c-pmhsb]} {0xc0009b64b0 map[namespace:monitoring-infra node:k8s-worker-51.example.com pod:default-http-backend-5f65599f44-gzh68]} {0xc000175100 map[namespace:elastic-infra node:k8s-worker-51.example.com pod:default-http-backend-5f65599f44-6stfb]} {0xc000175170 map[namespace:lxos-dev node:k8s-worker-51.example.com pod:default-http-backend-5f65599f44-m2l5s]} {0xc0001751e0 map[namespace:elastic-dev node:k8s-worker-51.example.com pod:default-http-backend-5f65599f44-kk6tp]} {0xc000175250 map[namespace:esdata-infra node:k8s-worker-51.example.com pod:default-http-backend-5f65599f44-gfml8]} {0xc0001752c0 map[namespace:gitlab-infra node:k8s-worker-51.example.com pod:default-http-backend-5f65599f44-4jwgh]} {0xc000175330 map[namespace:monitoring-dev node:k8s-worker-51.example.com pod:default-http-backend-5f65599f44-2scqx]} {0xc0001753a0 map[namespace:rene-dev node:k8s-worker-51.example.com pod:default-http-backend-5f65599f44-t82hj]} {0xc000175410 map[namespace:clair-infra node:k8s-worker-51.example.com pod:thanos-querier-85f59b5b87-b9lgk]} {0xc000175480 map[namespace:couchbase-infra node:k8s-worker-51.example.com pod:thanos-querier-bdccc947d-ghdph]} {0xc0001754f0 map[namespace:nuclio-infra node:k8s-worker-51.example.com pod:thanos-querier-5d5b8f467d-qf42w]} {0xc000175560 map[namespace:lxos-infra node:k8s-worker-51.example.com pod:nginx-ingress-controller-597d797d89-d5d22]} {0xc0001755d0 map[namespace:esdata-infra node:k8s-worker-51.example.com pod:thanos-querier-5f64bc4d76-9bpdk]} {0xc000175640 map[namespace:monitoring-infra node:k8s-worker-51.example.com pod:thanos-querier-58fbb5b7d4-4mzwl]} {0xc0001756b0 map[namespace:rene-dev node:k8s-worker-51.example.com pod:nginx-ingress-controller-7c9b58b755-bvxf8]} {0xc000175720 map[namespace:lxos-infra node:k8s-worker-51.example.com pod:thanos-querier-558cf4654d-fxdm2]} {0xc000175800 map[namespace:clair-infra node:k8s-worker-51.example.com pod:nginx-ingress-controller-5ccfdfd797-tcspg]} {0xc000175870 map[namespace:lxos-dev node:k8s-worker-51.example.com pod:nginx-ingress-controller-f95cb4d99-cqmwg]} {0xc0009b6520 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-15-58b74cfb89-wtv5d]} {0xc0009b6590 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-21-74fb68cd9b-695v6]} {0xc0009b66d0 map[namespace:esdata-infra node:k8s-worker-51.example.com pod:thanos-compactor-0]} {0xc0009b6740 map[namespace:rene-dev node:k8s-worker-51.example.com pod:nginx-ingress-controller-7c9b58b755-7skz4]} {0xc0009b67b0 map[namespace:kube-system node:k8s-worker-51.example.com pod:sealed-secrets-controller-56c5d66bcc-zkgdv]} {0xc0009b6820 map[namespace:kube-system node:k8s-worker-51.example.com pod:expire-check-worker-94gpb]} {0xc0009b6890 map[namespace:lxos-dev node:k8s-worker-51.example.com pod:nginx-ingress-controller-f95cb4d99-crqd7]} {0xc0009b6900 map[namespace:monitoring-dev node:k8s-worker-51.example.com pod:nginx-ingress-controller-565c5c46c-wbwq2]} {0xc0009b6970 map[namespace:gitlab-infra node:k8s-worker-51.example.com pod:thanos-compactor-0]} {0xc0009b69e0 map[namespace:elastic-infra node:k8s-worker-51.example.com pod:alertmanager-infra-alertmanager-0]} {0xc0009b6a50 map[namespace:lxos-infra node:k8s-worker-51.example.com pod:alertmanager-infra-alertmanager-0]} {0xc0009b6ac0 map[namespace:rene-dev node:k8s-worker-51.example.com pod:grafana-558dcbff8-mhw2n]} {0xc0009b6b30 map[namespace:cert-manager node:k8s-worker-51.example.com pod:cert-manager-9dd6685b9-6f8l4]} {0xc0009b6ba0 map[namespace:kube-system node:k8s-worker-51.example.com pod:node-labeler-786896fb46-p57ff]} {0xc0009b6c10 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-discover-9btlk]} {0xc0009b6c80 map[namespace:nuclio-infra node:k8s-worker-51.example.com pod:nginx-ingress-controller-85c85d9b85-grm9m]} {0xc0009b6cf0 map[namespace:esdata-dev node:k8s-worker-51.example.com pod:nginx-ingress-controller-5499bc654c-lfdfw]} {0xc0009b6d60 map[namespace:clair-infra node:k8s-worker-51.example.com pod:thanos-querier-85f59b5b87-qqmxp]} {0xc0009b6dd0 map[namespace:nuclio-infra node:k8s-worker-51.example.com pod:thanos-compactor-0]} {0xc0009b6e40 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-6-85cdc457-ggm8d]} {0xc0009b6eb0 map[namespace:couchbase-dev node:k8s-worker-51.example.com pod:default-http-backend-5f65599f44-b7w5l]} {0xc0009b6f20 map[namespace:kube-system node:k8s-worker-51.example.com pod:kube-iptables-tailer-fm5dr]} {0xc0009b6f90 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-9-7d7d959968-6rchn]} {0xc0009b7000 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-dev-primary-backup-6gkj7]} {0xc0009b7070 map[namespace:velero node:k8s-worker-51.example.com pod:restic-mtcnt]} {0xc0009b70e0 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-0-6fb44c7675-rlc4f]} {0xc0009b7150 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-21-74fb68cd9b-ngh6z]} {0xc0009b71c0 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-15-58b74cfb89-kwzrs]} {0xc0009b7230 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-12-5858886678-w5twb]} {0xc0009b72a0 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-mon-c-65b595c856-rlrzn]} {0xc0009b7310 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-3-6ccc57dff6-mk5hl]} {0xc0009b7380 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-18-5b9fd8d676-2n97g]} {0xc0009b73f0 map[namespace:monitoring-infra node:k8s-worker-51.example.com pod:thanos-compactor-0]} {0xc0009b75a0 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-18-5b9fd8d676-jw6d2]} {0xc0009b7610 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-mon-c-65b595c856-fwzfk]} {0xc0009b7680 map[namespace:rook-ceph-dev-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-12-5858886678-pgpkl]} {0xc0009b76f0 map[namespace:nuclio-infra node:k8s-worker-51.example.com pod:thanos-store-gateway-0]} {0xc0005e69c0 map[node:k8s-worker-51.example.com type:ipipTunnelAddress]}] k8s-worker-52.example.com:[{0xc0009b7c90 map[namespace:kube-system node:k8s-worker-52.example.com pod:kube-iptables-tailer-96qpq]} {0xc0009b7d00 map[namespace:kube-system node:k8s-worker-52.example.com pod:expire-check-worker-fkbw7]} {0xc0009b7d70 map[namespace:velero node:k8s-worker-52.example.com pod:restic-7ncbn]} {0xc0005c41d0 map[namespace:rene-dev node:k8s-worker-52.example.com pod:prometheus-kube-state-metrics-86b5b94b74-nr8z8]} {0xc0009b7de0 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-dev-primary-backup-6n5hj]} {0xc0009b7e50 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-discover-mtcbm]} {0xc0009b7ec0 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-mon-d-5dcbd5f699-l4r6g]} {0xc0009b7fa0 map[namespace:zhibek-test node:k8s-worker-52.example.com pod:hello-kubernetes-6795588ccf-jw8fz]} {0xc0009b77c0 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-22-5bff45d7d5-twg6b]} {0xc0009b7830 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-7-74cc67488d-vw2hx]} {0xc0009b78a0 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-4-59fb8d769-nvkq5]} {0xc0009b7910 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-1-686469857b-djcg9]} {0xc0009b7980 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-10-7677f46d98-9lvx2]} {0xc0009b79f0 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-19-f6d787999-hdq87]} {0xc0009b7a60 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-16-7c48dd956d-87xhq]} {0xc0009b7ad0 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-13-97d4db757-9x4tq]} {0xc0009b7f30 map[namespace:monitoring node:k8s-worker-52.example.com pod:mon-grafana-669bd4b55f-bbw22]} {0xc0005c4010 map[namespace:zhibek-test node:k8s-worker-52.example.com pod:hello-kubernetes-6795588ccf-vt9f2]} {0xc0005c4080 map[namespace:esdata-dev node:k8s-worker-52.example.com pod:default-http-backend-5f65599f44-jn9zv]} {0xc0005c40f0 map[namespace:monitoring-infra node:k8s-worker-52.example.com pod:kube-state-metrics-84d78dfbcf-ttg8n]} {0xc0009b7b40 map[namespace:prometheus node:k8s-worker-52.example.com pod:master-kube-state-metrics-7f4f8dfd55-djxsf]} {0xc0009b7bb0 map[namespace:cert-manager node:k8s-worker-52.example.com pod:cert-manager-webhook-8c5db9fb6-vfjvx]} {0xc0009b7c20 map[namespace:kube-system node:k8s-worker-52.example.com pod:external-dns-5c48cc494b-f844k]} {0xc0005c4160 map[namespace:prometheus node:k8s-worker-52.example.com pod:prometheus-msteams-7d6fcd79cc-md6ct]} {0xc0005c4470 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-10-7677f46d98-ncmdp]} {0xc0005c42b0 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-13-97d4db757-sh864]} {0xc0005c4320 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-22-5bff45d7d5-lsv99]} {0xc0005c4390 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-16-7c48dd956d-7wnxl]} {0xc0005c4400 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-1-686469857b-zrwrd]} {0xc0005c44e0 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-mon-d-5dcbd5f699-sfhss]} {0xc0005c4550 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-19-f6d787999-rcmh4]} {0xc0005c45c0 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-4-59fb8d769-slrpm]} {0xc0005c4240 map[namespace:velero node:k8s-worker-52.example.com pod:restic-xsj4f]} {0xc0005c4630 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-7-74cc67488d-dvtmk]} {0xc0005c4700 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-discover-lx9tq]} {0xc0005c47d0 map[namespace:max-test-alerting node:k8s-worker-52.example.com pod:test-alert-84778555f5-62m84]} {0xc0005c4840 map[namespace:kube-system node:k8s-worker-52.example.com pod:kube-iptables-tailer-9zk2n]} {0xc0005c48b0 map[namespace:kube-system node:k8s-worker-52.example.com pod:expire-check-worker-nmj84]} {0xc0005c4920 map[namespace:rook-ceph-dev-primary node:k8s-worker-52.example.com pod:rook-ceph-dev-primary-backup-tq2g7]} {0xc0005e67b0 map[node:k8s-worker-52.example.com type:ipipTunnelAddress]}] k8s-worker-53.example.com:[{0xc0005c5b30 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-discover-xslkh]} {0xc0005c5c10 map[namespace:kube-system node:k8s-worker-53.example.com pod:expire-check-worker-5l29m]} {0xc0005c4e10 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-dev-primary-backup-wcl4k]} {0xc0005c4e80 map[namespace:velero node:k8s-worker-53.example.com pod:restic-5j94c]} {0xc0005c4ef0 map[namespace:kube-system node:k8s-worker-53.example.com pod:expire-check-worker-zzfnt]} {0xc0005c4f60 map[namespace:kube-system node:k8s-worker-53.example.com pod:kube-iptables-tailer-28fvg]} {0xc0005c5120 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-discover-42ltp]} {0xc0005c5a50 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-discover-xslkh]} {0xc0005c5580 map[namespace:nuclio-infra node:k8s-worker-53.example.com pod:thanos-querier-5d5b8f467d-z6qpj]} {0xc0005c5190 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-28-5879549d95-jv6xx]} {0xc0005c49f0 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-29-559d4777fd-d7hwm]} {0xc0005c4a90 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-25-745b49dc78-pls2q]} {0xc0005c4b10 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-30-79d5bbc7d4-rbh2l]} {0xc0005c4ba0 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-24-55d4877779-cdnpd]} {0xc0005c4c10 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-26-84dfd74ccc-cdzbq]} {0xc0005c4d30 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-31-85949d9fcc-h7g5s]} {0xc0005c4da0 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-27-7ff4675b84-k6g9s]} {0xc0005c5200 map[namespace:elastic-infra node:k8s-worker-53.example.com pod:thanos-querier-dc886459b-r2qw7]} {0xc0005c5270 map[namespace:zhibek-test node:k8s-worker-53.example.com pod:hello-kubernetes-6795588ccf-299vp]} {0xc0005c52e0 map[namespace:zhibek-test node:k8s-worker-53.example.com pod:hello-kubernetes-6795588ccf-kkpf9]} {0xc0005c5350 map[namespace:rene-dev node:k8s-worker-53.example.com pod:prometheus-pushgateway-57b5f95d94-hsbpk]} {0xc0005c53c0 map[namespace:zhibek-test node:k8s-worker-53.example.com pod:hello-kubernetes-6795588ccf-q4fsc]} {0xc0005c5430 map[namespace:kube-system node:k8s-worker-53.example.com pod:external-dns-extern-top-zone-5887b4cc84-vndtf]} {0xc0005c54a0 map[namespace:aws-service-broker node:k8s-worker-53.example.com pod:aws-service-broker-aws-servicebroker-5b4cc95898-fn968]} {0xc0005c5510 map[namespace:clair-infra node:k8s-worker-53.example.com pod:nginx-ingress-controller-5ccfdfd797-zxbhq]} {0xc0005c55f0 map[namespace:elastic-dev node:k8s-worker-53.example.com pod:filebeat-syslog-5fd9947874-54tbd]} {0xc0005c5c80 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-dev-primary-backup-mqhqr]} {0xc0005c5cf0 map[namespace:velero node:k8s-worker-53.example.com pod:restic-xwhz8]} {0xc0005c5ac0 map[namespace:velero node:k8s-worker-53.example.com pod:restic-xwhz8]} {0xc0006d03f0 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-34-6ffb5bb788-9w5vm]} {0xc0006d0460 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-33-b68466fdc-wzxd6]} {0xc0006d0310 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-36-8594f8c758-f7v7h]} {0xc0006d02a0 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-37-596f87f6f5-j4xc4]} {0xc0006d0380 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-32-576b57c574-sw47g]} {0xc0005c5d60 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-27-7ff4675b84-hdbtm]} {0xc0005c5dd0 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-25-745b49dc78-rwj5x]} {0xc0005c5ba0 map[namespace:kube-system node:k8s-worker-53.example.com pod:kube-iptables-tailer-dll4v]} {0xc0005c5660 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-24-55d4877779-fjd5g]} {0xc0005c56d0 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-26-84dfd74ccc-5fpdk]} {0xc0005c5740 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-30-79d5bbc7d4-dxkcb]} {0xc0005c57b0 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-29-559d4777fd-4h6v4]} {0xc0005c5820 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-27-7ff4675b84-wrvb5]} {0xc0005c5890 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-28-5879549d95-fbdpt]} {0xc0005c5900 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-25-745b49dc78-pdmwr]} {0xc0005c5970 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-mon-f-7c97bbdb44-sflxb]} {0xc0005c59e0 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-31-85949d9fcc-78wxm]} {0xc0005c5e40 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-29-559d4777fd-7wh8z]} {0xc0005c5eb0 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-31-85949d9fcc-dlrzl]} {0xc0005c5f20 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-26-84dfd74ccc-qlh86]} {0xc0005c5f90 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-28-5879549d95-6jgqt]} {0xc0005c4fd0 map[namespace:lxos-dev node:k8s-worker-53.example.com pod:hello-kubernetes-759f5dd94-nwj4p]} {0xc0005c5040 map[namespace:cert-manager node:k8s-worker-53.example.com pod:cert-manager-cainjector-8545fdf87c-5lr9p]} {0xc0005c50b0 map[namespace:cert-manager node:k8s-worker-53.example.com pod:cert-manager-9dd6685b9-2q7hz]} {0xc0006d0000 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-24-55d4877779-ctmv7]} {0xc0006d0070 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-mon-f-7c97bbdb44-j8tgf]} {0xc0006d00e0 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-30-79d5bbc7d4-t6g48]} {0xc0006d0150 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-39-6b9695496f-qc7v8]} {0xc0006d01c0 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-38-7dcdd4674d-wnp62]} {0xc0006d0230 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-35-85555f9c49-46mrz]} {0xc0006d0530 map[namespace:catalog node:k8s-worker-53.example.com pod:catalog-catalog-webhook-dcbc9db67-774fl]} {0xc0006d05a0 map[namespace:prometheus node:k8s-worker-53.example.com pod:prometheus-msteams-7d6fcd79cc-k6tx6]} {0xc0006d0610 map[namespace:clair-dev node:k8s-worker-53.example.com pod:nginx-ingress-controller-866dd75bb9-9jsrt]} {0xc0006d0680 map[namespace:kube-system node:k8s-worker-53.example.com pod:coredns-c9fdf577f-d5sp8]} {0xc0006d06f0 map[namespace:lxos-infra node:k8s-worker-53.example.com pod:alertmanager-infra-alertmanager-1]} {0xc0006d0760 map[namespace:lxos-dev node:k8s-worker-53.example.com pod:hello-kubernetes-759f5dd94-sqk9k]} {0xc0006d07d0 map[namespace:elastic-dev node:k8s-worker-53.example.com pod:default-http-backend-5f65599f44-rwpsl]} {0xc0006d0840 map[namespace:elastic-system node:k8s-worker-53.example.com pod:elastic-operator-0]} {0xc0006d08b0 map[namespace:aws-service-broker node:k8s-worker-53.example.com pod:aws-service-broker-aws-servicebroker-5b4cc95898-l8g7s]} {0xc0006d0920 map[namespace:kube-system node:k8s-worker-53.example.com pod:coredns-c9fdf577f-qg845]} {0xc0006d0990 map[namespace:prometheus node:k8s-worker-53.example.com pod:master-kube-state-metrics-7f4f8dfd55-pkpc6]} {0xc0006d0a00 map[namespace:velero node:k8s-worker-53.example.com pod:velero-dev-676556f788-b2f2p]} {0xc0006d0a70 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-24-55d4877779-28h72]} {0xc0006d0ae0 map[namespace:prometheus node:k8s-worker-53.example.com pod:alertmanager-master-prometheus-operator-alertmanager-0]} {0xc0006d0b50 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-25-745b49dc78-lwfz6]} {0xc0006d0bd0 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-27-7ff4675b84-z5tqn]} {0xc0006d0c40 map[namespace:velero node:k8s-worker-53.example.com pod:restic-jn59h]} {0xc0006d0cb0 map[namespace:kube-system node:k8s-worker-53.example.com pod:external-dns-extern-top-zone-56c5b68b9c-75fgm]} {0xc0006d0d20 map[namespace:elastic-infra node:k8s-worker-53.example.com pod:kube-state-metrics-5bfdbb8d8-qdgfw]} {0xc0006d0d90 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-30-79d5bbc7d4-sm897]} {0xc0006d0e00 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-28-5879549d95-xpf9v]} {0xc0006d0e70 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-mon-f-7c97bbdb44-kvrb6]} {0xc0006d0ee0 map[namespace:elastic-dev node:k8s-worker-53.example.com pod:nginx-ingress-controller-7c8c64fd89-wqcr5]} {0xc0006d0f50 map[namespace:kube-system node:k8s-worker-53.example.com pod:kube-iptables-tailer-dll4v]} {0xc0006d0fc0 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-26-84dfd74ccc-ccw22]} {0xc0006d1090 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-29-559d4777fd-lr852]} {0xc0006d1100 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-31-85949d9fcc-xm7fb]} {0xc0006d1170 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-discover-xslkh]} {0xc0006d11e0 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-dev-primary-backup-mqhqr]} {0xc0006d1250 map[namespace:gitlab-infra node:k8s-worker-53.example.com pod:nginx-ingress-controller-9c448878c-bkhz6]} {0xc0006d1320 map[namespace:rook-ceph-dev-primary node:k8s-worker-53.example.com pod:rook-ceph-mon-f-7c97bbdb44-t6g4h]} {0xc0005e6860 map[node:k8s-worker-53.example.com type:ipipTunnelAddress]}]]
2020-10-23 14:46:31.412 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-master-01.example.com" k8sNode="k8s-master-01.example.com"
2020-10-23 14:46:31.416 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-02.example.com" k8sNode="k8s-worker-02.example.com"
2020-10-23 14:46:31.419 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-21.example.com" k8sNode="k8s-worker-21.example.com"
2020-10-23 14:46:31.422 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-52.example.com" k8sNode="k8s-worker-52.example.com"
2020-10-23 14:46:31.427 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-53.example.com" k8sNode="k8s-worker-53.example.com"
2020-10-23 14:46:31.432 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-26.example.com" k8sNode="k8s-worker-26.example.com"
2020-10-23 14:46:31.436 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-50.example.com" k8sNode="k8s-worker-50.example.com"
2020-10-23 14:46:31.439 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-27.example.com" k8sNode="k8s-worker-27.example.com"
2020-10-23 14:46:31.442 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-20.example.com" k8sNode="k8s-worker-20.example.com"
2020-10-23 14:46:31.446 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-28.example.com" k8sNode="k8s-worker-28.example.com"
2020-10-23 14:46:31.611 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-master-02.example.com" k8sNode="k8s-master-02.example.com"
2020-10-23 14:46:31.811 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-00.example.com" k8sNode="k8s-worker-00.example.com"
2020-10-23 14:46:32.012 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-01.example.com" k8sNode="k8s-worker-01.example.com"
2020-10-23 14:46:32.012 [DEBUG][1] k8s.go 564: Performing 'Get' for Node(k8s-worker-23.example.com)
2020-10-23 14:46:32.012 [DEBUG][1] node.go 114: Received Get request on Node type
2020-10-23 14:46:32.016 [INFO][1] ipam.go 281: Calico Node referenced in IPAM data does not exist error=resource does not exist: Node(k8s-worker-23.example.com) with error: nodes "k8s-worker-23.example.com" not found
2020-10-23 14:46:32.016 [INFO][1] ipam.go 137: Checking node calicoNode="k8s-worker-23.example.com" k8sNode=""
2020-10-23 14:46:32.021 [INFO][1] ipam.go 177: Cleaning up IPAM resources for deleted node calicoNode="k8s-worker-23.example.com" k8sNode=""
2020-10-23 14:46:32.021 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.7ecd07e742bb804bdf8fb37d7820985a5a3b0fd3bb098fc0fb4bd315f2152a05
eth0'
2020-10-23 14:46:32.021 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.7ecd07e742bb804bdf8fb37d7820985a5a3b0fd3bb098fc0fb4bd315f2152a05
eth0)
2020-10-23 14:46:32.021 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.7ecd07e742bb804bdf8fb37d7820985a5a3b0fd3bb098fc0fb4bd315f2152a05
eth0) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.021 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.7ecd07e742bb804bdf8fb37d7820985a5a3b0fd3bb098fc0fb4bd315f2152a05
eth0) Name="k8s-pod-network.7ecd07e742bb804bdf8fb37d7820985a5a3b0fd3bb098fc0fb4bd315f2152a05\r\neth0" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.024 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.7ecd07e742bb804bdf8fb37d7820985a5a3b0fd3bb098fc0fb4bd315f2152a05
eth0) Name="k8s-pod-network.7ecd07e742bb804bdf8fb37d7820985a5a3b0fd3bb098fc0fb4bd315f2152a05\r\neth0" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.7ecd07e742bb804bdf8fb37d7820985a5a3b0fd3bb098fc0fb4bd315f2152a05\r\neth0" not found
2020-10-23 14:46:32.025 [DEBUG][1] ipam.go 206: IP already released calicoNode="k8s-worker-23.example.com" handle="k8s-pod-network.7ecd07e742bb804bdf8fb37d7820985a5a3b0fd3bb098fc0fb4bd315f2152a05\r\neth0"
2020-10-23 14:46:32.025 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.34761b611ec2ce5329b0596b570c39f8bdc923cbef7b9d6e27ea455a9c1850c6
eth0'
2020-10-23 14:46:32.025 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.34761b611ec2ce5329b0596b570c39f8bdc923cbef7b9d6e27ea455a9c1850c6
eth0)
2020-10-23 14:46:32.025 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.34761b611ec2ce5329b0596b570c39f8bdc923cbef7b9d6e27ea455a9c1850c6
eth0) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.025 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.34761b611ec2ce5329b0596b570c39f8bdc923cbef7b9d6e27ea455a9c1850c6
eth0) Name="k8s-pod-network.34761b611ec2ce5329b0596b570c39f8bdc923cbef7b9d6e27ea455a9c1850c6\r\neth0" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.028 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.34761b611ec2ce5329b0596b570c39f8bdc923cbef7b9d6e27ea455a9c1850c6
eth0) Name="k8s-pod-network.34761b611ec2ce5329b0596b570c39f8bdc923cbef7b9d6e27ea455a9c1850c6\r\neth0" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.34761b611ec2ce5329b0596b570c39f8bdc923cbef7b9d6e27ea455a9c1850c6\r\neth0" not found
2020-10-23 14:46:32.028 [DEBUG][1] ipam.go 206: IP already released calicoNode="k8s-worker-23.example.com" handle="k8s-pod-network.34761b611ec2ce5329b0596b570c39f8bdc923cbef7b9d6e27ea455a9c1850c6\r\neth0"
2020-10-23 14:46:32.028 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.06796edd1251f1a85ffaed3ce388c95dcf94729580d575498662e954eaa4931d
eth0'
2020-10-23 14:46:32.028 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.06796edd1251f1a85ffaed3ce388c95dcf94729580d575498662e954eaa4931d
eth0)
2020-10-23 14:46:32.028 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.06796edd1251f1a85ffaed3ce388c95dcf94729580d575498662e954eaa4931d
eth0) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.028 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.06796edd1251f1a85ffaed3ce388c95dcf94729580d575498662e954eaa4931d
eth0) Name="k8s-pod-network.06796edd1251f1a85ffaed3ce388c95dcf94729580d575498662e954eaa4931d\r\neth0" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.031 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.06796edd1251f1a85ffaed3ce388c95dcf94729580d575498662e954eaa4931d
eth0) Name="k8s-pod-network.06796edd1251f1a85ffaed3ce388c95dcf94729580d575498662e954eaa4931d\r\neth0" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.06796edd1251f1a85ffaed3ce388c95dcf94729580d575498662e954eaa4931d\r\neth0" not found
2020-10-23 14:46:32.031 [DEBUG][1] ipam.go 206: IP already released calicoNode="k8s-worker-23.example.com" handle="k8s-pod-network.06796edd1251f1a85ffaed3ce388c95dcf94729580d575498662e954eaa4931d\r\neth0"
2020-10-23 14:46:32.031 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.f3d83aa6b282bb6a86d7095394481785b1656673400d1e9848ae854d12f5fd57
eth0'
2020-10-23 14:46:32.031 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.f3d83aa6b282bb6a86d7095394481785b1656673400d1e9848ae854d12f5fd57
eth0)
2020-10-23 14:46:32.031 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.f3d83aa6b282bb6a86d7095394481785b1656673400d1e9848ae854d12f5fd57
eth0) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.032 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.f3d83aa6b282bb6a86d7095394481785b1656673400d1e9848ae854d12f5fd57
eth0) Name="k8s-pod-network.f3d83aa6b282bb6a86d7095394481785b1656673400d1e9848ae854d12f5fd57\r\neth0" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.036 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.f3d83aa6b282bb6a86d7095394481785b1656673400d1e9848ae854d12f5fd57
eth0) Name="k8s-pod-network.f3d83aa6b282bb6a86d7095394481785b1656673400d1e9848ae854d12f5fd57\r\neth0" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.f3d83aa6b282bb6a86d7095394481785b1656673400d1e9848ae854d12f5fd57\r\neth0" not found
2020-10-23 14:46:32.036 [DEBUG][1] ipam.go 206: IP already released calicoNode="k8s-worker-23.example.com" handle="k8s-pod-network.f3d83aa6b282bb6a86d7095394481785b1656673400d1e9848ae854d12f5fd57\r\neth0"
2020-10-23 14:46:32.036 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.f7db62798993abb14f998fa4d823d74b12f42ec9ab98861fe89d42402e4ed93e
eth0'
2020-10-23 14:46:32.036 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.f7db62798993abb14f998fa4d823d74b12f42ec9ab98861fe89d42402e4ed93e
eth0)
2020-10-23 14:46:32.036 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.f7db62798993abb14f998fa4d823d74b12f42ec9ab98861fe89d42402e4ed93e
eth0) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.036 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.f7db62798993abb14f998fa4d823d74b12f42ec9ab98861fe89d42402e4ed93e
eth0) Name="k8s-pod-network.f7db62798993abb14f998fa4d823d74b12f42ec9ab98861fe89d42402e4ed93e\r\neth0" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.039 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.f7db62798993abb14f998fa4d823d74b12f42ec9ab98861fe89d42402e4ed93e
eth0) Name="k8s-pod-network.f7db62798993abb14f998fa4d823d74b12f42ec9ab98861fe89d42402e4ed93e\r\neth0" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.f7db62798993abb14f998fa4d823d74b12f42ec9ab98861fe89d42402e4ed93e\r\neth0" not found
2020-10-23 14:46:32.039 [DEBUG][1] ipam.go 206: IP already released calicoNode="k8s-worker-23.example.com" handle="k8s-pod-network.f7db62798993abb14f998fa4d823d74b12f42ec9ab98861fe89d42402e4ed93e\r\neth0"
2020-10-23 14:46:32.039 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.0d05b64f5e54ff3c2daa4074a7ef1c03319970ce31f40f7321943879f414d80f
eth0'
2020-10-23 14:46:32.039 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.0d05b64f5e54ff3c2daa4074a7ef1c03319970ce31f40f7321943879f414d80f
eth0)
2020-10-23 14:46:32.039 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.0d05b64f5e54ff3c2daa4074a7ef1c03319970ce31f40f7321943879f414d80f
eth0) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.039 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.0d05b64f5e54ff3c2daa4074a7ef1c03319970ce31f40f7321943879f414d80f
eth0) Name="k8s-pod-network.0d05b64f5e54ff3c2daa4074a7ef1c03319970ce31f40f7321943879f414d80f\r\neth0" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.042 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.0d05b64f5e54ff3c2daa4074a7ef1c03319970ce31f40f7321943879f414d80f
eth0) Name="k8s-pod-network.0d05b64f5e54ff3c2daa4074a7ef1c03319970ce31f40f7321943879f414d80f\r\neth0" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.0d05b64f5e54ff3c2daa4074a7ef1c03319970ce31f40f7321943879f414d80f\r\neth0" not found
2020-10-23 14:46:32.042 [DEBUG][1] ipam.go 206: IP already released calicoNode="k8s-worker-23.example.com" handle="k8s-pod-network.0d05b64f5e54ff3c2daa4074a7ef1c03319970ce31f40f7321943879f414d80f\r\neth0"
2020-10-23 14:46:32.042 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.b8acd91e18061bdc3bd012c733b8f590f1bdc2f227aaf29bdc947210b24df9fe
eth0'
2020-10-23 14:46:32.042 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.b8acd91e18061bdc3bd012c733b8f590f1bdc2f227aaf29bdc947210b24df9fe
eth0)
2020-10-23 14:46:32.042 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.b8acd91e18061bdc3bd012c733b8f590f1bdc2f227aaf29bdc947210b24df9fe
eth0) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.042 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.b8acd91e18061bdc3bd012c733b8f590f1bdc2f227aaf29bdc947210b24df9fe
eth0) Name="k8s-pod-network.b8acd91e18061bdc3bd012c733b8f590f1bdc2f227aaf29bdc947210b24df9fe\r\neth0" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.045 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.b8acd91e18061bdc3bd012c733b8f590f1bdc2f227aaf29bdc947210b24df9fe
eth0) Name="k8s-pod-network.b8acd91e18061bdc3bd012c733b8f590f1bdc2f227aaf29bdc947210b24df9fe\r\neth0" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.b8acd91e18061bdc3bd012c733b8f590f1bdc2f227aaf29bdc947210b24df9fe\r\neth0" not found
2020-10-23 14:46:32.045 [DEBUG][1] ipam.go 206: IP already released calicoNode="k8s-worker-23.example.com" handle="k8s-pod-network.b8acd91e18061bdc3bd012c733b8f590f1bdc2f227aaf29bdc947210b24df9fe\r\neth0"
2020-10-23 14:46:32.045 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.f7cec9b5e772927ed63a5b05890014cc56f33acf4fe2fe8d24193710eba01058
eth0'
2020-10-23 14:46:32.045 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.f7cec9b5e772927ed63a5b05890014cc56f33acf4fe2fe8d24193710eba01058
eth0)
2020-10-23 14:46:32.045 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.f7cec9b5e772927ed63a5b05890014cc56f33acf4fe2fe8d24193710eba01058
eth0) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.045 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.f7cec9b5e772927ed63a5b05890014cc56f33acf4fe2fe8d24193710eba01058
eth0) Name="k8s-pod-network.f7cec9b5e772927ed63a5b05890014cc56f33acf4fe2fe8d24193710eba01058\r\neth0" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.048 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.f7cec9b5e772927ed63a5b05890014cc56f33acf4fe2fe8d24193710eba01058
eth0) Name="k8s-pod-network.f7cec9b5e772927ed63a5b05890014cc56f33acf4fe2fe8d24193710eba01058\r\neth0" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.f7cec9b5e772927ed63a5b05890014cc56f33acf4fe2fe8d24193710eba01058\r\neth0" not found
2020-10-23 14:46:32.048 [DEBUG][1] ipam.go 206: IP already released calicoNode="k8s-worker-23.example.com" handle="k8s-pod-network.f7cec9b5e772927ed63a5b05890014cc56f33acf4fe2fe8d24193710eba01058\r\neth0"
2020-10-23 14:46:32.048 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.57d7110294e4f7f7bab8ba50d2c30cda6be7ee32b3788c5fbd554366b0291604
eth0'
2020-10-23 14:46:32.048 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.57d7110294e4f7f7bab8ba50d2c30cda6be7ee32b3788c5fbd554366b0291604
eth0)
2020-10-23 14:46:32.048 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.57d7110294e4f7f7bab8ba50d2c30cda6be7ee32b3788c5fbd554366b0291604
eth0) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.048 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.57d7110294e4f7f7bab8ba50d2c30cda6be7ee32b3788c5fbd554366b0291604
eth0) Name="k8s-pod-network.57d7110294e4f7f7bab8ba50d2c30cda6be7ee32b3788c5fbd554366b0291604\r\neth0" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.051 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.57d7110294e4f7f7bab8ba50d2c30cda6be7ee32b3788c5fbd554366b0291604
eth0) Name="k8s-pod-network.57d7110294e4f7f7bab8ba50d2c30cda6be7ee32b3788c5fbd554366b0291604\r\neth0" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.57d7110294e4f7f7bab8ba50d2c30cda6be7ee32b3788c5fbd554366b0291604\r\neth0" not found
2020-10-23 14:46:32.051 [DEBUG][1] ipam.go 206: IP already released calicoNode="k8s-worker-23.example.com" handle="k8s-pod-network.57d7110294e4f7f7bab8ba50d2c30cda6be7ee32b3788c5fbd554366b0291604\r\neth0"
2020-10-23 14:46:32.051 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.4a37f77450c30079ce8267cf9055cd8505b93b8b9a2117652a161a64ce9ffa50
eth0'
2020-10-23 14:46:32.051 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.4a37f77450c30079ce8267cf9055cd8505b93b8b9a2117652a161a64ce9ffa50
eth0)
2020-10-23 14:46:32.051 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.4a37f77450c30079ce8267cf9055cd8505b93b8b9a2117652a161a64ce9ffa50
eth0) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.051 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.4a37f77450c30079ce8267cf9055cd8505b93b8b9a2117652a161a64ce9ffa50
eth0) Name="k8s-pod-network.4a37f77450c30079ce8267cf9055cd8505b93b8b9a2117652a161a64ce9ffa50\r\neth0" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.055 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.4a37f77450c30079ce8267cf9055cd8505b93b8b9a2117652a161a64ce9ffa50
eth0) Name="k8s-pod-network.4a37f77450c30079ce8267cf9055cd8505b93b8b9a2117652a161a64ce9ffa50\r\neth0" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.4a37f77450c30079ce8267cf9055cd8505b93b8b9a2117652a161a64ce9ffa50\r\neth0" not found
2020-10-23 14:46:32.055 [DEBUG][1] ipam.go 206: IP already released calicoNode="k8s-worker-23.example.com" handle="k8s-pod-network.4a37f77450c30079ce8267cf9055cd8505b93b8b9a2117652a161a64ce9ffa50\r\neth0"
2020-10-23 14:46:32.055 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.9fc35479470f4c619ce93bd5ab12bbc7cae977c512f9490b8bde236f692ae226
eth0'
2020-10-23 14:46:32.055 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.9fc35479470f4c619ce93bd5ab12bbc7cae977c512f9490b8bde236f692ae226
eth0)
2020-10-23 14:46:32.055 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.9fc35479470f4c619ce93bd5ab12bbc7cae977c512f9490b8bde236f692ae226
eth0) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.055 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.9fc35479470f4c619ce93bd5ab12bbc7cae977c512f9490b8bde236f692ae226
eth0) Name="k8s-pod-network.9fc35479470f4c619ce93bd5ab12bbc7cae977c512f9490b8bde236f692ae226\r\neth0" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.061 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.9fc35479470f4c619ce93bd5ab12bbc7cae977c512f9490b8bde236f692ae226
eth0) Name="k8s-pod-network.9fc35479470f4c619ce93bd5ab12bbc7cae977c512f9490b8bde236f692ae226\r\neth0" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.9fc35479470f4c619ce93bd5ab12bbc7cae977c512f9490b8bde236f692ae226\r\neth0" not found
2020-10-23 14:46:32.061 [DEBUG][1] ipam.go 206: IP already released calicoNode="k8s-worker-23.example.com" handle="k8s-pod-network.9fc35479470f4c619ce93bd5ab12bbc7cae977c512f9490b8bde236f692ae226\r\neth0"
2020-10-23 14:46:32.061 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.7b7a0e304f516f12f7b4bb34e7c3b30fdef8aad2441367df1858b5aab04b5a5e
eth0'
2020-10-23 14:46:32.061 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.7b7a0e304f516f12f7b4bb34e7c3b30fdef8aad2441367df1858b5aab04b5a5e
eth0)
2020-10-23 14:46:32.061 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.7b7a0e304f516f12f7b4bb34e7c3b30fdef8aad2441367df1858b5aab04b5a5e
eth0) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.061 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.7b7a0e304f516f12f7b4bb34e7c3b30fdef8aad2441367df1858b5aab04b5a5e
eth0) Name="k8s-pod-network.7b7a0e304f516f12f7b4bb34e7c3b30fdef8aad2441367df1858b5aab04b5a5e\r\neth0" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.065 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.7b7a0e304f516f12f7b4bb34e7c3b30fdef8aad2441367df1858b5aab04b5a5e
eth0) Name="k8s-pod-network.7b7a0e304f516f12f7b4bb34e7c3b30fdef8aad2441367df1858b5aab04b5a5e\r\neth0" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.7b7a0e304f516f12f7b4bb34e7c3b30fdef8aad2441367df1858b5aab04b5a5e\r\neth0" not found
2020-10-23 14:46:32.065 [DEBUG][1] ipam.go 206: IP already released calicoNode="k8s-worker-23.example.com" handle="k8s-pod-network.7b7a0e304f516f12f7b4bb34e7c3b30fdef8aad2441367df1858b5aab04b5a5e\r\neth0"
2020-10-23 14:46:32.065 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.ee192b4a71c34af6f9a790e94f2acc8db69fc237e3542d1c529e860f6553c8e3
eth0'
2020-10-23 14:46:32.065 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.ee192b4a71c34af6f9a790e94f2acc8db69fc237e3542d1c529e860f6553c8e3
eth0)
2020-10-23 14:46:32.065 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.ee192b4a71c34af6f9a790e94f2acc8db69fc237e3542d1c529e860f6553c8e3
eth0) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.065 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.ee192b4a71c34af6f9a790e94f2acc8db69fc237e3542d1c529e860f6553c8e3
eth0) Name="k8s-pod-network.ee192b4a71c34af6f9a790e94f2acc8db69fc237e3542d1c529e860f6553c8e3\r\neth0" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.067 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.ee192b4a71c34af6f9a790e94f2acc8db69fc237e3542d1c529e860f6553c8e3
eth0) Name="k8s-pod-network.ee192b4a71c34af6f9a790e94f2acc8db69fc237e3542d1c529e860f6553c8e3\r\neth0" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.ee192b4a71c34af6f9a790e94f2acc8db69fc237e3542d1c529e860f6553c8e3\r\neth0" not found
2020-10-23 14:46:32.067 [DEBUG][1] ipam.go 206: IP already released calicoNode="k8s-worker-23.example.com" handle="k8s-pod-network.ee192b4a71c34af6f9a790e94f2acc8db69fc237e3542d1c529e860f6553c8e3\r\neth0"
2020-10-23 14:46:32.067 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.7b5de3337183f12c034d4c94c6dfea3f2eeb563c1079251a5017d706661d26f7
eth0'
2020-10-23 14:46:32.067 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.7b5de3337183f12c034d4c94c6dfea3f2eeb563c1079251a5017d706661d26f7
eth0)
2020-10-23 14:46:32.067 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.7b5de3337183f12c034d4c94c6dfea3f2eeb563c1079251a5017d706661d26f7
eth0) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.067 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.7b5de3337183f12c034d4c94c6dfea3f2eeb563c1079251a5017d706661d26f7
eth0) Name="k8s-pod-network.7b5de3337183f12c034d4c94c6dfea3f2eeb563c1079251a5017d706661d26f7\r\neth0" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.071 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.7b5de3337183f12c034d4c94c6dfea3f2eeb563c1079251a5017d706661d26f7
eth0) Name="k8s-pod-network.7b5de3337183f12c034d4c94c6dfea3f2eeb563c1079251a5017d706661d26f7\r\neth0" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.7b5de3337183f12c034d4c94c6dfea3f2eeb563c1079251a5017d706661d26f7\r\neth0" not found
2020-10-23 14:46:32.071 [DEBUG][1] ipam.go 206: IP already released calicoNode="k8s-worker-23.example.com" handle="k8s-pod-network.7b5de3337183f12c034d4c94c6dfea3f2eeb563c1079251a5017d706661d26f7\r\neth0"
2020-10-23 14:46:32.071 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.70a45c773015983a222cc978b0a19e2aa6db63582bbcd03b6b8a4add2b3b9f7d
eth0'
2020-10-23 14:46:32.071 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.70a45c773015983a222cc978b0a19e2aa6db63582bbcd03b6b8a4add2b3b9f7d
eth0)
2020-10-23 14:46:32.071 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.70a45c773015983a222cc978b0a19e2aa6db63582bbcd03b6b8a4add2b3b9f7d
eth0) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.071 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.70a45c773015983a222cc978b0a19e2aa6db63582bbcd03b6b8a4add2b3b9f7d
eth0) Name="k8s-pod-network.70a45c773015983a222cc978b0a19e2aa6db63582bbcd03b6b8a4add2b3b9f7d\r\neth0" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.074 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.70a45c773015983a222cc978b0a19e2aa6db63582bbcd03b6b8a4add2b3b9f7d
eth0) Name="k8s-pod-network.70a45c773015983a222cc978b0a19e2aa6db63582bbcd03b6b8a4add2b3b9f7d\r\neth0" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.70a45c773015983a222cc978b0a19e2aa6db63582bbcd03b6b8a4add2b3b9f7d\r\neth0" not found
2020-10-23 14:46:32.074 [DEBUG][1] ipam.go 206: IP already released calicoNode="k8s-worker-23.example.com" handle="k8s-pod-network.70a45c773015983a222cc978b0a19e2aa6db63582bbcd03b6b8a4add2b3b9f7d\r\neth0"
2020-10-23 14:46:32.074 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.84c0cb228aa52bdec4e1cbce8d518f1231ce2747038409c6f47d9c5e294d2e20
eth0'
2020-10-23 14:46:32.074 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.84c0cb228aa52bdec4e1cbce8d518f1231ce2747038409c6f47d9c5e294d2e20
eth0)
2020-10-23 14:46:32.074 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.84c0cb228aa52bdec4e1cbce8d518f1231ce2747038409c6f47d9c5e294d2e20
eth0) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.074 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.84c0cb228aa52bdec4e1cbce8d518f1231ce2747038409c6f47d9c5e294d2e20
eth0) Name="k8s-pod-network.84c0cb228aa52bdec4e1cbce8d518f1231ce2747038409c6f47d9c5e294d2e20\r\neth0" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.076 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.84c0cb228aa52bdec4e1cbce8d518f1231ce2747038409c6f47d9c5e294d2e20
eth0) Name="k8s-pod-network.84c0cb228aa52bdec4e1cbce8d518f1231ce2747038409c6f47d9c5e294d2e20\r\neth0" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.84c0cb228aa52bdec4e1cbce8d518f1231ce2747038409c6f47d9c5e294d2e20\r\neth0" not found
2020-10-23 14:46:32.077 [DEBUG][1] ipam.go 206: IP already released calicoNode="k8s-worker-23.example.com" handle="k8s-pod-network.84c0cb228aa52bdec4e1cbce8d518f1231ce2747038409c6f47d9c5e294d2e20\r\neth0"
2020-10-23 14:46:32.077 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.f7f8d544d7c62f694277dda3a57e1c42faf3f9193565cb5c3808bf856b50aa00
eth0'
2020-10-23 14:46:32.077 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.f7f8d544d7c62f694277dda3a57e1c42faf3f9193565cb5c3808bf856b50aa00
eth0)
2020-10-23 14:46:32.077 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.f7f8d544d7c62f694277dda3a57e1c42faf3f9193565cb5c3808bf856b50aa00
eth0) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.077 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.f7f8d544d7c62f694277dda3a57e1c42faf3f9193565cb5c3808bf856b50aa00
eth0) Name="k8s-pod-network.f7f8d544d7c62f694277dda3a57e1c42faf3f9193565cb5c3808bf856b50aa00\r\neth0" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.080 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.f7f8d544d7c62f694277dda3a57e1c42faf3f9193565cb5c3808bf856b50aa00
eth0) Name="k8s-pod-network.f7f8d544d7c62f694277dda3a57e1c42faf3f9193565cb5c3808bf856b50aa00\r\neth0" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.f7f8d544d7c62f694277dda3a57e1c42faf3f9193565cb5c3808bf856b50aa00\r\neth0" not found
2020-10-23 14:46:32.080 [DEBUG][1] ipam.go 206: IP already released calicoNode="k8s-worker-23.example.com" handle="k8s-pod-network.f7f8d544d7c62f694277dda3a57e1c42faf3f9193565cb5c3808bf856b50aa00\r\neth0"
2020-10-23 14:46:32.080 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229'
2020-10-23 14:46:32.080 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229)
2020-10-23 14:46:32.080 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.080 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229) Name="k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.083 [DEBUG][1] ipam.go 1317: Querying block so we can release IPs by handle cidr=100.80.6.0/26 handle="k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229"
2020-10-23 14:46:32.084 [DEBUG][1] k8s.go 564: Performing 'Get' for BlockKey(cidr=100.80.6.0/26)
2020-10-23 14:46:32.084 [DEBUG][1] cidr.go 34: Converted IPNet to resource name IPNet="100.80.6.0/26" Name="100-80-6-0-26"
2020-10-23 14:46:32.084 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMBlock(100-80-6-0-26) Resource="IPAMBlocks" Revision=""
2020-10-23 14:46:32.084 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMBlock(100-80-6-0-26) Name="100-80-6-0-26" Namespace="" Resource="IPAMBlocks" Revision=""
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 349: Attribute indexes to release: [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 0 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 1 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 2 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 3 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 4 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 5 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 6 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 7 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 8 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 9 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 10 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 11 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 12 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 13 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 14 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 15 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 17 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 18 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 19 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 20 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam_block.go 300: 21 in [16]
2020-10-23 14:46:32.088 [DEBUG][1] ipam.go 1339: Block has 1 IPs with the given handle cidr=100.80.6.0/26 handle="k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229"
2020-10-23 14:46:32.088 [DEBUG][1] ipam.go 1361: Updating block to release IPs cidr=100.80.6.0/26 handle="k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229"
2020-10-23 14:46:32.088 [DEBUG][1] k8s.go 489: Performing 'Update' for &{Key:BlockKey(cidr=100.80.6.0/26) Value:0xc0007acd80 Revision:534575645 UID:0xc00011e880 TTL:0s}
2020-10-23 14:46:32.088 [DEBUG][1] cidr.go 34: Converted IPNet to resource name IPNet="100.80.6.0/26" Name="100-80-6-0-26"
2020-10-23 14:46:32.088 [DEBUG][1] customresource.go 109: Update custom Kubernetes resource Key=IPAMBlock(100-80-6-0-26) Resource="IPAMBlocks" Value=&v3.IPAMBlock{TypeMeta:v1.TypeMeta{Kind:"IPAMBlock", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"100-80-6-0-26", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"534575645", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMBlockSpec{CIDR:"100.80.6.0/26", Affinity:(*string)(0xc000674e90), StrictAffinity:false, Allocations:[]*int{(*int)(nil), (*int)(nil), (*int)(0xc00038cfa0), (*int)(0xc00038d020), (*int)(0xc00038d180), (*int)(0xc00038d230), (*int)(0xc00038d320), (*int)(0xc00038d370), (*int)(0xc00038d3f0), (*int)(0xc00038db20), (*int)(0xc00038d590), (*int)(0xc00038d640), (*int)(0xc00038d7c0), (*int)(0xc00022a040), (*int)(0xc00022a500), (*int)(0xc00022ab70), (*int)(0xc00022ae00), (*int)(0xc00022ae50), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(0xc00022aea0), (*int)(0xc00022aef0), (*int)(0xc00022af60), (*int)(0xc00022afb0), (*int)(0xc00022b000), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil)}, Unallocated:[]int{51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 0, 1, 22, 18, 23, 24, 25, 26, 27, 20, 21, 28, 29, 30, 31, 32, 19, 38, 42, 39, 41, 40, 43, 33, 34, 35, 36, 37, 44, 45}, Attributes:[]v3.AllocationAttribute{v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674ec0), AttrSecondary:map[string]string{"namespace":"aws-service-broker", "node":"k8s-worker-23.example.com", "pod":"aws-servicebroker-fc58f95bb-bpb94"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674f30), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-9dd6685b9-cm5g2"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674fa0), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-cainjector-8545fdf87c-5hczm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675010), AttrSecondary:map[string]string{"namespace":"clair-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-85f59b5b87-xwqwb"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675080), AttrSecondary:map[string]string{"namespace":"clair-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-866dd75bb9-q7jgm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006750f0), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"filebeat-syslog-5fd9947874-tb988"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675160), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-7c8c64fd89-bmkr4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006751d0), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-qk574"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675240), AttrSecondary:map[string]string{"namespace":"lxos-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-558cf4654d-qhnzc"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006752b0), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"expire-check-worker-8nf9h"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675320), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"kube-state-metrics-597b7c5f5-kmgtv"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675390), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"tiller-deploy-786bc89dd8-h8lpg"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675400), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"default-http-backend-5f65599f44-mrdmz"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675470), AttrSecondary:map[string]string{"namespace":"snyk-monitor", "node":"k8s-worker-23.example.com", "pod":"snyk-monitor-6fc8c86d4c-w6rxl"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006754e0), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"prometheus-metrics-adapter-c74fd5b66-fk9p4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675550), AttrSecondary:map[string]string{"namespace":"monitoring", "node":"k8s-worker-23.example.com", "pod":"mon-prometheus-pushgateway-7987d7d4b6-jdjwr"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675630), AttrSecondary:map[string]string{"namespace":"rook-ceph-dev-primary", "node":"k8s-worker-23.example.com", "pod":"rook-ceph-dev-primary-backup-kp5pt"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006756a0), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"kube-iptables-tailer-ckml4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675710), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"expire-check-worker-jx7dd"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675780), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-6fjgw"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006757f0), AttrSecondary:map[string]string{"node":"k8s-worker-23.example.com", "type":"ipipTunnelAddress"}}}, Deleted:false}}
2020-10-23 14:46:32.088 [DEBUG][1] customresource.go 126: Update resource by name Key=IPAMBlock(100-80-6-0-26) Name="100-80-6-0-26" Resource="IPAMBlocks" Value=&v3.IPAMBlock{TypeMeta:v1.TypeMeta{Kind:"IPAMBlock", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"100-80-6-0-26", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"534575645", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMBlockSpec{CIDR:"100.80.6.0/26", Affinity:(*string)(0xc000674e90), StrictAffinity:false, Allocations:[]*int{(*int)(nil), (*int)(nil), (*int)(0xc00038cfa0), (*int)(0xc00038d020), (*int)(0xc00038d180), (*int)(0xc00038d230), (*int)(0xc00038d320), (*int)(0xc00038d370), (*int)(0xc00038d3f0), (*int)(0xc00038db20), (*int)(0xc00038d590), (*int)(0xc00038d640), (*int)(0xc00038d7c0), (*int)(0xc00022a040), (*int)(0xc00022a500), (*int)(0xc00022ab70), (*int)(0xc00022ae00), (*int)(0xc00022ae50), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(0xc00022aea0), (*int)(0xc00022aef0), (*int)(0xc00022af60), (*int)(0xc00022afb0), (*int)(0xc00022b000), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil)}, Unallocated:[]int{51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 0, 1, 22, 18, 23, 24, 25, 26, 27, 20, 21, 28, 29, 30, 31, 32, 19, 38, 42, 39, 41, 40, 43, 33, 34, 35, 36, 37, 44, 45}, Attributes:[]v3.AllocationAttribute{v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674ec0), AttrSecondary:map[string]string{"namespace":"aws-service-broker", "node":"k8s-worker-23.example.com", "pod":"aws-servicebroker-fc58f95bb-bpb94"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674f30), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-9dd6685b9-cm5g2"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674fa0), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-cainjector-8545fdf87c-5hczm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675010), AttrSecondary:map[string]string{"namespace":"clair-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-85f59b5b87-xwqwb"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675080), AttrSecondary:map[string]string{"namespace":"clair-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-866dd75bb9-q7jgm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006750f0), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"filebeat-syslog-5fd9947874-tb988"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675160), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-7c8c64fd89-bmkr4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006751d0), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-qk574"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675240), AttrSecondary:map[string]string{"namespace":"lxos-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-558cf4654d-qhnzc"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006752b0), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"expire-check-worker-8nf9h"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675320), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"kube-state-metrics-597b7c5f5-kmgtv"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675390), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"tiller-deploy-786bc89dd8-h8lpg"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675400), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"default-http-backend-5f65599f44-mrdmz"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675470), AttrSecondary:map[string]string{"namespace":"snyk-monitor", "node":"k8s-worker-23.example.com", "pod":"snyk-monitor-6fc8c86d4c-w6rxl"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006754e0), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"prometheus-metrics-adapter-c74fd5b66-fk9p4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675550), AttrSecondary:map[string]string{"namespace":"monitoring", "node":"k8s-worker-23.example.com", "pod":"mon-prometheus-pushgateway-7987d7d4b6-jdjwr"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675630), AttrSecondary:map[string]string{"namespace":"rook-ceph-dev-primary", "node":"k8s-worker-23.example.com", "pod":"rook-ceph-dev-primary-backup-kp5pt"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006756a0), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"kube-iptables-tailer-ckml4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675710), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"expire-check-worker-jx7dd"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000675780), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-6fjgw"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006757f0), AttrSecondary:map[string]string{"node":"k8s-worker-23.example.com", "type":"ipipTunnelAddress"}}}, Deleted:false}}
2020-10-23 14:46:32.098 [DEBUG][1] ipam.go 1374: Successfully released IPs from block cidr=100.80.6.0/26 handle="k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229"
2020-10-23 14:46:32.098 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229)
2020-10-23 14:46:32.098 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.098 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229) Name="k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.101 [DEBUG][1] ipam.go 1458: Deleting handle: k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229
2020-10-23 14:46:32.101 [DEBUG][1] k8s.go 536: Performing 'DeleteKVP' for IPAMHandleKey(id=k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229)
2020-10-23 14:46:32.102 [DEBUG][1] customresource.go 109: Update custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229) Resource="IPAMHandles" Value=&v3.IPAMHandle{TypeMeta:v1.TypeMeta{Kind:"IPAMHandle", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"534574977", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMHandleSpec{HandleID:"k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229", Block:map[string]int{}}}
2020-10-23 14:46:32.102 [DEBUG][1] customresource.go 126: Update resource by name Key=IPAMHandle(k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229) Name="k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229" Resource="IPAMHandles" Value=&v3.IPAMHandle{TypeMeta:v1.TypeMeta{Kind:"IPAMHandle", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"534574977", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMHandleSpec{HandleID:"k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229", Block:map[string]int{}}}
2020-10-23 14:46:32.108 [DEBUG][1] customresource.go 162: Delete custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229) Resource="IPAMHandles"
2020-10-23 14:46:32.108 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229) Resource="IPAMHandles" Revision="535777030"
2020-10-23 14:46:32.108 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229) Name="k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229" Namespace="" Resource="IPAMHandles" Revision="535777030"
2020-10-23 14:46:32.112 [DEBUG][1] customresource.go 185: Send delete request by name Key=IPAMHandle(k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229) Name="k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229" Resource="IPAMHandles"
2020-10-23 14:46:32.120 [DEBUG][1] ipam.go 1481: Decremented handle 'k8s-pod-network.008a4cb170c3763d58aefc0141871caac188150040943b775259dea7a1e25229' by 1
2020-10-23 14:46:32.120 [DEBUG][1] ipam.go 1625: Looking up node labels for host affinity cidr=100.80.6.0/26 host="k8s-worker-23.example.com"
2020-10-23 14:46:32.120 [DEBUG][1] k8s.go 564: Performing 'Get' for Node(k8s-worker-23.example.com)
2020-10-23 14:46:32.120 [DEBUG][1] node.go 114: Received Get request on Node type
2020-10-23 14:46:32.123 [INFO][1] ipam.go 1632: Node doesn't exist, no need to release affinity cidr=100.80.6.0/26 host="k8s-worker-23.example.com"
2020-10-23 14:46:32.123 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea'
2020-10-23 14:46:32.123 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea)
2020-10-23 14:46:32.123 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.123 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea) Name="k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.126 [DEBUG][1] ipam.go 1317: Querying block so we can release IPs by handle cidr=100.80.6.0/26 handle="k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea"
2020-10-23 14:46:32.126 [DEBUG][1] k8s.go 564: Performing 'Get' for BlockKey(cidr=100.80.6.0/26)
2020-10-23 14:46:32.126 [DEBUG][1] cidr.go 34: Converted IPNet to resource name IPNet="100.80.6.0/26" Name="100-80-6-0-26"
2020-10-23 14:46:32.126 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMBlock(100-80-6-0-26) Resource="IPAMBlocks" Revision=""
2020-10-23 14:46:32.126 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMBlock(100-80-6-0-26) Name="100-80-6-0-26" Namespace="" Resource="IPAMBlocks" Revision=""
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 349: Attribute indexes to release: [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 0 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 1 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 2 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 3 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 4 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 5 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 6 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 7 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 8 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 9 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 10 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 11 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 12 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 13 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 14 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 15 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 17 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 18 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 19 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam_block.go 300: 20 in [16]
2020-10-23 14:46:32.130 [DEBUG][1] ipam.go 1339: Block has 1 IPs with the given handle cidr=100.80.6.0/26 handle="k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea"
2020-10-23 14:46:32.130 [DEBUG][1] ipam.go 1361: Updating block to release IPs cidr=100.80.6.0/26 handle="k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea"
2020-10-23 14:46:32.130 [DEBUG][1] k8s.go 489: Performing 'Update' for &{Key:BlockKey(cidr=100.80.6.0/26) Value:0xc0006bc240 Revision:535777029 UID:0xc00011e060 TTL:0s}
2020-10-23 14:46:32.130 [DEBUG][1] cidr.go 34: Converted IPNet to resource name IPNet="100.80.6.0/26" Name="100-80-6-0-26"
2020-10-23 14:46:32.131 [DEBUG][1] customresource.go 109: Update custom Kubernetes resource Key=IPAMBlock(100-80-6-0-26) Resource="IPAMBlocks" Value=&v3.IPAMBlock{TypeMeta:v1.TypeMeta{Kind:"IPAMBlock", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"100-80-6-0-26", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"535777029", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMBlockSpec{CIDR:"100.80.6.0/26", Affinity:(*string)(0xc000674470), StrictAffinity:false, Allocations:[]*int{(*int)(nil), (*int)(nil), (*int)(0xc00047b9f0), (*int)(0xc00047baa0), (*int)(0xc00047bb20), (*int)(0xc00047bb70), (*int)(0xc00047bc00), (*int)(0xc00047bca0), (*int)(0xc00047bd10), (*int)(0xc0001329d0), (*int)(0xc00047bde0), (*int)(0xc00047bf90), (*int)(0xc000132500), (*int)(0xc000132bb0), (*int)(0xc000133240), (*int)(0xc000133a00), (*int)(0xc000042850), (*int)(0xc000042b60), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(0xc000042bc0), (*int)(0xc000042e80), (*int)(0xc000043200), (*int)(0xc000043640), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil)}, Unallocated:[]int{51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 0, 1, 22, 18, 23, 24, 25, 26, 27, 20, 21, 28, 29, 30, 31, 32, 19, 38, 42, 39, 41, 40, 43, 33, 34, 35, 36, 37, 44, 45, 46}, Attributes:[]v3.AllocationAttribute{v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006744a0), AttrSecondary:map[string]string{"namespace":"aws-service-broker", "node":"k8s-worker-23.example.com", "pod":"aws-servicebroker-fc58f95bb-bpb94"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674510), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-9dd6685b9-cm5g2"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674580), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-cainjector-8545fdf87c-5hczm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006745f0), AttrSecondary:map[string]string{"namespace":"clair-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-85f59b5b87-xwqwb"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674660), AttrSecondary:map[string]string{"namespace":"clair-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-866dd75bb9-q7jgm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006746d0), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"filebeat-syslog-5fd9947874-tb988"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674740), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-7c8c64fd89-bmkr4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006747b0), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-qk574"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674820), AttrSecondary:map[string]string{"namespace":"lxos-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-558cf4654d-qhnzc"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006748a0), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"expire-check-worker-8nf9h"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674920), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"kube-state-metrics-597b7c5f5-kmgtv"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674990), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"tiller-deploy-786bc89dd8-h8lpg"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674a00), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"default-http-backend-5f65599f44-mrdmz"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674a70), AttrSecondary:map[string]string{"namespace":"snyk-monitor", "node":"k8s-worker-23.example.com", "pod":"snyk-monitor-6fc8c86d4c-w6rxl"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674ae0), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"prometheus-metrics-adapter-c74fd5b66-fk9p4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674b50), AttrSecondary:map[string]string{"namespace":"monitoring", "node":"k8s-worker-23.example.com", "pod":"mon-prometheus-pushgateway-7987d7d4b6-jdjwr"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674c30), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"kube-iptables-tailer-ckml4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674ca0), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"expire-check-worker-jx7dd"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674d10), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-6fjgw"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674d80), AttrSecondary:map[string]string{"node":"k8s-worker-23.example.com", "type":"ipipTunnelAddress"}}}, Deleted:false}}
2020-10-23 14:46:32.131 [DEBUG][1] customresource.go 126: Update resource by name Key=IPAMBlock(100-80-6-0-26) Name="100-80-6-0-26" Resource="IPAMBlocks" Value=&v3.IPAMBlock{TypeMeta:v1.TypeMeta{Kind:"IPAMBlock", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"100-80-6-0-26", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"535777029", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMBlockSpec{CIDR:"100.80.6.0/26", Affinity:(*string)(0xc000674470), StrictAffinity:false, Allocations:[]*int{(*int)(nil), (*int)(nil), (*int)(0xc00047b9f0), (*int)(0xc00047baa0), (*int)(0xc00047bb20), (*int)(0xc00047bb70), (*int)(0xc00047bc00), (*int)(0xc00047bca0), (*int)(0xc00047bd10), (*int)(0xc0001329d0), (*int)(0xc00047bde0), (*int)(0xc00047bf90), (*int)(0xc000132500), (*int)(0xc000132bb0), (*int)(0xc000133240), (*int)(0xc000133a00), (*int)(0xc000042850), (*int)(0xc000042b60), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(0xc000042bc0), (*int)(0xc000042e80), (*int)(0xc000043200), (*int)(0xc000043640), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil)}, Unallocated:[]int{51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 0, 1, 22, 18, 23, 24, 25, 26, 27, 20, 21, 28, 29, 30, 31, 32, 19, 38, 42, 39, 41, 40, 43, 33, 34, 35, 36, 37, 44, 45, 46}, Attributes:[]v3.AllocationAttribute{v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006744a0), AttrSecondary:map[string]string{"namespace":"aws-service-broker", "node":"k8s-worker-23.example.com", "pod":"aws-servicebroker-fc58f95bb-bpb94"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674510), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-9dd6685b9-cm5g2"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674580), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-cainjector-8545fdf87c-5hczm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006745f0), AttrSecondary:map[string]string{"namespace":"clair-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-85f59b5b87-xwqwb"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674660), AttrSecondary:map[string]string{"namespace":"clair-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-866dd75bb9-q7jgm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006746d0), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"filebeat-syslog-5fd9947874-tb988"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674740), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-7c8c64fd89-bmkr4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006747b0), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-qk574"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674820), AttrSecondary:map[string]string{"namespace":"lxos-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-558cf4654d-qhnzc"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0006748a0), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"expire-check-worker-8nf9h"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674920), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"kube-state-metrics-597b7c5f5-kmgtv"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674990), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"tiller-deploy-786bc89dd8-h8lpg"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674a00), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"default-http-backend-5f65599f44-mrdmz"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674a70), AttrSecondary:map[string]string{"namespace":"snyk-monitor", "node":"k8s-worker-23.example.com", "pod":"snyk-monitor-6fc8c86d4c-w6rxl"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674ae0), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"prometheus-metrics-adapter-c74fd5b66-fk9p4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674b50), AttrSecondary:map[string]string{"namespace":"monitoring", "node":"k8s-worker-23.example.com", "pod":"mon-prometheus-pushgateway-7987d7d4b6-jdjwr"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674c30), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"kube-iptables-tailer-ckml4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674ca0), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"expire-check-worker-jx7dd"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674d10), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-6fjgw"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000674d80), AttrSecondary:map[string]string{"node":"k8s-worker-23.example.com", "type":"ipipTunnelAddress"}}}, Deleted:false}}
2020-10-23 14:46:32.142 [DEBUG][1] ipam.go 1374: Successfully released IPs from block cidr=100.80.6.0/26 handle="k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea"
2020-10-23 14:46:32.142 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea)
2020-10-23 14:46:32.142 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.142 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea) Name="k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.145 [DEBUG][1] ipam.go 1458: Deleting handle: k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea
2020-10-23 14:46:32.145 [DEBUG][1] k8s.go 536: Performing 'DeleteKVP' for IPAMHandleKey(id=k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea)
2020-10-23 14:46:32.145 [DEBUG][1] customresource.go 109: Update custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea) Resource="IPAMHandles" Value=&v3.IPAMHandle{TypeMeta:v1.TypeMeta{Kind:"IPAMHandle", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"534574996", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMHandleSpec{HandleID:"k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea", Block:map[string]int{}}}
2020-10-23 14:46:32.145 [DEBUG][1] customresource.go 126: Update resource by name Key=IPAMHandle(k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea) Name="k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea" Resource="IPAMHandles" Value=&v3.IPAMHandle{TypeMeta:v1.TypeMeta{Kind:"IPAMHandle", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"534574996", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMHandleSpec{HandleID:"k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea", Block:map[string]int{}}}
2020-10-23 14:46:32.150 [DEBUG][1] customresource.go 162: Delete custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea) Resource="IPAMHandles"
2020-10-23 14:46:32.150 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea) Resource="IPAMHandles" Revision="535777033"
2020-10-23 14:46:32.151 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea) Name="k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea" Namespace="" Resource="IPAMHandles" Revision="535777033"
2020-10-23 14:46:32.153 [DEBUG][1] customresource.go 185: Send delete request by name Key=IPAMHandle(k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea) Name="k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea" Resource="IPAMHandles"
2020-10-23 14:46:32.162 [DEBUG][1] ipam.go 1481: Decremented handle 'k8s-pod-network.cc4687b39f9fedc88793e65b4c1697276811f457f6a8960d9d4eb9ba29c773ea' by 1
2020-10-23 14:46:32.162 [DEBUG][1] ipam.go 1625: Looking up node labels for host affinity cidr=100.80.6.0/26 host="k8s-worker-23.example.com"
2020-10-23 14:46:32.162 [DEBUG][1] k8s.go 564: Performing 'Get' for Node(k8s-worker-23.example.com)
2020-10-23 14:46:32.162 [DEBUG][1] node.go 114: Received Get request on Node type
2020-10-23 14:46:32.165 [INFO][1] ipam.go 1632: Node doesn't exist, no need to release affinity cidr=100.80.6.0/26 host="k8s-worker-23.example.com"
2020-10-23 14:46:32.165 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006'
2020-10-23 14:46:32.165 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006)
2020-10-23 14:46:32.165 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.165 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006) Name="k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.169 [DEBUG][1] ipam.go 1317: Querying block so we can release IPs by handle cidr=100.80.6.0/26 handle="k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006"
2020-10-23 14:46:32.169 [DEBUG][1] k8s.go 564: Performing 'Get' for BlockKey(cidr=100.80.6.0/26)
2020-10-23 14:46:32.169 [DEBUG][1] cidr.go 34: Converted IPNet to resource name IPNet="100.80.6.0/26" Name="100-80-6-0-26"
2020-10-23 14:46:32.169 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMBlock(100-80-6-0-26) Resource="IPAMBlocks" Revision=""
2020-10-23 14:46:32.169 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMBlock(100-80-6-0-26) Name="100-80-6-0-26" Namespace="" Resource="IPAMBlocks" Revision=""
2020-10-23 14:46:32.172 [DEBUG][1] ipam_block.go 349: Attribute indexes to release: [16]
2020-10-23 14:46:32.172 [DEBUG][1] ipam_block.go 300: 0 in [16]
2020-10-23 14:46:32.172 [DEBUG][1] ipam_block.go 300: 1 in [16]
2020-10-23 14:46:32.172 [DEBUG][1] ipam_block.go 300: 2 in [16]
2020-10-23 14:46:32.172 [DEBUG][1] ipam_block.go 300: 3 in [16]
2020-10-23 14:46:32.172 [DEBUG][1] ipam_block.go 300: 4 in [16]
2020-10-23 14:46:32.173 [DEBUG][1] ipam_block.go 300: 5 in [16]
2020-10-23 14:46:32.173 [DEBUG][1] ipam_block.go 300: 6 in [16]
2020-10-23 14:46:32.173 [DEBUG][1] ipam_block.go 300: 7 in [16]
2020-10-23 14:46:32.173 [DEBUG][1] ipam_block.go 300: 8 in [16]
2020-10-23 14:46:32.173 [DEBUG][1] ipam_block.go 300: 9 in [16]
2020-10-23 14:46:32.173 [DEBUG][1] ipam_block.go 300: 10 in [16]
2020-10-23 14:46:32.173 [DEBUG][1] ipam_block.go 300: 11 in [16]
2020-10-23 14:46:32.173 [DEBUG][1] ipam_block.go 300: 12 in [16]
2020-10-23 14:46:32.173 [DEBUG][1] ipam_block.go 300: 13 in [16]
2020-10-23 14:46:32.173 [DEBUG][1] ipam_block.go 300: 14 in [16]
2020-10-23 14:46:32.173 [DEBUG][1] ipam_block.go 300: 15 in [16]
2020-10-23 14:46:32.173 [DEBUG][1] ipam_block.go 300: 17 in [16]
2020-10-23 14:46:32.173 [DEBUG][1] ipam_block.go 300: 18 in [16]
2020-10-23 14:46:32.173 [DEBUG][1] ipam_block.go 300: 19 in [16]
2020-10-23 14:46:32.173 [DEBUG][1] ipam.go 1339: Block has 1 IPs with the given handle cidr=100.80.6.0/26 handle="k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006"
2020-10-23 14:46:32.173 [DEBUG][1] ipam.go 1361: Updating block to release IPs cidr=100.80.6.0/26 handle="k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006"
2020-10-23 14:46:32.173 [DEBUG][1] k8s.go 489: Performing 'Update' for &{Key:BlockKey(cidr=100.80.6.0/26) Value:0xc0007ac1b0 Revision:535777032 UID:0xc00024c060 TTL:0s}
2020-10-23 14:46:32.173 [DEBUG][1] cidr.go 34: Converted IPNet to resource name IPNet="100.80.6.0/26" Name="100-80-6-0-26"
2020-10-23 14:46:32.173 [DEBUG][1] customresource.go 109: Update custom Kubernetes resource Key=IPAMBlock(100-80-6-0-26) Resource="IPAMBlocks" Value=&v3.IPAMBlock{TypeMeta:v1.TypeMeta{Kind:"IPAMBlock", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"100-80-6-0-26", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"535777032", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMBlockSpec{CIDR:"100.80.6.0/26", Affinity:(*string)(0xc000738520), StrictAffinity:false, Allocations:[]*int{(*int)(nil), (*int)(nil), (*int)(0xc000805ac0), (*int)(0xc000805bb0), (*int)(0xc000805c00), (*int)(0xc000805c80), (*int)(0xc000805df0), (*int)(0xc000805f40), (*int)(0xc00016a080), (*int)(0xc00016a390), (*int)(0xc00016a190), (*int)(0xc00016a2f0), (*int)(0xc00016a340), (*int)(0xc00016a3e0), (*int)(0xc00016a430), (*int)(0xc00016a480), (*int)(0xc00016a570), (*int)(0xc00016a6b0), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(0xc00016a7f0), (*int)(0xc00016a910), (*int)(0xc00016aa40), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil)}, Unallocated:[]int{51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 0, 1, 22, 18, 23, 24, 25, 26, 27, 20, 21, 28, 29, 30, 31, 32, 19, 38, 42, 39, 41, 40, 43, 33, 34, 35, 36, 37, 44, 45, 46, 47}, Attributes:[]v3.AllocationAttribute{v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738550), AttrSecondary:map[string]string{"namespace":"aws-service-broker", "node":"k8s-worker-23.example.com", "pod":"aws-servicebroker-fc58f95bb-bpb94"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0007385c0), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-9dd6685b9-cm5g2"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738640), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-cainjector-8545fdf87c-5hczm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0007386b0), AttrSecondary:map[string]string{"namespace":"clair-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-85f59b5b87-xwqwb"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738720), AttrSecondary:map[string]string{"namespace":"clair-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-866dd75bb9-q7jgm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738790), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"filebeat-syslog-5fd9947874-tb988"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738800), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-7c8c64fd89-bmkr4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738870), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-qk574"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0007388e0), AttrSecondary:map[string]string{"namespace":"lxos-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-558cf4654d-qhnzc"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738980), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"expire-check-worker-8nf9h"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0007389f0), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"kube-state-metrics-597b7c5f5-kmgtv"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738a80), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"tiller-deploy-786bc89dd8-h8lpg"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738af0), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"default-http-backend-5f65599f44-mrdmz"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738b60), AttrSecondary:map[string]string{"namespace":"snyk-monitor", "node":"k8s-worker-23.example.com", "pod":"snyk-monitor-6fc8c86d4c-w6rxl"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738bd0), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"prometheus-metrics-adapter-c74fd5b66-fk9p4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738c40), AttrSecondary:map[string]string{"namespace":"monitoring", "node":"k8s-worker-23.example.com", "pod":"mon-prometheus-pushgateway-7987d7d4b6-jdjwr"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738d20), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"expire-check-worker-jx7dd"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738d90), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-6fjgw"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738e00), AttrSecondary:map[string]string{"node":"k8s-worker-23.example.com", "type":"ipipTunnelAddress"}}}, Deleted:false}}
2020-10-23 14:46:32.173 [DEBUG][1] customresource.go 126: Update resource by name Key=IPAMBlock(100-80-6-0-26) Name="100-80-6-0-26" Resource="IPAMBlocks" Value=&v3.IPAMBlock{TypeMeta:v1.TypeMeta{Kind:"IPAMBlock", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"100-80-6-0-26", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"535777032", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMBlockSpec{CIDR:"100.80.6.0/26", Affinity:(*string)(0xc000738520), StrictAffinity:false, Allocations:[]*int{(*int)(nil), (*int)(nil), (*int)(0xc000805ac0), (*int)(0xc000805bb0), (*int)(0xc000805c00), (*int)(0xc000805c80), (*int)(0xc000805df0), (*int)(0xc000805f40), (*int)(0xc00016a080), (*int)(0xc00016a390), (*int)(0xc00016a190), (*int)(0xc00016a2f0), (*int)(0xc00016a340), (*int)(0xc00016a3e0), (*int)(0xc00016a430), (*int)(0xc00016a480), (*int)(0xc00016a570), (*int)(0xc00016a6b0), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(0xc00016a7f0), (*int)(0xc00016a910), (*int)(0xc00016aa40), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil)}, Unallocated:[]int{51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 0, 1, 22, 18, 23, 24, 25, 26, 27, 20, 21, 28, 29, 30, 31, 32, 19, 38, 42, 39, 41, 40, 43, 33, 34, 35, 36, 37, 44, 45, 46, 47}, Attributes:[]v3.AllocationAttribute{v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738550), AttrSecondary:map[string]string{"namespace":"aws-service-broker", "node":"k8s-worker-23.example.com", "pod":"aws-servicebroker-fc58f95bb-bpb94"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0007385c0), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-9dd6685b9-cm5g2"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738640), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-cainjector-8545fdf87c-5hczm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0007386b0), AttrSecondary:map[string]string{"namespace":"clair-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-85f59b5b87-xwqwb"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738720), AttrSecondary:map[string]string{"namespace":"clair-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-866dd75bb9-q7jgm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738790), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"filebeat-syslog-5fd9947874-tb988"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738800), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-7c8c64fd89-bmkr4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738870), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-qk574"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0007388e0), AttrSecondary:map[string]string{"namespace":"lxos-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-558cf4654d-qhnzc"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738980), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"expire-check-worker-8nf9h"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0007389f0), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"kube-state-metrics-597b7c5f5-kmgtv"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738a80), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"tiller-deploy-786bc89dd8-h8lpg"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738af0), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"default-http-backend-5f65599f44-mrdmz"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738b60), AttrSecondary:map[string]string{"namespace":"snyk-monitor", "node":"k8s-worker-23.example.com", "pod":"snyk-monitor-6fc8c86d4c-w6rxl"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738bd0), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"prometheus-metrics-adapter-c74fd5b66-fk9p4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738c40), AttrSecondary:map[string]string{"namespace":"monitoring", "node":"k8s-worker-23.example.com", "pod":"mon-prometheus-pushgateway-7987d7d4b6-jdjwr"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738d20), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"expire-check-worker-jx7dd"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738d90), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-6fjgw"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000738e00), AttrSecondary:map[string]string{"node":"k8s-worker-23.example.com", "type":"ipipTunnelAddress"}}}, Deleted:false}}
2020-10-23 14:46:32.185 [DEBUG][1] ipam.go 1374: Successfully released IPs from block cidr=100.80.6.0/26 handle="k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006"
2020-10-23 14:46:32.185 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006)
2020-10-23 14:46:32.185 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.185 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006) Name="k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.188 [DEBUG][1] ipam.go 1458: Deleting handle: k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006
2020-10-23 14:46:32.188 [DEBUG][1] k8s.go 536: Performing 'DeleteKVP' for IPAMHandleKey(id=k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006)
2020-10-23 14:46:32.188 [DEBUG][1] customresource.go 109: Update custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006) Resource="IPAMHandles" Value=&v3.IPAMHandle{TypeMeta:v1.TypeMeta{Kind:"IPAMHandle", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"534575016", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMHandleSpec{HandleID:"k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006", Block:map[string]int{}}}
2020-10-23 14:46:32.188 [DEBUG][1] customresource.go 126: Update resource by name Key=IPAMHandle(k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006) Name="k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006" Resource="IPAMHandles" Value=&v3.IPAMHandle{TypeMeta:v1.TypeMeta{Kind:"IPAMHandle", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"534575016", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMHandleSpec{HandleID:"k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006", Block:map[string]int{}}}
2020-10-23 14:46:32.195 [DEBUG][1] customresource.go 162: Delete custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006) Resource="IPAMHandles"
2020-10-23 14:46:32.195 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006) Resource="IPAMHandles" Revision="535777036"
2020-10-23 14:46:32.195 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006) Name="k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006" Namespace="" Resource="IPAMHandles" Revision="535777036"
2020-10-23 14:46:32.199 [DEBUG][1] customresource.go 185: Send delete request by name Key=IPAMHandle(k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006) Name="k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006" Resource="IPAMHandles"
2020-10-23 14:46:32.208 [DEBUG][1] ipam.go 1481: Decremented handle 'k8s-pod-network.8ca668cb0ef8af977a3858a4de99a8af4332d9ec15c273b5902eb575a4dd4006' by 1
2020-10-23 14:46:32.208 [DEBUG][1] ipam.go 1625: Looking up node labels for host affinity cidr=100.80.6.0/26 host="k8s-worker-23.example.com"
2020-10-23 14:46:32.208 [DEBUG][1] k8s.go 564: Performing 'Get' for Node(k8s-worker-23.example.com)
2020-10-23 14:46:32.208 [DEBUG][1] node.go 114: Received Get request on Node type
2020-10-23 14:46:32.212 [INFO][1] ipam.go 1632: Node doesn't exist, no need to release affinity cidr=100.80.6.0/26 host="k8s-worker-23.example.com"
2020-10-23 14:46:32.212 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318'
2020-10-23 14:46:32.212 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318)
2020-10-23 14:46:32.212 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.212 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318) Name="k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.215 [DEBUG][1] ipam.go 1317: Querying block so we can release IPs by handle cidr=100.80.6.0/26 handle="k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318"
2020-10-23 14:46:32.215 [DEBUG][1] k8s.go 564: Performing 'Get' for BlockKey(cidr=100.80.6.0/26)
2020-10-23 14:46:32.215 [DEBUG][1] cidr.go 34: Converted IPNet to resource name IPNet="100.80.6.0/26" Name="100-80-6-0-26"
2020-10-23 14:46:32.215 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMBlock(100-80-6-0-26) Resource="IPAMBlocks" Revision=""
2020-10-23 14:46:32.215 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMBlock(100-80-6-0-26) Name="100-80-6-0-26" Namespace="" Resource="IPAMBlocks" Revision=""
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 349: Attribute indexes to release: [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 300: 0 in [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 300: 1 in [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 300: 2 in [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 300: 3 in [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 300: 4 in [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 300: 5 in [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 300: 6 in [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 300: 7 in [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 300: 8 in [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 300: 9 in [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 300: 10 in [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 300: 11 in [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 300: 12 in [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 300: 13 in [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 300: 14 in [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 300: 15 in [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 300: 17 in [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam_block.go 300: 18 in [16]
2020-10-23 14:46:32.219 [DEBUG][1] ipam.go 1339: Block has 1 IPs with the given handle cidr=100.80.6.0/26 handle="k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318"
2020-10-23 14:46:32.219 [DEBUG][1] ipam.go 1361: Updating block to release IPs cidr=100.80.6.0/26 handle="k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318"
2020-10-23 14:46:32.220 [DEBUG][1] k8s.go 489: Performing 'Update' for &{Key:BlockKey(cidr=100.80.6.0/26) Value:0xc0006bc6c0 Revision:535777035 UID:0xc00011ef00 TTL:0s}
2020-10-23 14:46:32.220 [DEBUG][1] cidr.go 34: Converted IPNet to resource name IPNet="100.80.6.0/26" Name="100-80-6-0-26"
2020-10-23 14:46:32.220 [DEBUG][1] customresource.go 109: Update custom Kubernetes resource Key=IPAMBlock(100-80-6-0-26) Resource="IPAMBlocks" Value=&v3.IPAMBlock{TypeMeta:v1.TypeMeta{Kind:"IPAMBlock", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"100-80-6-0-26", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"535777035", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMBlockSpec{CIDR:"100.80.6.0/26", Affinity:(*string)(0xc0005b9120), StrictAffinity:false, Allocations:[]*int{(*int)(nil), (*int)(nil), (*int)(0xc000bd84f0), (*int)(0xc000bd8540), (*int)(0xc000bd8590), (*int)(0xc000bd85e0), (*int)(0xc000bd8630), (*int)(0xc000bd8680), (*int)(0xc000bd86d0), (*int)(0xc000bd8810), (*int)(0xc000bd8720), (*int)(0xc000bd8770), (*int)(0xc000bd87c0), (*int)(0xc000bd8860), (*int)(0xc000bd88b0), (*int)(0xc000bd8900), (*int)(0xc000bd8950), (*int)(0xc000bd89a0), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(0xc000bd89f0), (*int)(0xc000bd8a40), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil)}, Unallocated:[]int{51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 0, 1, 22, 18, 23, 24, 25, 26, 27, 20, 21, 28, 29, 30, 31, 32, 19, 38, 42, 39, 41, 40, 43, 33, 34, 35, 36, 37, 44, 45, 46, 47, 48}, Attributes:[]v3.AllocationAttribute{v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9150), AttrSecondary:map[string]string{"namespace":"aws-service-broker", "node":"k8s-worker-23.example.com", "pod":"aws-servicebroker-fc58f95bb-bpb94"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b91c0), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-9dd6685b9-cm5g2"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9230), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-cainjector-8545fdf87c-5hczm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b92a0), AttrSecondary:map[string]string{"namespace":"clair-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-85f59b5b87-xwqwb"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9310), AttrSecondary:map[string]string{"namespace":"clair-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-866dd75bb9-q7jgm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9380), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"filebeat-syslog-5fd9947874-tb988"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b93f0), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-7c8c64fd89-bmkr4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9460), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-qk574"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b94d0), AttrSecondary:map[string]string{"namespace":"lxos-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-558cf4654d-qhnzc"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9540), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"expire-check-worker-8nf9h"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b95b0), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"kube-state-metrics-597b7c5f5-kmgtv"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9620), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"tiller-deploy-786bc89dd8-h8lpg"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9690), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"default-http-backend-5f65599f44-mrdmz"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9700), AttrSecondary:map[string]string{"namespace":"snyk-monitor", "node":"k8s-worker-23.example.com", "pod":"snyk-monitor-6fc8c86d4c-w6rxl"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9770), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"prometheus-metrics-adapter-c74fd5b66-fk9p4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b97e0), AttrSecondary:map[string]string{"namespace":"monitoring", "node":"k8s-worker-23.example.com", "pod":"mon-prometheus-pushgateway-7987d7d4b6-jdjwr"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b98c0), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-6fjgw"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9940), AttrSecondary:map[string]string{"node":"k8s-worker-23.example.com", "type":"ipipTunnelAddress"}}}, Deleted:false}}
2020-10-23 14:46:32.220 [DEBUG][1] customresource.go 126: Update resource by name Key=IPAMBlock(100-80-6-0-26) Name="100-80-6-0-26" Resource="IPAMBlocks" Value=&v3.IPAMBlock{TypeMeta:v1.TypeMeta{Kind:"IPAMBlock", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"100-80-6-0-26", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"535777035", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMBlockSpec{CIDR:"100.80.6.0/26", Affinity:(*string)(0xc0005b9120), StrictAffinity:false, Allocations:[]*int{(*int)(nil), (*int)(nil), (*int)(0xc000bd84f0), (*int)(0xc000bd8540), (*int)(0xc000bd8590), (*int)(0xc000bd85e0), (*int)(0xc000bd8630), (*int)(0xc000bd8680), (*int)(0xc000bd86d0), (*int)(0xc000bd8810), (*int)(0xc000bd8720), (*int)(0xc000bd8770), (*int)(0xc000bd87c0), (*int)(0xc000bd8860), (*int)(0xc000bd88b0), (*int)(0xc000bd8900), (*int)(0xc000bd8950), (*int)(0xc000bd89a0), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(0xc000bd89f0), (*int)(0xc000bd8a40), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil)}, Unallocated:[]int{51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 0, 1, 22, 18, 23, 24, 25, 26, 27, 20, 21, 28, 29, 30, 31, 32, 19, 38, 42, 39, 41, 40, 43, 33, 34, 35, 36, 37, 44, 45, 46, 47, 48}, Attributes:[]v3.AllocationAttribute{v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9150), AttrSecondary:map[string]string{"namespace":"aws-service-broker", "node":"k8s-worker-23.example.com", "pod":"aws-servicebroker-fc58f95bb-bpb94"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b91c0), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-9dd6685b9-cm5g2"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9230), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-cainjector-8545fdf87c-5hczm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b92a0), AttrSecondary:map[string]string{"namespace":"clair-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-85f59b5b87-xwqwb"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9310), AttrSecondary:map[string]string{"namespace":"clair-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-866dd75bb9-q7jgm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9380), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"filebeat-syslog-5fd9947874-tb988"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b93f0), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-7c8c64fd89-bmkr4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9460), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-qk574"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b94d0), AttrSecondary:map[string]string{"namespace":"lxos-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-558cf4654d-qhnzc"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9540), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"expire-check-worker-8nf9h"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b95b0), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"kube-state-metrics-597b7c5f5-kmgtv"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9620), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"tiller-deploy-786bc89dd8-h8lpg"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9690), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"default-http-backend-5f65599f44-mrdmz"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9700), AttrSecondary:map[string]string{"namespace":"snyk-monitor", "node":"k8s-worker-23.example.com", "pod":"snyk-monitor-6fc8c86d4c-w6rxl"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9770), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"prometheus-metrics-adapter-c74fd5b66-fk9p4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b97e0), AttrSecondary:map[string]string{"namespace":"monitoring", "node":"k8s-worker-23.example.com", "pod":"mon-prometheus-pushgateway-7987d7d4b6-jdjwr"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b98c0), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-6fjgw"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc0005b9940), AttrSecondary:map[string]string{"node":"k8s-worker-23.example.com", "type":"ipipTunnelAddress"}}}, Deleted:false}}
2020-10-23 14:46:32.228 [DEBUG][1] ipam.go 1374: Successfully released IPs from block cidr=100.80.6.0/26 handle="k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318"
2020-10-23 14:46:32.228 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318)
2020-10-23 14:46:32.228 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.228 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318) Name="k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.232 [DEBUG][1] ipam.go 1458: Deleting handle: k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318
2020-10-23 14:46:32.232 [DEBUG][1] k8s.go 536: Performing 'DeleteKVP' for IPAMHandleKey(id=k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318)
2020-10-23 14:46:32.232 [DEBUG][1] customresource.go 109: Update custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318) Resource="IPAMHandles" Value=&v3.IPAMHandle{TypeMeta:v1.TypeMeta{Kind:"IPAMHandle", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"534575025", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMHandleSpec{HandleID:"k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318", Block:map[string]int{}}}
2020-10-23 14:46:32.232 [DEBUG][1] customresource.go 126: Update resource by name Key=IPAMHandle(k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318) Name="k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318" Resource="IPAMHandles" Value=&v3.IPAMHandle{TypeMeta:v1.TypeMeta{Kind:"IPAMHandle", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"534575025", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMHandleSpec{HandleID:"k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318", Block:map[string]int{}}}
2020-10-23 14:46:32.237 [DEBUG][1] customresource.go 162: Delete custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318) Resource="IPAMHandles"
2020-10-23 14:46:32.237 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318) Resource="IPAMHandles" Revision="535777039"
2020-10-23 14:46:32.237 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318) Name="k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318" Namespace="" Resource="IPAMHandles" Revision="535777039"
2020-10-23 14:46:32.240 [DEBUG][1] customresource.go 185: Send delete request by name Key=IPAMHandle(k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318) Name="k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318" Resource="IPAMHandles"
2020-10-23 14:46:32.249 [DEBUG][1] ipam.go 1481: Decremented handle 'k8s-pod-network.ff0377d9a4b78db553ea7813a767d59b920abe380bd2e87492c9f19718261318' by 1
2020-10-23 14:46:32.249 [DEBUG][1] ipam.go 1625: Looking up node labels for host affinity cidr=100.80.6.0/26 host="k8s-worker-23.example.com"
2020-10-23 14:46:32.249 [DEBUG][1] k8s.go 564: Performing 'Get' for Node(k8s-worker-23.example.com)
2020-10-23 14:46:32.249 [DEBUG][1] node.go 114: Received Get request on Node type
2020-10-23 14:46:32.252 [INFO][1] ipam.go 1632: Node doesn't exist, no need to release affinity cidr=100.80.6.0/26 host="k8s-worker-23.example.com"
2020-10-23 14:46:32.252 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b'
2020-10-23 14:46:32.252 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b)
2020-10-23 14:46:32.252 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.252 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b) Name="k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.255 [DEBUG][1] ipam.go 1317: Querying block so we can release IPs by handle cidr=100.80.6.0/26 handle="k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b"
2020-10-23 14:46:32.255 [DEBUG][1] k8s.go 564: Performing 'Get' for BlockKey(cidr=100.80.6.0/26)
2020-10-23 14:46:32.255 [DEBUG][1] cidr.go 34: Converted IPNet to resource name IPNet="100.80.6.0/26" Name="100-80-6-0-26"
2020-10-23 14:46:32.255 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMBlock(100-80-6-0-26) Resource="IPAMBlocks" Revision=""
2020-10-23 14:46:32.255 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMBlock(100-80-6-0-26) Name="100-80-6-0-26" Namespace="" Resource="IPAMBlocks" Revision=""
2020-10-23 14:46:32.259 [DEBUG][1] ipam_block.go 349: Attribute indexes to release: [16]
2020-10-23 14:46:32.259 [DEBUG][1] ipam_block.go 300: 0 in [16]
2020-10-23 14:46:32.259 [DEBUG][1] ipam_block.go 300: 1 in [16]
2020-10-23 14:46:32.259 [DEBUG][1] ipam_block.go 300: 2 in [16]
2020-10-23 14:46:32.259 [DEBUG][1] ipam_block.go 300: 3 in [16]
2020-10-23 14:46:32.259 [DEBUG][1] ipam_block.go 300: 4 in [16]
2020-10-23 14:46:32.259 [DEBUG][1] ipam_block.go 300: 5 in [16]
2020-10-23 14:46:32.259 [DEBUG][1] ipam_block.go 300: 6 in [16]
2020-10-23 14:46:32.259 [DEBUG][1] ipam_block.go 300: 7 in [16]
2020-10-23 14:46:32.259 [DEBUG][1] ipam_block.go 300: 8 in [16]
2020-10-23 14:46:32.259 [DEBUG][1] ipam_block.go 300: 9 in [16]
2020-10-23 14:46:32.259 [DEBUG][1] ipam_block.go 300: 10 in [16]
2020-10-23 14:46:32.260 [DEBUG][1] ipam_block.go 300: 11 in [16]
2020-10-23 14:46:32.260 [DEBUG][1] ipam_block.go 300: 12 in [16]
2020-10-23 14:46:32.260 [DEBUG][1] ipam_block.go 300: 13 in [16]
2020-10-23 14:46:32.260 [DEBUG][1] ipam_block.go 300: 14 in [16]
2020-10-23 14:46:32.260 [DEBUG][1] ipam_block.go 300: 15 in [16]
2020-10-23 14:46:32.260 [DEBUG][1] ipam_block.go 300: 17 in [16]
2020-10-23 14:46:32.260 [DEBUG][1] ipam.go 1339: Block has 1 IPs with the given handle cidr=100.80.6.0/26 handle="k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b"
2020-10-23 14:46:32.260 [DEBUG][1] ipam.go 1361: Updating block to release IPs cidr=100.80.6.0/26 handle="k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b"
2020-10-23 14:46:32.260 [DEBUG][1] k8s.go 489: Performing 'Update' for &{Key:BlockKey(cidr=100.80.6.0/26) Value:0xc00017c2d0 Revision:535777038 UID:0xc0005ff240 TTL:0s}
2020-10-23 14:46:32.260 [DEBUG][1] cidr.go 34: Converted IPNet to resource name IPNet="100.80.6.0/26" Name="100-80-6-0-26"
2020-10-23 14:46:32.260 [DEBUG][1] customresource.go 109: Update custom Kubernetes resource Key=IPAMBlock(100-80-6-0-26) Resource="IPAMBlocks" Value=&v3.IPAMBlock{TypeMeta:v1.TypeMeta{Kind:"IPAMBlock", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"100-80-6-0-26", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"535777038", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMBlockSpec{CIDR:"100.80.6.0/26", Affinity:(*string)(0xc00058e740), StrictAffinity:false, Allocations:[]*int{(*int)(nil), (*int)(nil), (*int)(0xc000ae6260), (*int)(0xc000ae62b0), (*int)(0xc000ae6300), (*int)(0xc000ae6350), (*int)(0xc000ae6660), (*int)(0xc000ae66b0), (*int)(0xc000ae6700), (*int)(0xc000ae68e0), (*int)(0xc000ae67d0), (*int)(0xc000ae6820), (*int)(0xc000ae6890), (*int)(0xc000ae6930), (*int)(0xc000ae6980), (*int)(0xc000ae6cf0), (*int)(0xc000ae6d40), (*int)(0xc000ae6d90), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(0xc000ae6ee0), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil)}, Unallocated:[]int{51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 0, 1, 22, 18, 23, 24, 25, 26, 27, 20, 21, 28, 29, 30, 31, 32, 19, 38, 42, 39, 41, 40, 43, 33, 34, 35, 36, 37, 44, 45, 46, 47, 48, 49}, Attributes:[]v3.AllocationAttribute{v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058e770), AttrSecondary:map[string]string{"namespace":"aws-service-broker", "node":"k8s-worker-23.example.com", "pod":"aws-servicebroker-fc58f95bb-bpb94"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058e820), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-9dd6685b9-cm5g2"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058e8a0), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-cainjector-8545fdf87c-5hczm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058e930), AttrSecondary:map[string]string{"namespace":"clair-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-85f59b5b87-xwqwb"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058e9b0), AttrSecondary:map[string]string{"namespace":"clair-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-866dd75bb9-q7jgm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058ea40), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"filebeat-syslog-5fd9947874-tb988"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058eac0), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-7c8c64fd89-bmkr4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058eb40), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-qk574"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058ebc0), AttrSecondary:map[string]string{"namespace":"lxos-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-558cf4654d-qhnzc"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058ec50), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"expire-check-worker-8nf9h"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058ece0), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"kube-state-metrics-597b7c5f5-kmgtv"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058ed60), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"tiller-deploy-786bc89dd8-h8lpg"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058ede0), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"default-http-backend-5f65599f44-mrdmz"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058ee50), AttrSecondary:map[string]string{"namespace":"snyk-monitor", "node":"k8s-worker-23.example.com", "pod":"snyk-monitor-6fc8c86d4c-w6rxl"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058eed0), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"prometheus-metrics-adapter-c74fd5b66-fk9p4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058ef50), AttrSecondary:map[string]string{"namespace":"monitoring", "node":"k8s-worker-23.example.com", "pod":"mon-prometheus-pushgateway-7987d7d4b6-jdjwr"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058f050), AttrSecondary:map[string]string{"node":"k8s-worker-23.example.com", "type":"ipipTunnelAddress"}}}, Deleted:false}}
2020-10-23 14:46:32.260 [DEBUG][1] customresource.go 126: Update resource by name Key=IPAMBlock(100-80-6-0-26) Name="100-80-6-0-26" Resource="IPAMBlocks" Value=&v3.IPAMBlock{TypeMeta:v1.TypeMeta{Kind:"IPAMBlock", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"100-80-6-0-26", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"535777038", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMBlockSpec{CIDR:"100.80.6.0/26", Affinity:(*string)(0xc00058e740), StrictAffinity:false, Allocations:[]*int{(*int)(nil), (*int)(nil), (*int)(0xc000ae6260), (*int)(0xc000ae62b0), (*int)(0xc000ae6300), (*int)(0xc000ae6350), (*int)(0xc000ae6660), (*int)(0xc000ae66b0), (*int)(0xc000ae6700), (*int)(0xc000ae68e0), (*int)(0xc000ae67d0), (*int)(0xc000ae6820), (*int)(0xc000ae6890), (*int)(0xc000ae6930), (*int)(0xc000ae6980), (*int)(0xc000ae6cf0), (*int)(0xc000ae6d40), (*int)(0xc000ae6d90), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(0xc000ae6ee0), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil)}, Unallocated:[]int{51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 0, 1, 22, 18, 23, 24, 25, 26, 27, 20, 21, 28, 29, 30, 31, 32, 19, 38, 42, 39, 41, 40, 43, 33, 34, 35, 36, 37, 44, 45, 46, 47, 48, 49}, Attributes:[]v3.AllocationAttribute{v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058e770), AttrSecondary:map[string]string{"namespace":"aws-service-broker", "node":"k8s-worker-23.example.com", "pod":"aws-servicebroker-fc58f95bb-bpb94"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058e820), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-9dd6685b9-cm5g2"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058e8a0), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-cainjector-8545fdf87c-5hczm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058e930), AttrSecondary:map[string]string{"namespace":"clair-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-85f59b5b87-xwqwb"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058e9b0), AttrSecondary:map[string]string{"namespace":"clair-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-866dd75bb9-q7jgm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058ea40), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"filebeat-syslog-5fd9947874-tb988"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058eac0), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-7c8c64fd89-bmkr4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058eb40), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-qk574"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058ebc0), AttrSecondary:map[string]string{"namespace":"lxos-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-558cf4654d-qhnzc"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058ec50), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"expire-check-worker-8nf9h"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058ece0), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"kube-state-metrics-597b7c5f5-kmgtv"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058ed60), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"tiller-deploy-786bc89dd8-h8lpg"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058ede0), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"default-http-backend-5f65599f44-mrdmz"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058ee50), AttrSecondary:map[string]string{"namespace":"snyk-monitor", "node":"k8s-worker-23.example.com", "pod":"snyk-monitor-6fc8c86d4c-w6rxl"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058eed0), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"prometheus-metrics-adapter-c74fd5b66-fk9p4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058ef50), AttrSecondary:map[string]string{"namespace":"monitoring", "node":"k8s-worker-23.example.com", "pod":"mon-prometheus-pushgateway-7987d7d4b6-jdjwr"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc00058f050), AttrSecondary:map[string]string{"node":"k8s-worker-23.example.com", "type":"ipipTunnelAddress"}}}, Deleted:false}}
2020-10-23 14:46:32.269 [DEBUG][1] ipam.go 1374: Successfully released IPs from block cidr=100.80.6.0/26 handle="k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b"
2020-10-23 14:46:32.269 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b)
2020-10-23 14:46:32.269 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.269 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b) Name="k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.272 [DEBUG][1] ipam.go 1458: Deleting handle: k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b
2020-10-23 14:46:32.272 [DEBUG][1] k8s.go 536: Performing 'DeleteKVP' for IPAMHandleKey(id=k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b)
2020-10-23 14:46:32.272 [DEBUG][1] customresource.go 109: Update custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b) Resource="IPAMHandles" Value=&v3.IPAMHandle{TypeMeta:v1.TypeMeta{Kind:"IPAMHandle", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"534575028", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMHandleSpec{HandleID:"k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b", Block:map[string]int{}}}
2020-10-23 14:46:32.272 [DEBUG][1] customresource.go 126: Update resource by name Key=IPAMHandle(k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b) Name="k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b" Resource="IPAMHandles" Value=&v3.IPAMHandle{TypeMeta:v1.TypeMeta{Kind:"IPAMHandle", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"534575028", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMHandleSpec{HandleID:"k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b", Block:map[string]int{}}}
2020-10-23 14:46:32.278 [DEBUG][1] customresource.go 162: Delete custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b) Resource="IPAMHandles"
2020-10-23 14:46:32.278 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b) Resource="IPAMHandles" Revision="535777043"
2020-10-23 14:46:32.278 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b) Name="k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b" Namespace="" Resource="IPAMHandles" Revision="535777043"
2020-10-23 14:46:32.280 [DEBUG][1] customresource.go 185: Send delete request by name Key=IPAMHandle(k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b) Name="k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b" Resource="IPAMHandles"
2020-10-23 14:46:32.290 [DEBUG][1] ipam.go 1481: Decremented handle 'k8s-pod-network.9bfb89ffe973b33a2b81ac1ddb4fb241859861ed23e984d074fd261656db7b2b' by 1
2020-10-23 14:46:32.290 [DEBUG][1] ipam.go 1625: Looking up node labels for host affinity cidr=100.80.6.0/26 host="k8s-worker-23.example.com"
2020-10-23 14:46:32.290 [DEBUG][1] k8s.go 564: Performing 'Get' for Node(k8s-worker-23.example.com)
2020-10-23 14:46:32.290 [DEBUG][1] node.go 114: Received Get request on Node type
2020-10-23 14:46:32.293 [INFO][1] ipam.go 1632: Node doesn't exist, no need to release affinity cidr=100.80.6.0/26 host="k8s-worker-23.example.com"
2020-10-23 14:46:32.293 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'ipip-tunnel-addr-k8s-worker-23.example.com'
2020-10-23 14:46:32.293 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=ipip-tunnel-addr-k8s-worker-23.example.com)
2020-10-23 14:46:32.293 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(ipip-tunnel-addr-k8s-worker-23.example.com) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.293 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(ipip-tunnel-addr-k8s-worker-23.example.com) Name="ipip-tunnel-addr-k8s-worker-23.example.com" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.295 [DEBUG][1] ipam.go 1317: Querying block so we can release IPs by handle cidr=100.80.6.0/26 handle="ipip-tunnel-addr-k8s-worker-23.example.com"
2020-10-23 14:46:32.295 [DEBUG][1] k8s.go 564: Performing 'Get' for BlockKey(cidr=100.80.6.0/26)
2020-10-23 14:46:32.295 [DEBUG][1] cidr.go 34: Converted IPNet to resource name IPNet="100.80.6.0/26" Name="100-80-6-0-26"
2020-10-23 14:46:32.295 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMBlock(100-80-6-0-26) Resource="IPAMBlocks" Revision=""
2020-10-23 14:46:32.295 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMBlock(100-80-6-0-26) Name="100-80-6-0-26" Namespace="" Resource="IPAMBlocks" Revision=""
2020-10-23 14:46:32.299 [DEBUG][1] ipam_block.go 349: Attribute indexes to release: [16]
2020-10-23 14:46:32.299 [DEBUG][1] ipam_block.go 300: 0 in [16]
2020-10-23 14:46:32.299 [DEBUG][1] ipam_block.go 300: 1 in [16]
2020-10-23 14:46:32.299 [DEBUG][1] ipam_block.go 300: 2 in [16]
2020-10-23 14:46:32.299 [DEBUG][1] ipam_block.go 300: 3 in [16]
2020-10-23 14:46:32.299 [DEBUG][1] ipam_block.go 300: 4 in [16]
2020-10-23 14:46:32.299 [DEBUG][1] ipam_block.go 300: 5 in [16]
2020-10-23 14:46:32.299 [DEBUG][1] ipam_block.go 300: 6 in [16]
2020-10-23 14:46:32.299 [DEBUG][1] ipam_block.go 300: 7 in [16]
2020-10-23 14:46:32.299 [DEBUG][1] ipam_block.go 300: 8 in [16]
2020-10-23 14:46:32.299 [DEBUG][1] ipam_block.go 300: 9 in [16]
2020-10-23 14:46:32.299 [DEBUG][1] ipam_block.go 300: 10 in [16]
2020-10-23 14:46:32.299 [DEBUG][1] ipam_block.go 300: 11 in [16]
2020-10-23 14:46:32.299 [DEBUG][1] ipam_block.go 300: 12 in [16]
2020-10-23 14:46:32.299 [DEBUG][1] ipam_block.go 300: 13 in [16]
2020-10-23 14:46:32.299 [DEBUG][1] ipam_block.go 300: 14 in [16]
2020-10-23 14:46:32.299 [DEBUG][1] ipam_block.go 300: 15 in [16]
2020-10-23 14:46:32.299 [DEBUG][1] ipam.go 1339: Block has 1 IPs with the given handle cidr=100.80.6.0/26 handle="ipip-tunnel-addr-k8s-worker-23.example.com"
2020-10-23 14:46:32.299 [DEBUG][1] ipam.go 1361: Updating block to release IPs cidr=100.80.6.0/26 handle="ipip-tunnel-addr-k8s-worker-23.example.com"
2020-10-23 14:46:32.299 [DEBUG][1] k8s.go 489: Performing 'Update' for &{Key:BlockKey(cidr=100.80.6.0/26) Value:0xc0007acb40 Revision:535777041 UID:0xc0000b2060 TTL:0s}
2020-10-23 14:46:32.299 [DEBUG][1] cidr.go 34: Converted IPNet to resource name IPNet="100.80.6.0/26" Name="100-80-6-0-26"
2020-10-23 14:46:32.299 [DEBUG][1] customresource.go 109: Update custom Kubernetes resource Key=IPAMBlock(100-80-6-0-26) Resource="IPAMBlocks" Value=&v3.IPAMBlock{TypeMeta:v1.TypeMeta{Kind:"IPAMBlock", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"100-80-6-0-26", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"535777041", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMBlockSpec{CIDR:"100.80.6.0/26", Affinity:(*string)(0xc000174650), StrictAffinity:false, Allocations:[]*int{(*int)(nil), (*int)(nil), (*int)(0xc000382948), (*int)(0xc000382998), (*int)(0xc0003829e8), (*int)(0xc000382a38), (*int)(0xc000382a88), (*int)(0xc000382ad8), (*int)(0xc000382b28), (*int)(0xc000382c68), (*int)(0xc000382b78), (*int)(0xc000382bc8), (*int)(0xc000382c18), (*int)(0xc000382cb8), (*int)(0xc000382d08), (*int)(0xc000382d58), (*int)(0xc000382da8), (*int)(0xc000382df8), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil)}, Unallocated:[]int{51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 0, 1, 22, 18, 23, 24, 25, 26, 27, 20, 21, 28, 29, 30, 31, 32, 19, 38, 42, 39, 41, 40, 43, 33, 34, 35, 36, 37, 44, 45, 46, 47, 48, 49, 50}, Attributes:[]v3.AllocationAttribute{v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174680), AttrSecondary:map[string]string{"namespace":"aws-service-broker", "node":"k8s-worker-23.example.com", "pod":"aws-servicebroker-fc58f95bb-bpb94"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174710), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-9dd6685b9-cm5g2"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174790), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-cainjector-8545fdf87c-5hczm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174810), AttrSecondary:map[string]string{"namespace":"clair-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-85f59b5b87-xwqwb"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174890), AttrSecondary:map[string]string{"namespace":"clair-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-866dd75bb9-q7jgm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174910), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"filebeat-syslog-5fd9947874-tb988"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174990), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-7c8c64fd89-bmkr4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174a00), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-qk574"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174a80), AttrSecondary:map[string]string{"namespace":"lxos-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-558cf4654d-qhnzc"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174b00), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"expire-check-worker-8nf9h"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174b80), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"kube-state-metrics-597b7c5f5-kmgtv"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174c00), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"tiller-deploy-786bc89dd8-h8lpg"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174c90), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"default-http-backend-5f65599f44-mrdmz"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174d10), AttrSecondary:map[string]string{"namespace":"snyk-monitor", "node":"k8s-worker-23.example.com", "pod":"snyk-monitor-6fc8c86d4c-w6rxl"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174d90), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"prometheus-metrics-adapter-c74fd5b66-fk9p4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174e10), AttrSecondary:map[string]string{"namespace":"monitoring", "node":"k8s-worker-23.example.com", "pod":"mon-prometheus-pushgateway-7987d7d4b6-jdjwr"}}}, Deleted:false}}
2020-10-23 14:46:32.299 [DEBUG][1] customresource.go 126: Update resource by name Key=IPAMBlock(100-80-6-0-26) Name="100-80-6-0-26" Resource="IPAMBlocks" Value=&v3.IPAMBlock{TypeMeta:v1.TypeMeta{Kind:"IPAMBlock", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"100-80-6-0-26", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"535777041", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMBlockSpec{CIDR:"100.80.6.0/26", Affinity:(*string)(0xc000174650), StrictAffinity:false, Allocations:[]*int{(*int)(nil), (*int)(nil), (*int)(0xc000382948), (*int)(0xc000382998), (*int)(0xc0003829e8), (*int)(0xc000382a38), (*int)(0xc000382a88), (*int)(0xc000382ad8), (*int)(0xc000382b28), (*int)(0xc000382c68), (*int)(0xc000382b78), (*int)(0xc000382bc8), (*int)(0xc000382c18), (*int)(0xc000382cb8), (*int)(0xc000382d08), (*int)(0xc000382d58), (*int)(0xc000382da8), (*int)(0xc000382df8), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil), (*int)(nil)}, Unallocated:[]int{51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 0, 1, 22, 18, 23, 24, 25, 26, 27, 20, 21, 28, 29, 30, 31, 32, 19, 38, 42, 39, 41, 40, 43, 33, 34, 35, 36, 37, 44, 45, 46, 47, 48, 49, 50}, Attributes:[]v3.AllocationAttribute{v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174680), AttrSecondary:map[string]string{"namespace":"aws-service-broker", "node":"k8s-worker-23.example.com", "pod":"aws-servicebroker-fc58f95bb-bpb94"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174710), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-9dd6685b9-cm5g2"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174790), AttrSecondary:map[string]string{"namespace":"cert-manager", "node":"k8s-worker-23.example.com", "pod":"cert-manager-cainjector-8545fdf87c-5hczm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174810), AttrSecondary:map[string]string{"namespace":"clair-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-85f59b5b87-xwqwb"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174890), AttrSecondary:map[string]string{"namespace":"clair-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-866dd75bb9-q7jgm"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174910), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"filebeat-syslog-5fd9947874-tb988"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174990), AttrSecondary:map[string]string{"namespace":"elastic-dev", "node":"k8s-worker-23.example.com", "pod":"nginx-ingress-controller-7c8c64fd89-bmkr4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174a00), AttrSecondary:map[string]string{"namespace":"velero", "node":"k8s-worker-23.example.com", "pod":"restic-qk574"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174a80), AttrSecondary:map[string]string{"namespace":"lxos-infra", "node":"k8s-worker-23.example.com", "pod":"thanos-querier-558cf4654d-qhnzc"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174b00), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"expire-check-worker-8nf9h"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174b80), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"kube-state-metrics-597b7c5f5-kmgtv"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174c00), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"tiller-deploy-786bc89dd8-h8lpg"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174c90), AttrSecondary:map[string]string{"namespace":"nuclio-infra", "node":"k8s-worker-23.example.com", "pod":"default-http-backend-5f65599f44-mrdmz"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174d10), AttrSecondary:map[string]string{"namespace":"snyk-monitor", "node":"k8s-worker-23.example.com", "pod":"snyk-monitor-6fc8c86d4c-w6rxl"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174d90), AttrSecondary:map[string]string{"namespace":"kube-system", "node":"k8s-worker-23.example.com", "pod":"prometheus-metrics-adapter-c74fd5b66-fk9p4"}}, v3.AllocationAttribute{AttrPrimary:(*string)(0xc000174e10), AttrSecondary:map[string]string{"namespace":"monitoring", "node":"k8s-worker-23.example.com", "pod":"mon-prometheus-pushgateway-7987d7d4b6-jdjwr"}}}, Deleted:false}}
2020-10-23 14:46:32.308 [DEBUG][1] ipam.go 1374: Successfully released IPs from block cidr=100.80.6.0/26 handle="ipip-tunnel-addr-k8s-worker-23.example.com"
2020-10-23 14:46:32.308 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=ipip-tunnel-addr-k8s-worker-23.example.com)
2020-10-23 14:46:32.308 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(ipip-tunnel-addr-k8s-worker-23.example.com) Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.308 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(ipip-tunnel-addr-k8s-worker-23.example.com) Name="ipip-tunnel-addr-k8s-worker-23.example.com" Namespace="" Resource="IPAMHandles" Revision=""
2020-10-23 14:46:32.310 [DEBUG][1] ipam.go 1458: Deleting handle: ipip-tunnel-addr-k8s-worker-23.example.com
2020-10-23 14:46:32.310 [DEBUG][1] k8s.go 536: Performing 'DeleteKVP' for IPAMHandleKey(id=ipip-tunnel-addr-k8s-worker-23.example.com)
2020-10-23 14:46:32.310 [DEBUG][1] customresource.go 109: Update custom Kubernetes resource Key=IPAMHandle(ipip-tunnel-addr-k8s-worker-23.example.com) Resource="IPAMHandles" Value=&v3.IPAMHandle{TypeMeta:v1.TypeMeta{Kind:"IPAMHandle", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"ipip-tunnel-addr-k8s-worker-23.example.com", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"534575643", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMHandleSpec{HandleID:"ipip-tunnel-addr-k8s-worker-23.example.com", Block:map[string]int{}}}
2020-10-23 14:46:32.310 [DEBUG][1] customresource.go 126: Update resource by name Key=IPAMHandle(ipip-tunnel-addr-k8s-worker-23.example.com) Name="ipip-tunnel-addr-k8s-worker-23.example.com" Resource="IPAMHandles" Value=&v3.IPAMHandle{TypeMeta:v1.TypeMeta{Kind:"IPAMHandle", APIVersion:"crd.projectcalico.org/v1"}, ObjectMeta:v1.ObjectMeta{Name:"ipip-tunnel-addr-k8s-worker-23.example.com", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"534575643", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.IPAMHandleSpec{HandleID:"ipip-tunnel-addr-k8s-worker-23.example.com", Block:map[string]int{}}}
2020-10-23 14:46:32.315 [DEBUG][1] customresource.go 162: Delete custom Kubernetes resource Key=IPAMHandle(ipip-tunnel-addr-k8s-worker-23.example.com) Resource="IPAMHandles"
2020-10-23 14:46:32.315 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(ipip-tunnel-addr-k8s-worker-23.example.com) Resource="IPAMHandles" Revision="535777048"
2020-10-23 14:46:32.315 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(ipip-tunnel-addr-k8s-worker-23.example.com) Name="ipip-tunnel-addr-k8s-worker-23.example.com" Namespace="" Resource="IPAMHandles" Revision="535777048"
2020-10-23 14:46:32.318 [DEBUG][1] customresource.go 185: Send delete request by name Key=IPAMHandle(ipip-tunnel-addr-k8s-worker-23.example.com) Name="ipip-tunnel-addr-k8s-worker-23.example.com" Resource="IPAMHandles"
2020-10-23 14:46:32.327 [DEBUG][1] ipam.go 1481: Decremented handle 'ipip-tunnel-addr-k8s-worker-23.example.com' by 1
2020-10-23 14:46:32.327 [DEBUG][1] ipam.go 1625: Looking up node labels for host affinity cidr=100.80.6.0/26 host="k8s-worker-23.example.com"
2020-10-23 14:46:32.327 [DEBUG][1] k8s.go 564: Performing 'Get' for Node(k8s-worker-23.example.com)
2020-10-23 14:46:32.327 [DEBUG][1] node.go 114: Received Get request on Node type
2020-10-23 14:46:32.332 [INFO][1] ipam.go 1632: Node doesn't exist, no need to release affinity cidr=100.80.6.0/26 host="k8s-worker-23.example.com"
2020-10-23 14:46:32.332 [INFO][1] ipam.go 217: Couldn't release all IPs for stale node, schedule retry calicoNode="k8s-worker-23.example.com"
2020-10-23 14:46:32.332 [WARNING][1] ipam.go 180: Error cleaning up node calicoNode="k8s-worker-23.example.com" error=Couldn't release all IPs k8sNode=""
2020-10-23 14:46:32.335 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-24.example.com" k8sNode="k8s-worker-24.example.com"
2020-10-23 14:46:32.411 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-25.example.com" k8sNode="k8s-worker-25.example.com"
2020-10-23 14:46:32.612 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-51.example.com" k8sNode="k8s-worker-51.example.com"
2020-10-23 14:46:32.812 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-22.example.com" k8sNode="k8s-worker-22.example.com"
2020-10-23 14:46:32.812 [INFO][1] ipam.go 45: Synchronizing IPAM data

[TheKangaroo]

@fasaxc, @caseydavenport this issue has an impact on our production environment and is pretty serious to us. Is there any additional information I can provide to help resolve this?

[caseydavenport]

@TheKangaroo I'm very sorry for the delay on this one. I've taken a look at the logs and this is my understanding of the issue:

2020-10-23 14:46:32.021 [INFO][1] ipam.go 1298: Releasing all IPs with handle 'k8s-pod-network.7ecd07e742bb804bdf8fb37d7820985a5a3b0fd3bb098fc0fb4bd315f2152a05
eth0'

Yeah, it looks very much like this is a result of "bad" data still in the IPAM allocations from the previous bug: https://github.com/projectcalico/cni-plugin/issues/821

There appears to be a mismatch in the Handle resource names - which seem to be properly formatted - and the attributes in the IPAM blocks - which seem to have the invalid format k8s-pod-network.<id>\r\neth0

The controller tries to release the allocation using the invalid handle, finds that no Handle object exists which matches the bad name, and hits this block: https://github.com/projectcalico/kube-controllers/blob/master/pkg/controllers/node/ipam.go#L208

When it comes around again to retry, the allocations are obviously still there since we didn't release anything, and it tries the whole thing over again. What I am not 100% on is how this is causing duplicate IP addresses to be assigned in the cluster, but I can imagine that it is related.

I think that a change like this would fix the endless retries that you are seeing by allowing the code to properly compare the correctly formatted handle IDs with the improperly formatted ones: https://github.com/projectcalico/libcalico-go/pull/1345

@fasaxc WDYT?

[caseydavenport]

I reproduced the cyclic behavior and failure to clean up the block with a test in PR: https://github.com/projectcalico/kube-controllers/pull/557

The fix I linked above makes that test pass.

[TheKangaroo]

Thats great news, I'll be happy to test if this resolves the duplicate IPs too :)

[TheKangaroo]

Quick questing @caseydavenport : is this https://github.com/projectcalico/libcalico-go/pull/1345 targeted for 3.18 or 3.17 milestone? Do you have an ETA for these release(s)?

[caseydavenport]

@TheKangaroo currently it's only targeted for v3.18, which is still a ways out.

v3.17 is currently in code freeze, but I can probably get this fix in as an exception. Let me see what I can do. If not, I think we could get it into a v3.17.1 patch release shortly after.

[caseydavenport]

Merged a cherry-pick to v3.17.0: https://github.com/projectcalico/libcalico-go/pull/1346

[TheKangaroo]

Thats great, thank you :)

[caseydavenport]

@TheKangaroo Calico v3.17 is out now! When you get a chance to try it I'd be very interested to see if this resolves the duplicate address issue you were seeing.

[TheKangaroo]

I've seen it yesterday and will update our testing environment the next days. I'll keep you updated about this issue :)

[TheKangaroo]

HI @caseydavenport, I upgraded the first cluster to 3.17.0, but unfortunately I still see the same errors in calico-kube-controller. I increased the loglevel to debug:

2020-12-08 17:19:40.779 [INFO][1] ipam.go 45: Synchronizing IPAM data
2020-12-08 17:19:40.784 [DEBUG][1] k8s.go 579: Performing 'List' for {IPVersion:0} model.BlockListOptions
2020-12-08 17:19:40.784 [DEBUG][1] customresource.go 242: List Custom K8s Resource ListInterface=IPAMBlock Resource="IPAMBlocks"
2020-12-08 17:19:41.005 [DEBUG][1] ipam.go 107: Calico nodes found in IPAM: map[k8s-master-01.example.com:[{0xc0007ea970 map[namespace:kube-system node:k8s-master-01.example.com pod:kubernetes-dashboard-84dfb859c8-7kg8v]} {0xc0007ea9f0 map[namespace:kube-system node:k8s-master-01.example.com pod:default-http-backend-74c858b7df-6bbpt]} {0xc0007eaa70 map[namespace:kube-system node:k8s-master-01.example.com pod:calico-kube-controllers-648f4868b8-9c6lv]} {0xc000849250 map[namespace:kube-system node:k8s-master-01.example.com pod:default-http-backend-74c858b7df-8cq4k]} {0xc000849bf0 map[namespace:kube-system node:k8s-master-01.example.com pod:kubernetes-dashboard-84dfb859c8-dffm5]} {0xc0008627a0 map[namespace:kube-system node:k8s-master-01.example.com pod:default-http-backend-74c858b7df-4d4z8]} {0xc0008625c0 map[namespace:kube-system node:k8s-master-01.example.com pod:expire-check-master-9mj2v]} {0xc000862640 map[node:k8s-master-01.example.com type:ipipTunnelAddress]} {0xc0008626a0 map[namespace:monitoring node:k8s-master-01.example.com pod:ephemeral-demo-645f5f7f95-4hzwz]} {0xc000862720 map[namespace:kube-system node:k8s-master-01.example.com pod:expire-check-master-6djzs]} {0xc000861840 map[node:k8s-master-01.example.com type:ipipTunnelAddress]}] k8s-master-02.example.com:[{0xc0005cad30 map[namespace:kube-system node:k8s-master-02.example.com pod:expire-check-master-7knhw]} {0xc0005cadb0 map[namespace:kube-system node:k8s-master-02.example.com pod:kubernetes-dashboard-84dfb859c8-9ch6q]} {0xc0005caea0 map[namespace:kube-system node:k8s-master-02.example.com pod:calico-kube-controllers-648f4868b8-xvhpv]} {0xc0005caf20 map[namespace:kube-system node:k8s-master-02.example.com pod:default-http-backend-74c858b7df-b2zl5]} {0xc000849cf0 map[namespace:kube-system node:k8s-master-02.example.com pod:expire-check-master-dd5gr]} {0xc000849c70 map[namespace:kube-system node:k8s-master-02.example.com pod:expire-check-master-m5fxf]} {0xc000861660 map[node:k8s-master-02.example.com type:ipipTunnelAddress]} {0xc0008616b0 map[namespace:kube-system node:k8s-master-02.example.com pod:expire-check-master-2dkl8]} {0xc000861720 map[namespace:kube-system node:k8s-master-02.example.com pod:calico-kube-controllers-646cf4cccf-w6wjf]}] k8s-worker-00.example.com:[{0xc000849330 map[namespace:velero node:k8s-worker-00.example.com pod:restic-gbbnc]} {0xc0008493b0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-00.example.com pod:rook-discover-wsgck]} {0xc000849430 map[namespace:kube-system node:k8s-worker-00.example.com pod:expire-check-worker-xhvqh]} {0xc000849530 map[namespace:kube-system node:k8s-worker-00.example.com pod:expire-check-worker-vv5zl]} {0xc0008494b0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-00.example.com pod:rook-ceph-sbx-primary-backup-w79hr]} {0xc0008495b0 map[namespace:kube-system node:k8s-worker-00.example.com pod:coredns-5889fdf6fc-2t7pw]} {0xc000849630 map[namespace:rook-ceph-sbx-primary node:k8s-worker-00.example.com pod:rook-ceph-sbx-primary-backup-m4lrg]} {0xc0008496b0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-00.example.com pod:rook-discover-cggmb]} {0xc000849730 map[namespace:rook-ceph-sbx-primary node:k8s-worker-00.example.com pod:rook-ceph-mon-a-667b6dc754-r22t9]} {0xc000862d10 map[namespace:kube-system node:k8s-worker-00.example.com pod:expire-check-worker-jwjhw]} {0xc000862d90 map[namespace:kube-system node:k8s-worker-00.example.com pod:expire-check-worker-678v7]} {0xc000862e00 map[namespace:velero node:k8s-worker-00.example.com pod:restic-6fvhx]} {0xc000862e80 map[namespace:rook-ceph-sbx-primary node:k8s-worker-00.example.com pod:rook-discover-hk6df]} {0xc000862f00 map[namespace:rook-ceph-sbx-primary node:k8s-worker-00.example.com pod:rook-ceph-mon-a-85748f9d94-g8jw8]} {0xc000862f80 map[namespace:rook-ceph-sbx-primary node:k8s-worker-00.example.com pod:rook-ceph-sbx-primary-backup-k4bfh]} {0xc000863000 map[node:k8s-worker-00.example.com type:ipipTunnelAddress]} {0xc000863060 map[namespace:kube-system node:k8s-worker-00.example.com pod:expire-check-worker-ds5b7]} {0xc0008630e0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-00.example.com pod:rook-discover-ng8k8]} {0xc000863160 map[namespace:velero node:k8s-worker-00.example.com pod:restic-6p6l6]} {0xc0008631e0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-00.example.com pod:rook-ceph-sbx-primary-backup-w2pjf]} {0xc000863260 map[namespace:rook-ceph-sbx-primary node:k8s-worker-00.example.com pod:rook-ceph-mon-a-85748f9d94-p7zxm]}] k8s-worker-20.example.com:[{0xc0007eaf90 map[namespace:rook-ceph-sbx-primary node:k8s-worker-20.example.com pod:rook-discover-vntp8]} {0xc0007eb1c0 map[namespace:velero node:k8s-worker-20.example.com pod:velero-sbx-776b9fd4f4-spqqz]} {0xc0007eb040 map[namespace:lxos-sbx node:k8s-worker-20.example.com pod:default-http-backend-6fd4978fb-j9r65]} {0xc0007eb0d0 map[namespace:kube-system node:k8s-worker-20.example.com pod:external-dns-top-zone-6bb48b5b77-wgncc]} {0xc0007eb140 map[namespace:kube-system node:k8s-worker-20.example.com pod:sealed-secrets-controller-65cf6f6fc7-m2hl7]} {0xc0007eab10 map[namespace:kube-system node:k8s-worker-20.example.com pod:external-dns-extern-top-zone-7cbccfcb8d-9976r]} {0xc0007eab90 map[namespace:rook-ceph-sbx-primary node:k8s-worker-20.example.com pod:rook-ceph-tools-74c899d786-c5pcz]} {0xc0007eac20 map[namespace:monitoring node:k8s-worker-20.example.com pod:mon-prometheus-kube-state-metrics-6cc667cd78-7s5c9]} {0xc0007eace0 map[namespace:kube-system node:k8s-worker-20.example.com pod:metrics-server-64d955cbc6-nnj9t]} {0xc0007ead80 map[namespace:monitoring node:k8s-worker-20.example.com pod:deploy-547f6c9d8b-zd5vq]} {0xc0007eae00 map[namespace:catalog node:k8s-worker-20.example.com pod:catalog-catalog-controller-manager-6866cd78b6-b9xq8]} {0xc0007eae80 map[namespace:monitoring node:k8s-worker-20.example.com pod:mon-redis-master-0]} {0xc0007eaf10 map[namespace:monitoring node:k8s-worker-20.example.com pod:mon-redis-ha-server-1]} {0xc0007eb240 map[namespace:rook-ceph-sbx-primary node:k8s-worker-20.example.com pod:rook-discover-cdfkk]} {0xc000833e20 map[namespace:rook-ceph-sbx-primary node:k8s-worker-20.example.com pod:rook-ceph-sbx-primary-backup-x5gbj]} {0xc000833e90 map[namespace:kube-system node:k8s-worker-20.example.com pod:expire-check-worker-hwmpp]} {0xc000833f00 map[namespace:monitoring node:k8s-worker-20.example.com pod:mon-redis-ha-server-1]} {0xc000848610 map[namespace:rook-ceph-sbx-primary node:k8s-worker-20.example.com pod:rook-ceph-sbx-primary-backup-m8g5p]} {0xc000833fe0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-20.example.com pod:rook-ceph-sbx-primary-backup-w7ggf]} {0xc000848050 map[namespace:kube-system node:k8s-worker-20.example.com pod:sealed-secrets-controller-65cf6f6fc7-t7bfc]} {0xc0008480c0 map[namespace:lxos-sbx node:k8s-worker-20.example.com pod:nginx-ingress-controller-855f55f857-5dn25]} {0xc000848130 map[namespace:kube-system node:k8s-worker-20.example.com pod:metrics-server-64d955cbc6-4679d]} {0xc000833f70 map[namespace:monitoring node:k8s-worker-20.example.com pod:mon-redis-ha-server-1]} {0xc0008481a0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-20.example.com pod:rook-ceph-operator-86dccf9fcc-5xm75]} {0xc000848220 map[namespace:metallb-system node:k8s-worker-20.example.com pod:metallb-controller-75cbdf4c77-zmqxh]} {0xc0008482a0 map[namespace:monitoring node:k8s-worker-20.example.com pod:mon-grafana-5b9ccf9bb-z9qws]} {0xc000848320 map[namespace:catalog node:k8s-worker-20.example.com pod:catalog-catalog-controller-manager-6866cd78b6-6dzfm]} {0xc000848390 map[namespace:monitoring node:k8s-worker-20.example.com pod:mon-redis-slave-0]} {0xc000848410 map[namespace:rook-ceph-sbx-primary node:k8s-worker-20.example.com pod:rook-ceph-tools-74c899d786-95htx]} {0xc000848490 map[namespace:velero node:k8s-worker-20.example.com pod:velero-sbx-776b9fd4f4-lkjmm]} {0xc000848510 map[namespace:monitoring node:k8s-worker-20.example.com pod:mon-redis-master-0]} {0xc000848590 map[namespace:monitoring node:k8s-worker-20.example.com pod:mon-redis-ha-server-0]} {0xc000862990 map[namespace:kube-system node:k8s-worker-20.example.com pod:expire-check-worker-zkhsp]} {0xc000862a10 map[namespace:velero node:k8s-worker-20.example.com pod:restic-bsrvq]} {0xc000862a90 map[namespace:cert-manager node:k8s-worker-20.example.com pod:cert-manager-5c47f46f57-2hvkf]} {0xc000862b10 map[namespace:monitoring node:k8s-worker-20.example.com pod:nginx-ingress-controller-74c88944d7-rn6kl]} {0xc000862ba0 map[namespace:monitoring node:k8s-worker-20.example.com pod:mon-prometheus-server-1]} {0xc000862c20 map[namespace:rook-ceph-sbx-primary node:k8s-worker-20.example.com pod:rook-discover-xcqg2]} {0xc000862910 map[namespace:rook-ceph-sbx-primary node:k8s-worker-20.example.com pod:rook-ceph-sbx-primary-backup-g9k2t]} {0xc000862890 map[namespace:kube-system node:k8s-worker-20.example.com pod:expire-check-worker-ccdcr]}] k8s-worker-21.example.com:[{0xc0005cac40 map[namespace:monitoring node:k8s-worker-21.example.com pod:mon-redis-ha-server-1]} {0xc000832310 map[namespace:velero node:k8s-worker-21.example.com pod:restic-lkczk]} {0xc000832380 map[namespace:kube-system node:k8s-worker-21.example.com pod:expire-check-worker-zqqbb]} {0xc000832400 map[namespace:rook-ceph-sbx-primary node:k8s-worker-21.example.com pod:rook-ceph-sbx-primary-backup-qfpjn]} {0xc000832480 map[namespace:rook-ceph-sbx-primary node:k8s-worker-21.example.com pod:rook-ceph-mgr-a-7dfbc5ffd5-kzfff]} {0xc000832500 map[namespace:kube-system node:k8s-worker-21.example.com pod:expire-check-worker-hk2sk]} {0xc000832580 map[namespace:rook-ceph-sbx-primary node:k8s-worker-21.example.com pod:rook-ceph-sbx-primary-backup-4x8tf]} {0xc000832980 map[namespace:kube-system node:k8s-worker-21.example.com pod:metrics-server-64d955cbc6-k7fm6]} {0xc000832a00 map[namespace:lxos-sbx node:k8s-worker-21.example.com pod:nginx-ingress-controller-855f55f857-zvsp2]} {0xc000832600 map[namespace:monitoring node:k8s-worker-21.example.com pod:mon-redis-ha-server-1]} {0xc000832680 map[namespace:monitoring node:k8s-worker-21.example.com pod:default-http-backend-6fd4978fb-2pcrk]} {0xc000832700 map[namespace:kube-system node:k8s-worker-21.example.com pod:external-dns-5d6686c558-8jqtr]} {0xc000832780 map[namespace:monitoring node:k8s-worker-21.example.com pod:mon-prometheus-kube-state-metrics-6cc667cd78-vv9f8]} {0xc000832800 map[namespace:lxos-sbx node:k8s-worker-21.example.com pod:default-http-backend-6fd4978fb-dq9xv]} {0xc000832880 map[namespace:catalog node:k8s-worker-21.example.com pod:catalog-catalog-apiserver-7764cdb499-z6njh]} {0xc000832900 map[namespace:monitoring node:k8s-worker-21.example.com pod:mon-prometheus-server-1]} {0xc000832a80 map[namespace:monitoring node:k8s-worker-21.example.com pod:mon-redis-ha-server-0]} {0xc000832b00 map[namespace:monitoring node:k8s-worker-21.example.com pod:mon-redis-slave-0]} {0xc000832b80 map[namespace:rook-ceph-sbx-primary node:k8s-worker-21.example.com pod:rook-ceph-mgr-a-7dfbc5ffd5-cvdgt]} {0xc000832c00 map[namespace:monitoring node:k8s-worker-21.example.com pod:mon-prometheus-server-1]} {0xc000832c80 map[namespace:monitoring node:k8s-worker-21.example.com pod:mon-prometheus-server-1]} {0xc000848690 map[namespace:aws-service-broker node:k8s-worker-21.example.com pod:aws-servicebroker-55dc94bb7-xx4gw]} {0xc000848710 map[namespace:monitoring node:k8s-worker-21.example.com pod:mon-redis-slave-0]} {0xc000848790 map[namespace:rook-ceph-sbx-primary node:k8s-worker-21.example.com pod:rook-ceph-mgr-a-7dfbc5ffd5-pnt78]} {0xc000848810 map[namespace:rook-ceph-sbx-primary node:k8s-worker-21.example.com pod:rook-ceph-mgr-a-7dfbc5ffd5-jwjdc]} {0xc000848890 map[namespace:rook-ceph-sbx-primary node:k8s-worker-21.example.com pod:rook-ceph-mgr-a-7dfbc5ffd5-k5br8]} {0xc000848910 map[namespace:rook-ceph-sbx-primary node:k8s-worker-21.example.com pod:rook-ceph-sbx-primary-backup-j4f9b]} {0xc000848990 map[namespace:rook-ceph-sbx-primary node:k8s-worker-21.example.com pod:rook-discover-p4fwp]} {0xc000848a10 map[namespace:rook-ceph-sbx-primary node:k8s-worker-21.example.com pod:rook-ceph-tools-6b94f5f84b-f8rll]} {0xc000848a90 map[namespace:rook-ceph-sbx-primary node:k8s-worker-21.example.com pod:rook-ceph-mgr-a-dd5b9bb54-t2lkz]} {0xc000849e50 map[namespace:kube-system node:k8s-worker-21.example.com pod:k8s-toolbox-6bbdb4cb86-q9n5z]} {0xc000849ed0 map[namespace:kube-system node:k8s-worker-21.example.com pod:nginx-ingress-controller-7d7f565fb-6djw9]} {0xc000849f50 map[namespace:kube-system node:k8s-worker-21.example.com pod:prometheus-operator-58f5f5789d-98bk7]} {0xc000849fd0 map[namespace:kube-system node:k8s-worker-21.example.com pod:sealed-secrets-75756cb8b8-ztsv4]} {0xc000849dd0 map[namespace:kube-system node:k8s-worker-21.example.com pod:external-dns-extern-top-zone-7cbccfcb8d-shrp6]} {0xc000860dd0 map[namespace:kube-system node:k8s-worker-21.example.com pod:node-labeler-786896fb46-zgrd5]} {0xc000860e40 map[namespace:velero node:k8s-worker-21.example.com pod:restic-5bch9]} {0xc000860eb0 map[namespace:kube-system node:k8s-worker-21.example.com pod:expire-check-worker-qpwdb]} {0xc000861000 map[namespace:kube-system node:k8s-worker-21.example.com pod:expire-check-worker-k62xm]} {0xc000861070 map[namespace:velero node:k8s-worker-21.example.com pod:restic-n9hl2]} {0xc000861150 map[node:k8s-worker-21.example.com type:ipipTunnelAddress]} {0xc0008610e0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-21.example.com pod:rook-discover-5smnv]} {0xc000860f20 map[namespace:rook-ceph-sbx-primary node:k8s-worker-21.example.com pod:rook-ceph-mgr-a-7c5c47cff8-dw6qn]} {0xc000860f90 map[namespace:rook-ceph-sbx-primary node:k8s-worker-21.example.com pod:rook-ceph-sbx-primary-backup-wdkls]} {0xc000861360 map[namespace:velero node:k8s-worker-21.example.com pod:restic-zgvqg]} {0xc000861590 map[namespace:monitoring node:k8s-worker-21.example.com pod:mon-prometheus-server-0]} {0xc0008613d0 map[namespace:monitoring node:k8s-worker-21.example.com pod:nginx-ingress-controller-74c88944d7-d266k]} {0xc000861520 map[namespace:opsview node:k8s-worker-21.example.com pod:kube-ops-view-6d99f87db9-2p92v]} {0xc000861440 map[namespace:rook-ceph-sbx-primary node:k8s-worker-21.example.com pod:rook-ceph-tools-57644c8849-9gwbs]} {0xc0008614b0 map[namespace:monitoring node:k8s-worker-21.example.com pod:default-http-backend-6fd4978fb-fv8dd]} {0xc000861280 map[namespace:kube-system node:k8s-worker-21.example.com pod:expire-check-worker-77zjx]} {0xc000861210 map[namespace:rook-ceph-sbx-primary node:k8s-worker-21.example.com pod:rook-discover-rznxz]} {0xc0008611a0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-21.example.com pod:rook-ceph-sbx-primary-backup-tthtq]} {0xc0008612f0 map[namespace:kube-system node:k8s-worker-21.example.com pod:coredns-6fbcb44b7f-8vdr4]}] k8s-worker-22.example.com:[{0xc0005cb290 map[namespace:kube-system node:k8s-worker-22.example.com pod:expire-check-worker-cq5dx]} {0xc0005cb310 map[namespace:rook-ceph-sbx-primary node:k8s-worker-22.example.com pod:rook-discover-ghmrx]} {0xc0005cb390 map[namespace:kube-system node:k8s-worker-22.example.com pod:node-labeler-6f9dcc7c49-zscxk]} {0xc0005cb410 map[namespace:kube-system node:k8s-worker-22.example.com pod:calico-kube-controllers-648f4868b8-k74v2]} {0xc0005cb490 map[namespace:aws-service-broker node:k8s-worker-22.example.com pod:aws-servicebroker-55dc94bb7-4t88n]} {0xc0005cb510 map[namespace:metallb-system node:k8s-worker-22.example.com pod:metallb-controller-75cbdf4c77-dxcfk]} {0xc0005cafa0 map[namespace:catalog node:k8s-worker-22.example.com pod:catalog-catalog-apiserver-7764cdb499-drdnr]} {0xc0005cb020 map[namespace:kube-system node:k8s-worker-22.example.com pod:kube-ops-view-58ddfb8d4f-7c8v7]} {0xc0005cb0a0 map[namespace:velero node:k8s-worker-22.example.com pod:velero-sbx-776b9fd4f4-pdtnq]} {0xc0005cb120 map[namespace:monitoring node:k8s-worker-22.example.com pod:mon-prometheus-pushgateway-779464cc74-8jq8b]} {0xc0005cb1a0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-22.example.com pod:testmount-698dc786cc-7wxb8]} {0xc0005cb220 map[namespace:monitoring node:k8s-worker-22.example.com pod:mon-redis-slave-0]} {0xc0005cb590 map[namespace:monitoring node:k8s-worker-22.example.com pod:mon-redis-slave-0]} {0xc0005cb620 map[namespace:monitoring node:k8s-worker-22.example.com pod:mon-redis-master-0]} {0xc0005cb6a0 map[namespace:metallb-system node:k8s-worker-22.example.com pod:metallb-controller-7cc79d4b4b-gpmrp]} {0xc0005cb720 map[namespace:rook-ceph-sbx-primary node:k8s-worker-22.example.com pod:rook-ceph-mgr-a-dd5b9bb54-vqnt2]} {0xc000848b70 map[namespace:monitoring node:k8s-worker-22.example.com pod:mon-redis-master-0]} {0xc000848f60 map[namespace:kube-system node:k8s-worker-22.example.com pod:expire-check-worker-7p44x]} {0xc000848bf0 map[namespace:kube-system node:k8s-worker-22.example.com pod:expire-check-worker-x2k86]} {0xc000848c70 map[namespace:rook-ceph-sbx-primary node:k8s-worker-22.example.com pod:rook-ceph-sbx-primary-backup-gdxhs]} {0xc000848e60 map[namespace:monitoring node:k8s-worker-22.example.com pod:mon-prometheus-server-1]} {0xc000848ee0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-22.example.com pod:rook-ceph-sbx-primary-backup-6ss7v]} {0xc000848cf0 map[namespace:aws-service-broker node:k8s-worker-22.example.com pod:aws-servicebroker-55dc94bb7-6pdrx]} {0xc000849160 map[namespace:monitoring node:k8s-worker-22.example.com pod:mon-prometheus-pushgateway-779464cc74-r7f5c]} {0xc000848fe0 map[namespace:kube-system node:k8s-worker-22.example.com pod:external-dns-extern-top-zone-7cbccfcb8d-p8p4q]} {0xc000849060 map[namespace:aws-service-broker node:k8s-worker-22.example.com pod:aws-servicebroker-55dc94bb7-m2nnf]} {0xc0008490e0 map[namespace:monitoring node:k8s-worker-22.example.com pod:mon-redis-master-0]} {0xc000848d70 map[namespace:velero node:k8s-worker-22.example.com pod:restic-szplm]} {0xc000848df0 map[namespace:kube-system node:k8s-worker-22.example.com pod:expire-check-worker-flw5p]} {0xc0008605c0 map[namespace:kube-system node:k8s-worker-22.example.com pod:expire-check-worker-dssb8]} {0xc000860260 map[namespace:kube-system node:k8s-worker-22.example.com pod:expire-check-worker-ftlhd]} {0xc000860a10 map[node:k8s-worker-22.example.com type:ipipTunnelAddress]} {0xc000860850 map[namespace:velero node:k8s-worker-22.example.com pod:restic-mkbff]} {0xc0008608c0 map[namespace:cert-manager node:k8s-worker-22.example.com pod:cert-manager-webhook-547567b88f-dvzz5]} {0xc000860930 map[namespace:kube-system node:k8s-worker-22.example.com pod:coredns-6fbcb44b7f-vlffn]} {0xc0008609a0 map[namespace:kube-system node:k8s-worker-22.example.com pod:kubernetes-dashboard-c584d7cc-brnmb]} {0xc0008602d0 map[namespace:kube-system node:k8s-worker-22.example.com pod:metrics-server-64d955cbc6-s5jdr]} {0xc000860340 map[namespace:rook-ceph-sbx-primary node:k8s-worker-22.example.com pod:rook-discover-klrbt]} {0xc0008603c0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-22.example.com pod:rook-ceph-tools-57644c8849-kjgwd]} {0xc000860bb0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-22.example.com pod:rook-discover-4h7tx]} {0xc000860440 map[namespace:rook-ceph-sbx-primary node:k8s-worker-22.example.com pod:testmount-7757b9dc57-7x8v2]} {0xc0008604c0 map[namespace:monitoring node:k8s-worker-22.example.com pod:mon-grafana-5b9ccf9bb-zcxmh]} {0xc000860540 map[namespace:rook-ceph-sbx-primary node:k8s-worker-22.example.com pod:rook-ceph-sbx-primary-backup-46tql]} {0xc000860b40 map[namespace:velero node:k8s-worker-22.example.com pod:restic-nmqqd]} {0xc000860c20 map[namespace:prometheus node:k8s-worker-22.example.com pod:master-kube-prometheus-sta-operator-7945d4fbb-s5ng2]} {0xc000860d00 map[namespace:rook-ceph-sbx-primary node:k8s-worker-22.example.com pod:rook-ceph-mgr-a-7c5c47cff8-bddcv]} {0xc000860c90 map[namespace:catalog node:k8s-worker-22.example.com pod:catalog-catalog-webhook-5d575f58cf-prqtc]} {0xc000860640 map[namespace:catalog node:k8s-worker-22.example.com pod:catalog-catalog-webhook-c9c7f9c6-l4r65]} {0xc0008606d0 map[namespace:catalog node:k8s-worker-22.example.com pod:catalog-catalog-controller-manager-75d98fbc4d-zd69p]} {0xc000860750 map[namespace:aws-service-broker node:k8s-worker-22.example.com pod:aws-servicebroker-b754458b7-lwt2h]} {0xc0008607d0 map[namespace:catalog node:k8s-worker-22.example.com pod:catalog-catalog-apiserver-946d9cf64-hmv2k]} {0xc000860a60 map[namespace:rook-ceph-sbx-primary node:k8s-worker-22.example.com pod:rook-ceph-sbx-primary-backup-99gv9]} {0xc000860ad0 map[namespace:kube-system node:k8s-worker-22.example.com pod:expire-check-worker-69qxn]}] k8s-worker-23.example.com:[{0xc0005cb810 map[namespace:velero node:k8s-worker-23.example.com pod:restic-77hrv]} {0xc0005cb890 map[namespace:rook-ceph-sbx-primary node:k8s-worker-23.example.com pod:rook-discover-xzkvg]} {0xc0005cb910 map[namespace:monitoring node:k8s-worker-23.example.com pod:mon-redis-slave-0]} {0xc0005cb990 map[namespace:kube-system node:k8s-worker-23.example.com pod:expire-check-worker-fhmcc]} {0xc0005cba10 map[namespace:rook-ceph-sbx-primary node:k8s-worker-23.example.com pod:rook-ceph-sbx-primary-backup-l9grt]} {0xc0005cba90 map[namespace:monitoring node:k8s-worker-23.example.com pod:mon-prometheus-server-1]} {0xc0005cbca0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-23.example.com pod:rook-discover-v8458]} {0xc0005cbd20 map[namespace:rook-ceph-sbx-primary node:k8s-worker-23.example.com pod:rook-ceph-sbx-primary-backup-75vg7]} {0xc0005cbb10 map[namespace:monitoring node:k8s-worker-23.example.com pod:mon-redis-slave-1]} {0xc0005cbb90 map[namespace:rook-ceph-sbx-primary node:k8s-worker-23.example.com pod:testmount-698dc786cc-w8b8j]} {0xc0005cbc10 map[namespace:rook-ceph-sbx-primary node:k8s-worker-23.example.com pod:rook-ceph-mgr-a-7dfbc5ffd5-xf5kw]} {0xc0005cbda0 map[namespace:kube-system node:k8s-worker-23.example.com pod:external-dns-5d6686c558-xgdqs]} {0xc0005cbe20 map[namespace:kube-system node:k8s-worker-23.example.com pod:sealed-secrets-controller-65cf6f6fc7-zs8sl]} {0xc0005cbea0 map[namespace:catalog node:k8s-worker-23.example.com pod:catalog-catalog-apiserver-7764cdb499-sfw9x]} {0xc0005cbf20 map[namespace:monitoring node:k8s-worker-23.example.com pod:mon-redis-ha-server-1]} {0xc0005cbfa0 map[namespace:monitoring node:k8s-worker-23.example.com pod:mon-grafana-5b9ccf9bb-8f24c]} {0xc000832020 map[namespace:monitoring node:k8s-worker-23.example.com pod:mon-redis-master-0]} {0xc0008320a0 map[namespace:monitoring node:k8s-worker-23.example.com pod:mon-redis-slave-0]} {0xc000832120 map[namespace:monitoring node:k8s-worker-23.example.com pod:mon-redis-ha-server-1]} {0xc0008321a0 map[namespace:velero node:k8s-worker-23.example.com pod:velero-sbx-776b9fd4f4-dr57t]} {0xc000832220 map[namespace:monitoring node:k8s-worker-23.example.com pod:mon-redis-master-0]} {0xc0008623d0 map[namespace:velero node:k8s-worker-23.example.com pod:restic-75fqz]} {0xc000862450 map[namespace:rook-ceph-sbx-primary node:k8s-worker-23.example.com pod:rook-discover-6mclj]} {0xc0008624d0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-23.example.com pod:rook-ceph-sbx-primary-backup-k98bj]} {0xc000862050 map[namespace:elastic-sbx node:k8s-worker-23.example.com pod:nginx-ingress-controller-76bff6c8b8-v5lnp]} {0xc0008620d0 map[namespace:elastic-sbx node:k8s-worker-23.example.com pod:nginx-ingress-controller-76bff6c8b8-crb7s]} {0xc000862150 map[namespace:monitoring node:k8s-worker-23.example.com pod:deploy-547f6c9d8b-96rms]} {0xc0008621d0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-23.example.com pod:rook-ceph-operator-6c656d4995-z6f48]} {0xc000862250 map[namespace:kube-system node:k8s-worker-23.example.com pod:nginx-ingress-controller-6f8f6bd84b-wvqhc]} {0xc0008622d0 map[namespace:monitoring node:k8s-worker-23.example.com pod:mon-redis-master-0]} {0xc000862350 map[namespace:kube-system node:k8s-worker-23.example.com pod:kube-ops-view-7cd88b99b7-x42z8]} {0xc000861c30 map[namespace:kube-system node:k8s-worker-23.example.com pod:expire-check-worker-7xkv8]} {0xc000861ca0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-23.example.com pod:rook-discover-flpvf]} {0xc0008619b0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-23.example.com pod:rook-ceph-sbx-primary-backup-vxfpl]} {0xc000861940 map[namespace:rook-ceph-sbx-primary node:k8s-worker-23.example.com pod:rook-discover-jzdtx]} {0xc000861d10 map[namespace:aws-service-broker node:k8s-worker-23.example.com pod:aws-service-broker-aws-servicebroker-5b8765c8f7-98m7w]} {0xc000861d80 map[namespace:monitoring node:k8s-worker-23.example.com pod:mon-prometheus-server-1]} {0xc000861a90 map[namespace:velero node:k8s-worker-23.example.com pod:restic-6wcfs]} {0xc000861b00 map[namespace:rook-ceph-sbx-primary node:k8s-worker-23.example.com pod:rook-ceph-sbx-primary-backup-gmrvr]} {0xc000861b70 map[namespace:velero node:k8s-worker-23.example.com pod:restic-2g482]} {0xc000861be0 map[node:k8s-worker-23.example.com type:ipipTunnelAddress]} {0xc000861a20 map[namespace:kube-system node:k8s-worker-23.example.com pod:expire-check-worker-gkjmm]}] k8s-worker-50.example.com:[{0xc000832d70 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-discover-j87nh]} {0xc000832df0 map[namespace:velero node:k8s-worker-50.example.com pod:restic-4kthq]} {0xc000832e70 map[namespace:kube-system node:k8s-worker-50.example.com pod:expire-check-worker-8fmnw]} {0xc000832ef0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-sbx-primary-backup-2vr8w]} {0xc000832f70 map[namespace:elastic-sbx node:k8s-worker-50.example.com pod:default-http-backend-6fd4978fb-p9qlt]} {0xc000832ff0 map[namespace:elastic-sbx node:k8s-worker-50.example.com pod:nginx-ingress-controller-77bdb89dcc-xrzvn]} {0xc000833070 map[namespace:monitoring node:k8s-worker-50.example.com pod:nginx-ingress-controller-74c88944d7-gkzvv]} {0xc0008330f0 map[namespace:kube-system node:k8s-worker-50.example.com pod:prometheus-metrics-adapter-ddd7cfc7b-xxb87]} {0xc000833160 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-30-7966f694d7-89tl7]} {0xc0008331e0 map[namespace:kube-system node:k8s-worker-50.example.com pod:expire-check-worker-8nxx8]} {0xc000833260 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-31-697fbb75b8-jhrrs]} {0xc0008332e0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-sbx-primary-backup-vbr8w]} {0xc000833360 map[namespace:velero node:k8s-worker-50.example.com pod:restic-rldcf]} {0xc0008333e0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-mon-c-b798db7db-2jsj6]} {0xc000833460 map[namespace:kube-system node:k8s-worker-50.example.com pod:nginx-ingress-controller-85bdc89bb7-jvq6g]} {0xc0008334e0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-32-5887bb46b-g47xx]} {0xc000833560 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-34-6968b94d79-wprjs]} {0xc0008335e0 map[namespace:kube-system node:k8s-worker-50.example.com pod:prometheus-operator-58f5f5789d-8p92h]} {0xc000833660 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-discover-hd7lg]} {0xc0008336e0 map[namespace:lxos-sbx node:k8s-worker-50.example.com pod:nginx-ingress-controller-855f55f857-29xk4]} {0xc000833760 map[namespace:kube-system node:k8s-worker-50.example.com pod:coredns-5889fdf6fc-wj8hd]} {0xc0008337e0 map[namespace:kube-system node:k8s-worker-50.example.com pod:coredns-5889fdf6fc-s9sfd]} {0xc000833850 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-sbx-primary-backup-pj77q]} {0xc0008338d0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-discover-hpqwh]} {0xc000833940 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-mon-b-76d49f5f6f-j6jbx]} {0xc0008339c0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-21-78fcb8ff97-66l75]} {0xc000833a40 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-24-866646d5d8-csp46]} {0xc000833ac0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-25-66c757588f-pw44w]} {0xc000833b40 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-5-6984d5dc67-872js]} {0xc000833bc0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-9-576db48b79-jsrb5]} {0xc000833c40 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-13-7c645bfb6-bqvvj]} {0xc000833cc0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-1-6bc4f84785-wjlrp]} {0xc000833d40 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-19-6bc8cc584-gj4nv]} {0xc000863350 map[namespace:kube-system node:k8s-worker-50.example.com pod:expire-check-worker-lnszs]} {0xc0008633d0 map[namespace:velero node:k8s-worker-50.example.com pod:restic-j6znd]} {0xc000863450 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-discover-q5vkh]} {0xc000863fd0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-tools-57644c8849-sgl44]} {0xc000863dd0 map[namespace:kube-system node:k8s-worker-50.example.com pod:external-dns-extern-78cdb9fbc6-jnbxj]} {0xc000863e50 map[namespace:kube-system node:k8s-worker-50.example.com pod:prometheus-operator-58f5f5789d-r5q25]} {0xc000863ed0 map[namespace:monitoring node:k8s-worker-50.example.com pod:mon-grafana-5b9ccf9bb-fhr6j]} {0xc0008634d0 map[namespace:metallb-system node:k8s-worker-50.example.com pod:metallb-controller-7cc79d4b4b-4mqfd]} {0xc000863550 map[namespace:kube-system node:k8s-worker-50.example.com pod:external-dns-extern-eai-677c4d6f7b-6pktm]} {0xc0008635d0 map[namespace:lxos-sbx node:k8s-worker-50.example.com pod:nginx-ingress-controller-855f55f857-4gw97]} {0xc000863650 map[namespace:velero node:k8s-worker-50.example.com pod:restic-7gxxw]} {0xc0008636d0 map[namespace:kube-system node:k8s-worker-50.example.com pod:expire-check-worker-szkst]} {0xc000863f50 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:testmount-7757b9dc57-g8sz4]} {0xc000863750 map[namespace:monitoring node:k8s-worker-50.example.com pod:mon-redis-ha-server-1]} {0xc0008637d0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-operator-6c656d4995-j29wr]} {0xc000863850 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-discover-s8cfc]} {0xc0008638d0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-mon-b-7b994ff4dc-nlxrs]} {0xc0007f4820 map[namespace:monitoring node:k8s-worker-50.example.com pod:mon-redis-ha-server-1]} {0xc000863950 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-1-5fb7d4db4c-lnvp5]} {0xc0008639d0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-13-76c5d7df9b-hrxmb]} {0xc000863a50 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-17-85b9dfddd4-6bx6d]} {0xc000863ad0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-24-84bc56d9d-86zls]} {0xc000863b50 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-20-6bd674f8d6-7szfp]} {0xc000863bd0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-28-55c77cc8f4-5vw75]} {0xc000863c50 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-5-7655865f69-tf9qr]} {0xc000863cd0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-9-7c8cd46f97-87czb]} {0xc0007f47a0 map[namespace:monitoring node:k8s-worker-50.example.com pod:mon-redis-slave-1]} {0xc000863d50 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-sbx-primary-backup-z2vfx]} {0xc0007f4050 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-24-84bc56d9d-k2cmr]} {0xc0007f40d0 map[node:k8s-worker-50.example.com type:ipipTunnelAddress]} {0xc0007f4120 map[namespace:velero node:k8s-worker-50.example.com pod:restic-vhwdt]} {0xc0007f41a0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-20-6bd674f8d6-6qdkr]} {0xc0007f4220 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-9-7c8cd46f97-g5xnp]} {0xc0007f42b0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-mon-b-7b994ff4dc-hxwmb]} {0xc0007f4320 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-sbx-primary-backup-p9kwd]} {0xc0007f43a0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-discover-2rxmm]} {0xc0007f4420 map[namespace:kube-system node:k8s-worker-50.example.com pod:expire-check-worker-hxxkl]} {0xc0007f44a0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-17-85b9dfddd4-bx9wn]} {0xc0007f4520 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-5-7655865f69-jrzz8]} {0xc0007f45a0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-1-5fb7d4db4c-jvcdg]} {0xc0007f4620 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-28-55c77cc8f4-sk2nk]} {0xc0007f46a0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-osd-13-76c5d7df9b-n725x]} {0xc0007f4720 map[namespace:rook-ceph-sbx-primary node:k8s-worker-50.example.com pod:rook-ceph-operator-6c656d4995-z5fsj]}] k8s-worker-51.example.com:[{0xc0007eb330 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-discover-7pvns]} {0xc0007eb3b0 map[namespace:velero node:k8s-worker-51.example.com pod:restic-7hrwb]} {0xc0007eb430 map[namespace:kube-system node:k8s-worker-51.example.com pod:expire-check-worker-4pbfh]} {0xc0007eb4b0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-sbx-primary-backup-95vrl]} {0xc0007eb530 map[namespace:elastic-sbx node:k8s-worker-51.example.com pod:nginx-ingress-controller-77bdb89dcc-tdxjx]} {0xc0007eba60 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-23-5746f7f978-m2jpn]} {0xc0007ebae0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-22-df9d56954-g2wnp]} {0xc0007ebb60 map[namespace:kube-system node:k8s-worker-51.example.com pod:nginx-ingress-controller-85bdc89bb7-rm76j]} {0xc0007eb5c0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-mon-b-6fcdb55fb-bf6ts]} {0xc0007eb640 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-27-67dc99b5d6-jvzb4]} {0xc0007eb6c0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-discover-bqfvt]} {0xc0007eb740 map[namespace:velero node:k8s-worker-51.example.com pod:restic-q8ptb]} {0xc0007eb7e0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-26-6f7c868bf9-p95cw]} {0xc0007eb860 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-25-556b7469b7-znslp]} {0xc0007eb8e0 map[namespace:monitoring node:k8s-worker-51.example.com pod:nginx-ingress-controller-74c88944d7-wwss4]} {0xc0007eb960 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-sbx-primary-backup-mrtsh]} {0xc0007eb9e0 map[namespace:kube-system node:k8s-worker-51.example.com pod:expire-check-worker-2kvhk]} {0xc0007ebc50 map[namespace:kube-system node:k8s-worker-51.example.com pod:external-dns-5d6686c558-qnzxg]} {0xc0007ebd00 map[namespace:kube-system node:k8s-worker-51.example.com pod:external-dns-extern-78cdb9fbc6-ggzrv]} {0xc0007ebbe0 map[namespace:monitoring node:k8s-worker-51.example.com pod:default-http-backend-6fd4978fb-xzbpj]} {0xc0007ebd80 map[namespace:monitoring node:k8s-worker-51.example.com pod:mon-grafana-5b9ccf9bb-964qk]} {0xc0007ebe50 map[namespace:lxos-sbx node:k8s-worker-51.example.com pod:default-http-backend-6fd4978fb-hnngf]} {0xc0007ebef0 map[namespace:kube-system node:k8s-worker-51.example.com pod:kube-ops-view-7cd88b99b7-hp6gz]} {0xc0007ebfc0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-sbx-primary-backup-gjhvg]} {0xc000840100 map[namespace:kube-system node:k8s-worker-51.example.com pod:prometheus-metrics-adapter-c74fd5b66-9wl5x]} {0xc000813320 map[namespace:kube-system node:k8s-worker-51.example.com pod:expire-check-worker-66k6z]} {0xc0008124c0 map[namespace:velero node:k8s-worker-51.example.com pod:restic-g2dgv]} {0xc000812540 map[namespace:kube-system node:k8s-worker-51.example.com pod:expire-check-worker-rr62q]} {0xc0008125c0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-discover-245ds]} {0xc000840200 map[namespace:metallb-system node:k8s-worker-51.example.com pod:metallb-controller-7cc79d4b4b-6mnpb]} {0xc000812f20 map[namespace:monitoring node:k8s-worker-51.example.com pod:default-http-backend-6fd4978fb-mlnqn]} {0xc000812640 map[namespace:monitoring node:k8s-worker-51.example.com pod:mon-redis-ha-server-2]} {0xc000812fa0 map[namespace:kube-system node:k8s-worker-51.example.com pod:coredns-6fbcb44b7f-7prvh]} {0xc000813020 map[namespace:monitoring node:k8s-worker-51.example.com pod:mon-prometheus-alertmanager-59b857f46-fn5pl]} {0xc000813e80 map[namespace:cert-manager node:k8s-worker-51.example.com pod:certmanager-cert-manager-webhook-7959675d59-6grvs]} {0xc0008126c0 map[namespace:kube-system node:k8s-worker-51.example.com pod:node-labeler-786896fb46-hncvf]} {0xc000812740 map[namespace:elastic-sbx node:k8s-worker-51.example.com pod:default-http-backend-6fd4978fb-s5q6f]} {0xc0008127c0 map[namespace:kube-system node:k8s-worker-51.example.com pod:prometheus-metrics-adapter-ddd7cfc7b-2sgrb]} {0xc000812840 map[namespace:monitoring node:k8s-worker-51.example.com pod:mon-redis-master-0]} {0xc0008128c0 map[namespace:kube-system node:k8s-worker-51.example.com pod:kube-ops-view-7cd88b99b7-bngfb]} {0xc0008130a0 map[namespace:kube-system node:k8s-worker-51.example.com pod:metrics-server-64d955cbc6-hk2fz]} {0xc000813120 map[namespace:catalog node:k8s-worker-51.example.com pod:catalog-catalog-webhook-c9c7f9c6-6wtzc]} {0xc000812940 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-discover-v8gq9]} {0xc0008131a0 map[namespace:catalog node:k8s-worker-51.example.com pod:catalog-catalog-apiserver-946d9cf64-4dz9l]} {0xc0008129b0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-mon-c-84d69df88-6t27d]} {0xc000813220 map[namespace:kube-system node:k8s-worker-51.example.com pod:kubernetes-dashboard-c584d7cc-wx69w]} {0xc000812a30 map[namespace:monitoring node:k8s-worker-51.example.com pod:mon-redis-slave-1]} {0xc000812aa0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-23-5555c75797-78v4v]} {0xc000812b20 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-27-76ccd4b77c-wrgnn]} {0xc000812ba0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-4-57fbc4d444-t2ppg]} {0xc000812c20 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-0-7944f87978-kqs6h]} {0xc000812ca0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-8-78cb4b95b6-ptbpg]} {0xc000812d20 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-12-b759bdf5c-m62gr]} {0xc000812da0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-19-5864484d66-pcbtt]} {0xc000812e20 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-16-74c487b9-79b2t]} {0xc0008132a0 map[namespace:aws-service-broker node:k8s-worker-51.example.com pod:aws-servicebroker-b754458b7-4gj77]} {0xc000812ea0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-sbx-primary-backup-7vt7w]} {0xc0008134a0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-discover-4c6dl]} {0xc000813780 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-19-5864484d66-9kvph]} {0xc000840280 map[namespace:monitoring node:k8s-worker-51.example.com pod:mon-redis-ha-server-0]} {0xc000813800 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-12-b759bdf5c-gmrt6]} {0xc000813880 map[namespace:catalog node:k8s-worker-51.example.com pod:catalog-catalog-controller-manager-959946d4d-pfs8x]} {0xc000813f80 map[namespace:external-dns node:k8s-worker-51.example.com pod:default-internal-external-dns-6bff4bd58d-4tgjm]} {0xc000813d00 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-sbx-primary-backup-8fxdw]} {0xc000813c80 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-4-57fbc4d444-bsm7t]} {0xc000813900 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-27-76ccd4b77c-nd9lg]} {0xc000813980 map[namespace:kube-system node:k8s-worker-51.example.com pod:expire-check-worker-s2vgk]} {0xc000813a00 map[namespace:monitoring node:k8s-worker-51.example.com pod:mon-prometheus-pushgateway-779464cc74-9qhkh]} {0xc000813a80 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-8-78cb4b95b6-shmgf]} {0xc000813b00 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-16-74c487b9-2f6n8]} {0xc000813f00 map[namespace:external-dns node:k8s-worker-51.example.com pod:default-external-external-dns-675d7dc758-77wb7]} {0xc000813e00 map[namespace:cert-manager node:k8s-worker-51.example.com pod:certmanager-cert-manager-84484dbc7d-vl2qq]} {0xc0008402f0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-mon-c-84d69df88-nw4xg]} {0xc000813d80 map[namespace:monitoring node:k8s-worker-51.example.com pod:mon-redis-slave-0]} {0xc000813420 map[namespace:velero node:k8s-worker-51.example.com pod:restic-w8wdm]} {0xc000813c00 map[namespace:kube-system node:k8s-worker-51.example.com pod:node-labeler-557b7c6596-5j4w5]} {0xc000840000 map[namespace:external-dns node:k8s-worker-51.example.com pod:top-zone-external-external-dns-c6b866d8-rqmdq]} {0xc000840080 map[namespace:external-dns node:k8s-worker-51.example.com pod:top-zone-internal-external-dns-5c97c49568-lgfgl]} {0xc000813b80 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-discover-bptzl]} {0xc000813520 map[node:k8s-worker-51.example.com type:ipipTunnelAddress]} {0xc000813580 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-0-7944f87978-d9ll4]} {0xc0008133a0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-sbx-primary-backup-mswvp]} {0xc000813600 map[namespace:velero node:k8s-worker-51.example.com pod:restic-4z886]} {0xc000840180 map[namespace:kube-system node:k8s-worker-51.example.com pod:coredns-6fbcb44b7f-d2r6x]} {0xc000813680 map[namespace:cert-manager node:k8s-worker-51.example.com pod:certmanager-cert-manager-cainjector-5d6c58bf66-kngvf]} {0xc000813700 map[namespace:rook-ceph-sbx-primary node:k8s-worker-51.example.com pod:rook-ceph-osd-23-5555c75797-788xn]}] k8s-worker-52.example.com:[{0xc0005ca0b0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-sbx-primary-backup-z9n97]} {0xc0005ca130 map[namespace:kube-system node:k8s-worker-52.example.com pod:expire-check-worker-z4ftg]} {0xc0005ca1b0 map[namespace:monitoring node:k8s-worker-52.example.com pod:mon-redis-ha-server-2]} {0xc0005cab50 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-discover-f4s4x]} {0xc0005ca950 map[namespace:velero node:k8s-worker-52.example.com pod:restic-6jvt8]} {0xc0005ca9d0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-sbx-primary-backup-ws588]} {0xc0005caa50 map[namespace:monitoring node:k8s-worker-52.example.com pod:default-http-backend-6fd4978fb-lmmk8]} {0xc0005caad0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-mon-e-74cd84fd5d-qdlw4]} {0xc0005ca230 map[namespace:lxos-sbx node:k8s-worker-52.example.com pod:default-http-backend-6fd4978fb-vp24w]} {0xc0005ca2b0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-10-5c79dc8c8f-gnpsd]} {0xc0005ca330 map[namespace:kube-system node:k8s-worker-52.example.com pod:expire-check-worker-464nd]} {0xc0005ca3b0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-operator-86dccf9fcc-g4ccd]} {0xc0005ca430 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-19-b4b8b888f-6q2sv]} {0xc0005ca4c0 map[namespace:metallb-system node:k8s-worker-52.example.com pod:metallb-controller-75cbdf4c77-qxm77]} {0xc0005ca560 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-7-5cd8cfccfc-95tcb]} {0xc0005ca5e0 map[namespace:monitoring node:k8s-worker-52.example.com pod:mon-prometheus-kube-state-metrics-6cc667cd78-62rlb]} {0xc0005ca660 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-1-54b7dc9695-q5q89]} {0xc0005ca6e0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-18-89cccdbf7-s7bvh]} {0xc0005ca760 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-15-9db58595d-kxw6k]} {0xc0005ca7e0 map[namespace:monitoring node:k8s-worker-52.example.com pod:mon-redis-ha-server-2]} {0xc0005ca850 map[namespace:monitoring node:k8s-worker-52.example.com pod:mon-grafana-5b9ccf9bb-xr8vq]} {0xc0005ca8d0 map[namespace:monitoring node:k8s-worker-52.example.com pod:mon-prometheus-server-0]} {0xc000860010 map[namespace:monitoring node:k8s-worker-52.example.com pod:deploy-547f6c9d8b-tqmh7]} {0xc0008403e0 map[namespace:kube-system node:k8s-worker-52.example.com pod:expire-check-worker-ct86r]} {0xc000840460 map[namespace:velero node:k8s-worker-52.example.com pod:restic-mr6md]} {0xc0008404e0 map[namespace:monitoring node:k8s-worker-52.example.com pod:mon-redis-slave-1]} {0xc000840f50 map[namespace:kube-system node:k8s-worker-52.example.com pod:expire-check-worker-rxqt8]} {0xc000840fd0 map[namespace:kube-system node:k8s-worker-52.example.com pod:external-dns-5d6686c558-2ztcm]} {0xc000841050 map[namespace:velero node:k8s-worker-52.example.com pod:velero-sbx-776b9fd4f4-bksmp]} {0xc0008410d0 map[namespace:lxos-sbx node:k8s-worker-52.example.com pod:default-http-backend-6fd4978fb-hm7tg]} {0xc000841150 map[namespace:kube-system node:k8s-worker-52.example.com pod:k8s-toolbox-6fd9d7f698-k9hsg]} {0xc000840560 map[namespace:kube-system node:k8s-worker-52.example.com pod:external-dns-eai-5695f9ddd9-pnwpf]} {0xc0008405e0 map[namespace:kube-system node:k8s-worker-52.example.com pod:external-dns-extern-top-zone-5ff88f6bc7-vrfpn]} {0xc0008411d0 map[namespace:kube-system node:k8s-worker-52.example.com pod:expire-check-worker-vnqfv]} {0xc000840660 map[namespace:kube-system node:k8s-worker-52.example.com pod:coredns-6fbcb44b7f-vzzjr]} {0xc0008406e0 map[namespace:elastic-sbx node:k8s-worker-52.example.com pod:nginx-ingress-controller-76bff6c8b8-7fl5g]} {0xc000840760 map[namespace:monitoring node:k8s-worker-52.example.com pod:mon-prometheus-kube-state-metrics-6cc667cd78-mmvd8]} {0xc0008407e0 map[namespace:kube-system node:k8s-worker-52.example.com pod:external-dns-top-zone-79bbbbd96b-whf5t]} {0xc000841250 map[namespace:velero node:k8s-worker-52.example.com pod:restic-cr6hj]} {0xc000840860 map[namespace:kube-system node:k8s-worker-52.example.com pod:sealed-secrets-controller-65cf6f6fc7-srnhl]} {0xc000860090 map[namespace:monitoring node:k8s-worker-52.example.com pod:mon-prometheus-kube-state-metrics-6cc667cd78-79ssg]} {0xc000841f20 map[namespace:kube-system node:k8s-worker-52.example.com pod:pdb-controller-687f679c49-r6vr2]} {0xc000841ea0 map[namespace:prometheus node:k8s-worker-52.example.com pod:alertmanager-master-kube-prometheus-sta-alertmanager-1]} {0xc0008408e0 map[namespace:monitoring node:k8s-worker-52.example.com pod:mon-redis-ha-server-2]} {0xc000840950 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-discover-w9b8f]} {0xc0008409d0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-mon-d-54f5f6bff-hb5gd]} {0xc000840a50 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-25-86dbc999f7-slzgg]} {0xc000840ad0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-29-77bfbdfd85-cvltz]} {0xc000840b50 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-3-7d99c48f47-wzcpj]} {0xc000840bd0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-31-677b4dbcc8-gd49f]} {0xc000840c50 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-7-58565bd855-qljz2]} {0xc000840cd0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-11-7c7b47c5dd-ld4mf]} {0xc000840d50 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-15-866588fb-gvnlk]} {0xc000840dd0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-21-68d64777fc-przlj]} {0xc000841e20 map[namespace:prometheus node:k8s-worker-52.example.com pod:prometheus-master-kube-prometheus-sta-prometheus-0]} {0xc000840e50 map[namespace:monitoring node:k8s-worker-52.example.com pod:mon-prometheus-server-577c57785c-8lpwk]} {0xc000840ed0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-sbx-primary-backup-4ljs2]} {0xc000841f90 map[namespace:kube-system node:k8s-worker-52.example.com pod:sealed-secrets-controller-7ccb6648c5-gxdhz]} {0xc0008412d0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-sbx-primary-backup-b7c8w]} {0xc000841d20 map[namespace:elastic-sbx node:k8s-worker-52.example.com pod:nginx-ingress-controller-76bff6c8b8-wjph7]} {0xc000841c20 map[namespace:kube-system node:k8s-worker-52.example.com pod:kubernetes-dashboard-c584d7cc-8qdw6]} {0xc000841350 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-discover-fb9t7]} {0xc000841910 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-21-68d64777fc-gdkdq]} {0xc000841ca0 map[namespace:kube-system node:k8s-worker-52.example.com pod:nginx-ingress-controller-6f8f6bd84b-vdf8c]} {0xc000841da0 map[namespace:kube-system node:k8s-worker-52.example.com pod:coredns-6fbcb44b7f-4rr66]} {0xc000860110 map[namespace:monitoring node:k8s-worker-52.example.com pod:nginx-ingress-controller-74c88944d7-ckn65]} {0xc000841ba0 map[namespace:velero node:k8s-worker-52.example.com pod:backup-velero-84f95868ff-lbwhl]} {0xc000860180 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-mon-d-54f5f6bff-qqpqs]} {0xc000841990 map[namespace:elastic-sbx node:k8s-worker-52.example.com pod:default-http-backend-6fd4978fb-jghc9]} {0xc000841a10 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-discover-gt7pg]} {0xc000841aa0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-25-86dbc999f7-vcgmc]} {0xc000841b20 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-11-7c7b47c5dd-x8n89]} {0xc0008413d0 map[node:k8s-worker-52.example.com type:ipipTunnelAddress]} {0xc000841430 map[namespace:velero node:k8s-worker-52.example.com pod:restic-wffzw]} {0xc0008414b0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-31-677b4dbcc8-xxp7t]} {0xc000841530 map[namespace:prometheus node:k8s-worker-52.example.com pod:master-grafana-665d79dc5b-27f9t]} {0xc0008415a0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-29-77bfbdfd85-qhvtj]} {0xc000841620 map[namespace:kube-system node:k8s-worker-52.example.com pod:dashboard-metrics-scraper-6fb9d4c667-7z8ft]} {0xc0008416a0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-3-7d99c48f47-6467p]} {0xc000841720 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-7-58565bd855-r5vb5]} {0xc0008417a0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-osd-15-866588fb-xnnkd]} {0xc000841820 map[namespace:kube-system node:k8s-worker-52.example.com pod:expire-check-worker-6pd2p]} {0xc0008418a0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-52.example.com pod:rook-ceph-sbx-primary-backup-p2sms]}] k8s-worker-53.example.com:[{0xc000405990 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-sbx-primary-backup-nc744]} {0xc000405a00 map[namespace:kube-system node:k8s-worker-53.example.com pod:expire-check-worker-dlnxp]} {0xc0007ea830 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-sbx-primary-backup-cszgx]} {0xc000405e60 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-3-c46bcd56b-4crxh]} {0xc000405ed0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-14-85d9d97f58-nxtsm]} {0xc000405f40 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-mon-a-7ffdfb99d9-dvz5j]} {0xc000405fb0 map[namespace:kube-system node:k8s-worker-53.example.com pod:expire-check-worker-zglmt]} {0xc000405a70 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-13-84597f6fff-vfs2g]} {0xc000405ae0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-12-756c685d88-f4kfl]} {0xc000405b50 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-8-99d59c7b7-lzh66]} {0xc000405bc0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-sbx-primary-backup-v7k22]} {0xc000405c30 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-discover-5qm9d]} {0xc000405ca0 map[namespace:kube-system node:k8s-worker-53.example.com pod:coredns-d5999b86b-k49bj]} {0xc000405d10 map[namespace:monitoring node:k8s-worker-53.example.com pod:mon-prometheus-alertmanager-59b857f46-hp925]} {0xc000405d80 map[namespace:velero node:k8s-worker-53.example.com pod:restic-q5g4q]} {0xc000405df0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-16-7d554b969c-x2z7r]} {0xc0007ea1a0 map[namespace:monitoring node:k8s-worker-53.example.com pod:mon-redis-ha-server-0]} {0xc0007ea020 map[namespace:monitoring node:k8s-worker-53.example.com pod:mon-redis-slave-1]} {0xc0007ea0a0 map[namespace:kube-system node:k8s-worker-53.example.com pod:external-dns-extern-78cdb9fbc6-d6pvk]} {0xc0007ea120 map[namespace:lxos-sbx node:k8s-worker-53.example.com pod:default-http-backend-6fd4978fb-dvpfd]} {0xc0007ea230 map[namespace:monitoring node:k8s-worker-53.example.com pod:mon-prometheus-server-1]} {0xc0007ea2b0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-sbx-primary-backup-2f56j]} {0xc0007ea330 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-discover-kl4mn]} {0xc0007ea3b0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-mon-e-7557749798-rvwgt]} {0xc0007ea420 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-17-cc578fbb4-6jqhx]} {0xc0007ea4a0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-18-775d7ccf-v5pd7]} {0xc0007ea520 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-16-6664f9cc77-sknnz]} {0xc0007ea5b0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-6-5dbdc78988-n2zg5]} {0xc0007ea630 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-3-5f4f7445bc-kbxlx]} {0xc0007ea6b0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-20-6fdc55798b-g6szp]} {0xc0007ea730 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-10-6f899f975f-tg6bn]} {0xc0007ea7b0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-0-6f7c44796b-lcqnf]} {0xc0007f4c80 map[namespace:cert-manager node:k8s-worker-53.example.com pod:cert-manager-cainjector-6659d6844d-qdtgf]} {0xc0007f4910 map[namespace:kube-system node:k8s-worker-53.example.com pod:expire-check-worker-fqg7x]} {0xc0007f5100 map[namespace:velero node:k8s-worker-53.example.com pod:restic-gs6vb]} {0xc0007f5180 map[namespace:kube-system node:k8s-worker-53.example.com pod:pdb-controller-687f679c49-l5wlj]} {0xc0007f5580 map[namespace:kube-system node:k8s-worker-53.example.com pod:expire-check-worker-4jmgw]} {0xc0007f5200 map[namespace:kube-system node:k8s-worker-53.example.com pod:expire-check-worker-wf9tt]} {0xc0007f5280 map[namespace:lxos-sbx node:k8s-worker-53.example.com pod:nginx-ingress-controller-855f55f857-lnz28]} {0xc0007f5300 map[namespace:kube-system node:k8s-worker-53.example.com pod:nginx-ingress-controller-6f8f6bd84b-w864p]} {0xc0007f4990 map[namespace:elastic-sbx node:k8s-worker-53.example.com pod:nginx-ingress-controller-76bff6c8b8-2jc8p]} {0xc0007f4a00 map[namespace:kube-system node:k8s-worker-53.example.com pod:dashboard-metrics-scraper-6fb9d4c667-xmc7q]} {0xc0007f4a80 map[namespace:kube-system node:k8s-worker-53.example.com pod:calico-kube-controllers-648f4868b8-nxqsp]} {0xc0008123d0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-mon-e-596c86cc96-z5q8s]} {0xc0007f5380 map[namespace:cert-manager node:k8s-worker-53.example.com pod:cert-manager-webhook-547567b88f-mbflj]} {0xc0007f5400 map[namespace:monitoring node:k8s-worker-53.example.com pod:deploy-547f6c9d8b-vcp7f]} {0xc0007f4b00 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-discover-45mk8]} {0xc0007f4b80 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-mon-e-596c86cc96-k7tfm]} {0xc0007f5480 map[namespace:catalog node:k8s-worker-53.example.com pod:catalog-catalog-controller-manager-75d98fbc4d-jt8l8]} {0xc0007f4c00 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-18-5b7bb4f6f-5bp5f]} {0xc0007f4d00 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-2-84f4ccd5bb-d9msq]} {0xc0007f4d80 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-26-59b9bbbb54-9cg64]} {0xc0007f4e00 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-22-85ccd74f66-wtfsq]} {0xc0007f4e80 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-30-bcf498ccf-9659q]} {0xc0007f4f00 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-10-5bf8f68557-5z6qf]} {0xc0007f4f80 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-6-6d4c4dbc44-nswmj]} {0xc0007f5000 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-14-786d68dbc7-lgq4l]} {0xc0007f54f0 map[namespace:kube-system node:k8s-worker-53.example.com pod:coredns-6fbcb44b7f-rd8df]} {0xc0007f5080 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-sbx-primary-backup-6sjr9]} {0xc000812250 map[namespace:external-dns node:k8s-worker-53.example.com pod:eai-internal-external-dns-8699f9d89-q52d5]} {0xc0007f5600 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-sbx-primary-backup-svflf]} {0xc000812350 map[namespace:prometheus node:k8s-worker-53.example.com pod:prometheus-master-kube-prometheus-sta-prometheus-1]} {0xc0008121d0 map[namespace:elastic-sbx node:k8s-worker-53.example.com pod:nginx-ingress-controller-76bff6c8b8-8l6lg]} {0xc0007f5680 map[namespace:velero node:k8s-worker-53.example.com pod:restic-6zb29]} {0xc0007f5700 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-discover-7m5dr]} {0xc0007f5f50 map[namespace:prometheus node:k8s-worker-53.example.com pod:master-kube-state-metrics-69cdbd8958-j84pd]} {0xc0008122d0 map[namespace:monitoring node:k8s-worker-53.example.com pod:mon-redis-ha-server-2]} {0xc0007f5fd0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-30-bcf498ccf-9gb8w]} {0xc000812150 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-2-84f4ccd5bb-mz8xm]} {0xc000812050 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-18-5b7bb4f6f-kxcf8]} {0xc0008120d0 map[namespace:monitoring node:k8s-worker-53.example.com pod:mon-redis-master-0]} {0xc0007f5780 map[node:k8s-worker-53.example.com type:ipipTunnelAddress]} {0xc0007f57d0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-6-6d4c4dbc44-5j4ft]} {0xc0007f5850 map[namespace:kube-system node:k8s-worker-53.example.com pod:nginx-ingress-controller-6f8f6bd84b-4n2q9]} {0xc0007f58d0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-discover-2ffbl]} {0xc0007f5950 map[namespace:velero node:k8s-worker-53.example.com pod:restic-p5lkm]} {0xc0007f59d0 map[namespace:external-dns node:k8s-worker-53.example.com pod:eai-external-external-dns-5b44d4c4d5-k7rss]} {0xc0007f5a50 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-26-59b9bbbb54-hpf9k]} {0xc0007f5ad0 map[namespace:k8s-updates node:k8s-worker-53.example.com pod:k8s-updates-7d6f4b56f7-nhdj5]} {0xc0007f5b50 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-14-786d68dbc7-jk29b]} {0xc0007f5bd0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-10-5bf8f68557-fpjrl]} {0xc0007f5c50 map[namespace:kube-system node:k8s-worker-53.example.com pod:metrics-server-64d955cbc6-qsmgg]} {0xc0007f5cd0 map[namespace:kube-system node:k8s-worker-53.example.com pod:expire-check-worker-8jw5s]} {0xc0007f5d50 map[namespace:prometheus node:k8s-worker-53.example.com pod:alertmanager-master-kube-prometheus-sta-alertmanager-0]} {0xc0007f5dd0 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-osd-22-85ccd74f66-v2vwk]} {0xc0007f5e50 map[namespace:rook-ceph-sbx-primary node:k8s-worker-53.example.com pod:rook-ceph-sbx-primary-backup-qtdg5]} {0xc0007f5ed0 map[namespace:monitoring node:k8s-worker-53.example.com pod:mon-prometheus-alertmanager-59b857f46-x7j6t]}]]
2020-12-08 17:19:41.011 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-22.example.com" k8sNode="k8s-worker-22.example.com"
2020-12-08 17:19:41.015 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-50.example.com" k8sNode="k8s-worker-50.example.com"
2020-12-08 17:19:41.020 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-00.example.com" k8sNode="k8s-worker-00.example.com"
2020-12-08 17:19:41.026 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-53.example.com" k8sNode="k8s-worker-53.example.com"
2020-12-08 17:19:41.030 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-master-01.example.com" k8sNode="k8s-master-01.example.com"
2020-12-08 17:19:41.030 [DEBUG][1] k8s.go 564: Performing 'Get' for Node(k8s-worker-20.example.com)
2020-12-08 17:19:41.030 [DEBUG][1] node.go 114: Received Get request on Node type
2020-12-08 17:19:41.034 [INFO][1] ipam.go 282: Calico Node referenced in IPAM data does not exist error=resource does not exist: Node(k8s-worker-20.example.com) with error: nodes "k8s-worker-20.example.com" not found
2020-12-08 17:19:41.034 [INFO][1] ipam.go 137: Checking node calicoNode="k8s-worker-20.example.com" k8sNode=""
2020-12-08 17:19:42.314 [INFO][1] ipam.go 178: Cleaning up IPAM resources for deleted node calicoNode="k8s-worker-20.example.com" k8sNode=""

[...]

2020-12-08 17:19:48.772 [INFO][1] ipam.go 1325: Releasing all IPs with handle 'k8s-pod-network.ea59dfc594ff620a37a7936051d94f287dfe30fc34f0680212fe9f9548b64ff6'
2020-12-08 17:19:48.772 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.ea59dfc594ff620a37a7936051d94f287dfe30fc34f0680212fe9f9548b64ff6)
2020-12-08 17:19:48.772 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.ea59dfc594ff620a37a7936051d94f287dfe30fc34f0680212fe9f9548b64ff6) Resource="IPAMHandles" Revision=""
2020-12-08 17:19:48.772 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.ea59dfc594ff620a37a7936051d94f287dfe30fc34f0680212fe9f9548b64ff6) Name="k8s-pod-network.ea59dfc594ff620a37a7936051d94f287dfe30fc34f0680212fe9f9548b64ff6" Namespace="" Resource="IPAMHandles" Revision=""
2020-12-08 17:19:48.974 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.ea59dfc594ff620a37a7936051d94f287dfe30fc34f0680212fe9f9548b64ff6) Name="k8s-pod-network.ea59dfc594ff620a37a7936051d94f287dfe30fc34f0680212fe9f9548b64ff6" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.ea59dfc594ff620a37a7936051d94f287dfe30fc34f0680212fe9f9548b64ff6" not found
2020-12-08 17:19:48.974 [DEBUG][1] ipam.go 207: IP already released calicoNode="k8s-worker-20.example.com" handle="k8s-pod-network.ea59dfc594ff620a37a7936051d94f287dfe30fc34f0680212fe9f9548b64ff6\r\neth0"

2020-12-08 17:19:48.974 [INFO][1] ipam.go 1325: Releasing all IPs with handle 'k8s-pod-network.5f934f9128f834ad21de044bce97524e6ff595c95798d26b7dd5c5aa323575eb'
2020-12-08 17:19:48.974 [DEBUG][1] k8s.go 564: Performing 'Get' for IPAMHandleKey(id=k8s-pod-network.5f934f9128f834ad21de044bce97524e6ff595c95798d26b7dd5c5aa323575eb)
2020-12-08 17:19:48.974 [DEBUG][1] customresource.go 208: Get custom Kubernetes resource Key=IPAMHandle(k8s-pod-network.5f934f9128f834ad21de044bce97524e6ff595c95798d26b7dd5c5aa323575eb) Resource="IPAMHandles" Revision=""
2020-12-08 17:19:48.974 [DEBUG][1] customresource.go 219: Get custom Kubernetes resource by name Key=IPAMHandle(k8s-pod-network.5f934f9128f834ad21de044bce97524e6ff595c95798d26b7dd5c5aa323575eb) Name="k8s-pod-network.5f934f9128f834ad21de044bce97524e6ff595c95798d26b7dd5c5aa323575eb" Namespace="" Resource="IPAMHandles" Revision=""
2020-12-08 17:19:49.172 [DEBUG][1] customresource.go 228: Error getting resource Key=IPAMHandle(k8s-pod-network.5f934f9128f834ad21de044bce97524e6ff595c95798d26b7dd5c5aa323575eb) Name="k8s-pod-network.5f934f9128f834ad21de044bce97524e6ff595c95798d26b7dd5c5aa323575eb" Namespace="" Resource="IPAMHandles" Revision="" error=ipamhandles.crd.projectcalico.org "k8s-pod-network.5f934f9128f834ad21de044bce97524e6ff595c95798d26b7dd5c5aa323575eb" not found
2020-12-08 17:19:49.173 [DEBUG][1] ipam.go 207: IP already released calicoNode="k8s-worker-20.example.com" handle="k8s-pod-network.5f934f9128f834ad21de044bce97524e6ff595c95798d26b7dd5c5aa323575eb\r\neth0"

2020-12-08 17:19:49.173 [INFO][1] ipam.go 218: Couldn't release all IPs for stale node, schedule retry calicoNode="k8s-worker-20.example.com"
2020-12-08 17:19:49.173 [WARNING][1] ipam.go 181: Error cleaning up node calicoNode="k8s-worker-20.example.com" error=Couldn't release all IPs k8sNode=""
2020-12-08 17:19:49.178 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-51.example.com" k8sNode="k8s-worker-51.example.com"
2020-12-08 17:19:49.183 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-master-02.example.com" k8sNode="k8s-master-02.example.com"
2020-12-08 17:19:49.187 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-52.example.com" k8sNode="k8s-worker-52.example.com"
2020-12-08 17:19:49.191 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-21.example.com" k8sNode="k8s-worker-21.example.com"
2020-12-08 17:19:49.195 [DEBUG][1] ipam.go 133: Node still exists, continue calicoNode="k8s-worker-23.example.com" k8sNode="k8s-worker-23.example.com"

I split the logs in multiple code blocks since the "Releasing all IPs"-blocks are examples and logged multiple times for different ipamhandles. The "Node still exists, continue"-lines change every time it appears for different hosts and different host counts.

[caseydavenport]

@TheKangaroo sorry for the delay...

2020-12-08 17:19:49.173 [DEBUG][1] ipam.go 207: IP already released calicoNode="k8s-worker-20.example.com" handle="k8s-pod-network.5f934f9128f834ad21de044bce97524e6ff595c95798d26b7dd5c5aa323575eb\r\neth0"

It looks like your data is slightly more corrupted than I originally thought. From the logs, it looks like the IPAM data is not in sync. Normally, we expect an allocation and a handle to be 1:1 and kept in sync. However, it seems that the IPAM handle has been deleted but the allocation has not been. As a result, we keep spotting the allocation and trying to release it, but when we go to release it we see that the handle doesn't exist, and assume that as a result the IP must have already been released.

I'm not sure what the fix is, or how you got into this state. In normal operating conditions I wouldn't expect this to occur, since our code guarantees that the handle is deleted only after the allocation itself has been freed.

Ideally what we'd do here is fix up your corrupted data by freeing the addresses which are now "floating" without a handle.

[TheKangaroo]

@caseydavenport thank you for your reply. I did another research based on your recent comment. First of all, by allocation you mean the handle_id entries in the IPAMBlock resources, right? Based on this hint I did some additional research. First I have to admit I saw some old IPAMBlocks left over from before we redeployed Calico with the new pod CIDR. I thought they were gone and we did a clean reinstall. Im not sure if I'm safe to just delete those.

kubectl get ipamblocks
NAME                 AGE
100-192-0-0-26       364d
[...]
100-80-10-0-26       88d
[...]

With that out of the way, I checked some entries in an IPAMBlock and can confirm, there are existing and nonexisiting entries. Here are three examples:

❯ k get ipamblock 100-80-7-0-26 -oyaml
[...]
    - handle_id: "k8s-pod-network.df7edfe420b338934f51bfd9ec2c1cd619bb1613f7f4863ab526a2631adf1f19\r\neth0"
      secondary:
        namespace: kube-system
        node: k8s-worker-52.example.com
        pod: k8s-toolbox-6fd9d7f698-k9hsg
    - handle_id: k8s-pod-network.10e3c088fd57051b142249f22667f781580facce1e76498f8df511939a388d37
      secondary:
        namespace: kube-system
        node: k8s-worker-52.example.com
        pod: expire-check-worker-vnqfv
[...]
    - handle_id: k8s-pod-network.471f102cf0564bc0195589f1f23fe63fe9186249304072ab4bf32302ac4fd7b6
      secondary:
        namespace: rook-ceph-sbx-primary
        node: k8s-worker-52.example.com
        pod: rook-ceph-osd-13-58f7468bb9-njh45
[...]

Of course I cannot find a matching pod for the first handle (with suffix \r\neth0), but the second entry is also a stale entry with no existig pod. The third one has an existing pod as expected:

❯ k get pod -owide --all-namespaces|grep k8s-toolbox-6fd9d7f698-k9hsg
❯ k get pod -owide --all-namespaces|grep expire-check-worker-vnqfv
❯ k get pod -owide --all-namespaces|grep rook-ceph-osd-13-58f7468bb9-njh45
rook-ceph-sbx-primary          rook-ceph-osd-13-58f7468bb9-njh45                              1/1     Running            0          4d21h   100.80.7.19      k8s-worker-52.example.com   <none>           <none>

Next I checkt the ipamhandles and all but the \r\neth0 suffix exist:

❯  k get ipamhandle k8s-pod-network.df7edfe420b338934f51bfd9ec2c1cd619bb1613f7f4863ab526a2631adf1f19\r\neth0
Error from server (NotFound): ipamhandles.crd.projectcalico.org "k8s-pod-network.df7edfe420b338934f51bfd9ec2c1cd619bb1613f7f4863ab526a2631adf1f19rneth0" not found
❯  k get ipamhandle k8s-pod-network.10e3c088fd57051b142249f22667f781580facce1e76498f8df511939a388d37
NAME                                                                               AGE
k8s-pod-network.10e3c088fd57051b142249f22667f781580facce1e76498f8df511939a388d37   37d
❯  k get ipamhandle k8s-pod-network.471f102cf0564bc0195589f1f23fe63fe9186249304072ab4bf32302ac4fd7b6
NAME                                                                               AGE
k8s-pod-network.471f102cf0564bc0195589f1f23fe63fe9186249304072ab4bf32302ac4fd7b6   4d21h

This leads me to the conclutions:

• There is an IPAMHandle with "\r\neth0" suffix left over as an allocation in the IPAMBlock but no IPAMHandle with this name exists (with or without the suffix).
• We have an allocation in the IPAMBlock with no existing pod but an matching IPAMHandle object.
• we have an allocation in the IPAMBlock with an existing pod and an existing IPAMHandle object.

To test things are working at all, I deleted the pod which is working fine in the example above and the pod, the IPAMHandle and the allocation in the IPAMBlock are gone. This seems to work perfectly fine.

As a last step, I checked if there are any allocations in IPAMBlocks which have no IPAMHandle (besides the ones with \r\neth0). I did so with the very basic/hacky command:

❯ for handle in $(k get ipamblock -oyaml |grep handle_id|grep k8s-pod-network|grep -v eth0 |awk '{ print $3 }') ;do k get ipamhandle $handle; done

It turned out, that there are no allocations in IPAMBlocks without a matching IPAMHandle, so this looks good (besides the ones with \r\neth0 of course). (I run this against two other clusters which use calico 1.16.x and had one case where an IPAMHandle did not exist)

[TheKangaroo]

I used the same command from above to get all ipamhandles referenced in ipamblocks and ran a diff against the actual ipamblocks:

❯ diff /tmp/ipamblockreference.txt /tmp/ipamhandle.txt
0a1,11
> NAME
> ipip-tunnel-addr-k8s-master-01.example.com
> ipip-tunnel-addr-k8s-master-02.example.com
> ipip-tunnel-addr-k8s-worker-00.example.com
> ipip-tunnel-addr-k8s-worker-20.example.com
> ipip-tunnel-addr-k8s-worker-21.example.com
> ipip-tunnel-addr-k8s-worker-23.example.com
> ipip-tunnel-addr-k8s-worker-50.example.com
> ipip-tunnel-addr-k8s-worker-51.example.com
> ipip-tunnel-addr-k8s-worker-52.example.com
> ipip-tunnel-addr-k8s-worker-53.example.com
135a147
> k8s-pod-network.d86b9714ae409330aa73498c820f482faebf073f61942cb9291857c8f53931df
144a157
> k8s-pod-network.e81a9d879d60c90eb7385b4ea0af7c11cbda700ab7032b8e7b20d80403e766ca

This does not look too bad ;) Note that I excluded the references with \r\neth0 from the IPAMBlocks list. These are still present there without corresponding IPAMHandles.

Here are the yaml outputs of the IPAMHandles above:

❯ k get ipamhandle k8s-pod-network.d86b9714ae409330aa73498c820f482faebf073f61942cb9291857c8f53931df -oyaml
apiVersion: crd.projectcalico.org/v1
kind: IPAMHandle
metadata:
  annotations:
    projectcalico.org/metadata: '{"creationTimestamp":null}'
  creationTimestamp: "2020-12-07T09:32:01Z"
  generation: 1
  name: k8s-pod-network.d86b9714ae409330aa73498c820f482faebf073f61942cb9291857c8f53931df
  resourceVersion: "120206815"
  selfLink: /apis/crd.projectcalico.org/v1/ipamhandles/k8s-pod-network.d86b9714ae409330aa73498c820f482faebf073f61942cb9291857c8f53931df
  uid: 150f2352-91de-4c2a-932e-b87ee79f5974
spec:
  block:
    100.80.10.0/26: 1
  handleID: k8s-pod-network.d86b9714ae409330aa73498c820f482faebf073f61942cb9291857c8f53931df
❯ k get ipamhandle k8s-pod-network.e81a9d879d60c90eb7385b4ea0af7c11cbda700ab7032b8e7b20d80403e766ca -oyaml
apiVersion: crd.projectcalico.org/v1
kind: IPAMHandle
metadata:
  annotations:
    projectcalico.org/metadata: '{"creationTimestamp":null}'
  creationTimestamp: "2020-12-07T05:13:53Z"
  generation: 1
  name: k8s-pod-network.e81a9d879d60c90eb7385b4ea0af7c11cbda700ab7032b8e7b20d80403e766ca
  resourceVersion: "120109825"
  selfLink: /apis/crd.projectcalico.org/v1/ipamhandles/k8s-pod-network.e81a9d879d60c90eb7385b4ea0af7c11cbda700ab7032b8e7b20d80403e766ca
  uid: 81a1d616-f7f3-4840-9454-d60a6d743d6d
spec:
  block:
    100.80.10.0/26: 1
  handleID: k8s-pod-network.e81a9d879d60c90eb7385b4ea0af7c11cbda700ab7032b8e7b20d80403e766ca

Both are on the same IPAMBlock 100.80.10.0, but do not appear as a reference there:

❯ k get ipamblock 100-80-10-0-26 -oyaml
apiVersion: crd.projectcalico.org/v1
kind: IPAMBlock
metadata:
  annotations:
    projectcalico.org/metadata: '{"creationTimestamp":null}'
  creationTimestamp: "2020-10-17T10:10:25Z"
  generation: 378
  name: 100-80-10-0-26
  resourceVersion: "141069737"
  selfLink: /apis/crd.projectcalico.org/v1/ipamblocks/100-80-10-0-26
  uid: 868cfe56-dba6-4f67-bf03-6bf59bb98439
spec:
  affinity: host:k8s-master-01.example.com
  allocations:
  - 3
  - null
  - 0
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - null
  - 1
  - null
  - null
  - null
  - null
  - null
  - 4
  - null
  - 2
  - null
  - null
  attributes:
  - handle_id: "k8s-pod-network.cc6c64ecba6510b1b5e092e090a600214e77aadda2304754dd69a680dde1adea\r\neth0"
    secondary:
      namespace: kube-system
      node: k8s-master-01.example.com
      pod: expire-check-master-9mj2v
  - handle_id: k8s-pod-network.e4fdf1e9f7f52cbcd1ca9aaa21e8caf198583dd5a2c73f8502a267973fba9454
    secondary:
      namespace: kube-system
      node: k8s-master-01.example.com
      pod: expire-check-master-zqdgn
  - handle_id: ipip-tunnel-addr-k8s-master-01.lxsbx.ka.de.dm-drogeriemarkt.com
    secondary:
      node: k8s-master-01.example.com
      type: ipipTunnelAddress
  - handle_id: k8s-pod-network.bb2a5c8d89085f5a8bd98f3e29b99536b0fd1293bee5227a706ecf93f6785edc
    secondary:
      namespace: kube-system
      node: k8s-master-01.example.com
      pod: expire-check-master-lfvjb
  - handle_id: k8s-pod-network.476556f03fbd13380d393d552118d3d6aefb8c9605d0624dd18c2f83fbf6cf1b
    secondary:
      namespace: monitoring
      node: k8s-master-01.example.com
      pod: ephemeral-demo-645f5f7f95-mrfqc
  cidr: 100.80.10.0/26
  deleted: false
  strictAffinity: false
  unallocated:
  - 5
  - 3
  - 4
  - 63
  - 7
  - 8
  - 6
  - 12
  - 9
  - 11
  - 10
  - 19
  - 14
  - 15
  - 17
  - 16
  - 13
  - 20
  - 21
  - 18
  - 22
  - 23
  - 32
  - 25
  - 24
  - 29
  - 26
  - 28
  - 27
  - 31
  - 33
  - 30
  - 34
  - 36
  - 37
  - 39
  - 38
  - 35
  - 41
  - 40
  - 45
  - 46
  - 42
  - 43
  - 54
  - 44
  - 47
  - 52
  - 55
  - 51
  - 50
  - 48
  - 57
  - 60
  - 56
  - 58
  - 1
  - 62
  - 49

But this are "only" two IPAMHandles. May be these will be deleted by the garbage collection soon. Anyways I just wanted to add this as an additional information.

[oldthreefeng]

same isuue with @TheKangaroo

[jayunit100]

is this possibly related to https://github.com/projectcalico/calico/issues/4179 ? (updated, earlier i posted the wrong link) :) thanks to kangaroo for noticing :)

[TheKangaroo]

@jayunit100 3940 is this issue ;-) Maybe you misstook the issue id?

[caseydavenport]

@TheKangaroo I'm afraid I sort of lost track of this one.

It looks like we resolved one issue with the invalid handle format, but that there is still a corrupted state for just a couple of allocations where the handle no longer exists but the IP is still allocated within the block.

your detailed comment above seems to confirm this. I think there is only one way to free those IPs, which would be to use the calicoctl command like so:

calicoctl ipam release --ip=<IP>

I would hope that releases the IPs from the blocks, even if they have no handle remaining.

[TheKangaroo]

Thank you :) I'll try to release the ip's with the cli, but I didn't work much with our affected clusters lately. So it could take a while till I can provide some feedback.