Some pods in kube-system ns do not seem to respond to KubePlus

[mohsaman]

Pods in kube-system that do not seem to respond to KubePlus are: (log below)

metrics-server-5f956b6d5f-zv8fr kube-scheduler-master kube-apiserver-master etcd-master

vagrant@master:~/kubeplus/scripts$ kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE calico-kube-controllers-578894d4cd-dk7cz 1/1 Running 14 23d calico-node-5frgs 1/1 Running 10 23d calico-node-hc5k2 1/1 Running 14 23d calico-node-pvt6m 1/1 Running 9 23d coredns-66bff467f8-4m5qc 1/1 Running 14 23d coredns-66bff467f8-wsvtw 1/1 Running 14 23d etcd-master 1/1 Running 14 23d ksniff-hsmgw 1/1 Running 0 86m kube-apiserver-master 1/1 Running 16 23d kube-controller-manager-master 1/1 Running 14 23d kube-proxy-bnd4z 1/1 Running 10 23d kube-proxy-dh7c4 1/1 Running 14 23d kube-proxy-xcdz9 1/1 Running 9 23d kube-scheduler-master 1/1 Running 14 23d metrics-server-5f956b6d5f-zv8fr 1/1 Running 5 19d

vagrant@master:~/kubeplus/scripts$ kubectl connections Pod etcd-master kube-system Discovering node - Level: 1, Kind:Pod, instance:etcd-master Resource master of kind Node in namespace kube-system does not exist.

vagrant@master:~/kubeplus/scripts$ kubectl connections Pod kube-apiserver-master kube-system Discovering node - Level: 1, Kind:Pod, instance:kube-apiserver-master Resource master of kind Node in namespace kube-system does not exist.

[devdattakulkarni]

Thanks @mohsaman

[devdattakulkarni]

@mohsaman Can you share output of the following commands:

kubectl get pods kube-api-server-master -n kube-system -o json

kubectl get pods etcd-master -n kube-system -o json

[mohsaman]

@devdattakulkarni Sorry, just saw this. Here you are, two commands and output one after the other:

kubectl get pods kube-api-server-master -n kube-system -o json

{ "apiVersion": "v1", "kind": "Pod", "metadata": { "annotations": { "kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint": "192.168.7.50:6443", "kubernetes.io/config.hash": "23a8b14b3552feacc12ae8f94ab13196", "kubernetes.io/config.mirror": "23a8b14b3552feacc12ae8f94ab13196", "kubernetes.io/config.seen": "2020-08-08T22:50:06.758522254Z", "kubernetes.io/config.source": "file" }, "creationTimestamp": "2020-08-08T22:50:07Z", "labels": { "component": "kube-apiserver", "tier": "control-plane" }, "managedFields": [ { "apiVersion": "v1", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:annotations": { ".": {}, "f:kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint": {}, "f:kubernetes.io/config.hash": {}, "f:kubernetes.io/config.mirror": {}, "f:kubernetes.io/config.seen": {}, "f:kubernetes.io/config.source": {} }, "f:labels": { ".": {}, "f:component": {}, "f:tier": {} }, "f:ownerReferences": { ".": {}, "k:{\"uid\":\"77eb0e7c-51a6-45b9-89f6-8993f3d19fce\"}": { ".": {}, "f:apiVersion": {}, "f:controller": {}, "f:kind": {}, "f:name": {}, "f:uid": {} } } }, "f:spec": { "f:containers": { "k:{\"name\":\"kube-apiserver\"}": { ".": {}, "f:command": {}, "f:image": {}, "f:imagePullPolicy": {}, "f:livenessProbe": { ".": {}, "f:failureThreshold": {}, "f:httpGet": { ".": {}, "f:host": {}, "f:path": {}, "f:port": {}, "f:scheme": {} }, "f:initialDelaySeconds": {}, "f:periodSeconds": {}, "f:successThreshold": {}, "f:timeoutSeconds": {} }, "f:name": {}, "f:resources": { ".": {}, "f:requests": { ".": {}, "f:cpu": {} } }, "f:terminationMessagePath": {}, "f:terminationMessagePolicy": {}, "f:volumeMounts": { ".": {}, "k:{\"mountPath\":\"/etc/ca-certificates\"}": { ".": {}, "f:mountPath": {}, "f:name": {}, "f:readOnly": {} }, "k:{\"mountPath\":\"/etc/kubernetes/pki\"}": { ".": {}, "f:mountPath": {}, "f:name": {}, "f:readOnly": {} }, "k:{\"mountPath\":\"/etc/pki\"}": { ".": {}, "f:mountPath": {}, "f:name": {}, "f:readOnly": {} }, "k:{\"mountPath\":\"/etc/ssl/certs\"}": { ".": {}, "f:mountPath": {}, "f:name": {}, "f:readOnly": {} }, "k:{\"mountPath\":\"/usr/local/share/ca-certificates\"}": { ".": {}, "f:mountPath": {}, "f:name": {}, "f:readOnly": {} }, "k:{\"mountPath\":\"/usr/share/ca-certificates\"}": { ".": {}, "f:mountPath": {}, "f:name": {}, "f:readOnly": {} } } } }, "f:dnsPolicy": {}, "f:enableServiceLinks": {}, "f:hostNetwork": {}, "f:nodeName": {}, "f:priorityClassName": {}, "f:restartPolicy": {}, "f:schedulerName": {}, "f:securityContext": {}, "f:terminationGracePeriodSeconds": {}, "f:tolerations": {}, "f:volumes": { ".": {}, "k:{\"name\":\"ca-certs\"}": { ".": {}, "f:hostPath": { ".": {}, "f:path": {}, "f:type": {} }, "f:name": {} }, "k:{\"name\":\"etc-ca-certificates\"}": { ".": {}, "f:hostPath": { ".": {}, "f:path": {}, "f:type": {} }, "f:name": {} }, "k:{\"name\":\"etc-pki\"}": { ".": {}, "f:hostPath": { ".": {}, "f:path": {}, "f:type": {} }, "f:name": {} }, "k:{\"name\":\"k8s-certs\"}": { ".": {}, "f:hostPath": { ".": {}, "f:path": {}, "f:type": {} }, "f:name": {} }, "k:{\"name\":\"usr-local-share-ca-certificates\"}": { ".": {}, "f:hostPath": { ".": {}, "f:path": {}, "f:type": {} }, "f:name": {} }, "k:{\"name\":\"usr-share-ca-certificates\"}": { ".": {}, "f:hostPath": { ".": {}, "f:path": {}, "f:type": {} }, "f:name": {} } } }, "f:status": { "f:conditions": { ".": {}, "k:{\"type\":\"ContainersReady\"}": { ".": {}, "f:lastProbeTime": {}, "f:lastTransitionTime": {}, "f:status": {}, "f:type": {} }, "k:{\"type\":\"Initialized\"}": { ".": {}, "f:lastProbeTime": {}, "f:lastTransitionTime": {}, "f:status": {}, "f:type": {} }, "k:{\"type\":\"PodScheduled\"}": { ".": {}, "f:lastProbeTime": {}, "f:lastTransitionTime": {}, "f:status": {}, "f:type": {} }, "k:{\"type\":\"Ready\"}": { ".": {}, "f:lastProbeTime": {}, "f:lastTransitionTime": {}, "f:status": {}, "f:type": {} } }, "f:containerStatuses": {}, "f:hostIP": {}, "f:phase": {}, "f:podIP": {}, "f:podIPs": { ".": {}, "k:{\"ip\":\"192.168.7.50\"}": { ".": {}, "f:ip": {} } }, "f:startTime": {} } }, "manager": "kubelet", "operation": "Update", "time": "2020-09-24T18:49:15Z" } ], "name": "kube-apiserver-master", "namespace": "kube-system", "ownerReferences": [ { "apiVersion": "v1", "controller": true, "kind": "Node", "name": "master", "uid": "77eb0e7c-51a6-45b9-89f6-8993f3d19fce" } ], "resourceVersion": "760406", "selfLink": "/api/v1/namespaces/kube-system/pods/kube-apiserver-master", "uid": "71267297-3be2-4214-be91-21ef07312f99" }, "spec": { "containers": [ { "command": [ "kube-apiserver", "--advertise-address=192.168.7.50", "--allow-privileged=true", "--authorization-mode=Node,RBAC", "--client-ca-file=/etc/kubernetes/pki/ca.crt", "--enable-admission-plugins=NodeRestriction", "--enable-bootstrap-token-auth=true", "--etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt", "--etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt", "--etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key", "--etcd-servers=https://127.0.0.1:2379", "--insecure-port=0", "--kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt", "--kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key", "--kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname", "--proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt", "--proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key", "--requestheader-allowed-names=front-proxy-client", "--requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt", "--requestheader-extra-headers-prefix=X-Remote-Extra-", "--requestheader-group-headers=X-Remote-Group", "--requestheader-username-headers=X-Remote-User", "--secure-port=6443", "--service-account-key-file=/etc/kubernetes/pki/sa.pub", "--service-cluster-ip-range=10.96.0.0/12", "--tls-cert-file=/etc/kubernetes/pki/apiserver.crt", "--tls-private-key-file=/etc/kubernetes/pki/apiserver.key" ], "image": "k8s.gcr.io/kube-apiserver:v1.18.6", "imagePullPolicy": "IfNotPresent", "livenessProbe": { "failureThreshold": 8, "httpGet": { "host": "192.168.7.50", "path": "/healthz", "port": 6443, "scheme": "HTTPS" }, "initialDelaySeconds": 15, "periodSeconds": 10, "successThreshold": 1, "timeoutSeconds": 15 }, "name": "kube-apiserver", "resources": { "requests": { "cpu": "250m" } }, "terminationMessagePath": "/dev/termination-log", "terminationMessagePolicy": "File", "volumeMounts": [ { "mountPath": "/etc/ssl/certs", "name": "ca-certs", "readOnly": true }, { "mountPath": "/etc/ca-certificates", "name": "etc-ca-certificates", "readOnly": true }, { "mountPath": "/etc/pki", "name": "etc-pki", "readOnly": true }, { "mountPath": "/etc/kubernetes/pki", "name": "k8s-certs", "readOnly": true }, { "mountPath": "/usr/local/share/ca-certificates", "name": "usr-local-share-ca-certificates", "readOnly": true }, { "mountPath": "/usr/share/ca-certificates", "name": "usr-share-ca-certificates", "readOnly": true } ] } ], "dnsPolicy": "ClusterFirst", "enableServiceLinks": true, "hostNetwork": true, "nodeName": "master", "priority": 2000000000, "priorityClassName": "system-cluster-critical", "restartPolicy": "Always", "schedulerName": "default-scheduler", "securityContext": {}, "terminationGracePeriodSeconds": 30, "tolerations": [ { "effect": "NoExecute", "operator": "Exists" } ], "volumes": [ { "hostPath": { "path": "/etc/ssl/certs", "type": "DirectoryOrCreate" }, "name": "ca-certs" }, { "hostPath": { "path": "/etc/ca-certificates", "type": "DirectoryOrCreate" }, "name": "etc-ca-certificates" }, { "hostPath": { "path": "/etc/pki", "type": "DirectoryOrCreate" }, "name": "etc-pki" }, { "hostPath": { "path": "/etc/kubernetes/pki", "type": "DirectoryOrCreate" }, "name": "k8s-certs" }, { "hostPath": { "path": "/usr/local/share/ca-certificates", "type": "DirectoryOrCreate" }, "name": "usr-local-share-ca-certificates" }, { "hostPath": { "path": "/usr/share/ca-certificates", "type": "DirectoryOrCreate" }, "name": "usr-share-ca-certificates" } ] }, "status": { "conditions": [ { "lastProbeTime": null, "lastTransitionTime": "2020-09-24T18:47:45Z", "status": "True", "type": "Initialized" }, { "lastProbeTime": null, "lastTransitionTime": "2020-09-24T18:48:27Z", "status": "True", "type": "Ready" }, { "lastProbeTime": null, "lastTransitionTime": "2020-09-24T18:48:27Z", "status": "True", "type": "ContainersReady" }, { "lastProbeTime": null, "lastTransitionTime": "2020-09-24T18:47:45Z", "status": "True", "type": "PodScheduled" } ], "containerStatuses": [ { "containerID": "docker://d329b24ac0148e006ae297ec76f31d23157ebece2eb9ab66cc3bc84de9de7ece", "image": "k8s.gcr.io/kube-apiserver:v1.18.6", "imageID": "docker-pullable://k8s.gcr.io/kube-apiserver@sha256:67c8050918d28041163d973fa7cca9a84031a58e28e4ad3eb48a7a05a285520a", "lastState": { "terminated": { "containerID": "docker://57999f0e633c69bed1c332c7c711ddf1746ad75940bd125bb7a381c18499765a", "exitCode": 255, "finishedAt": "2020-09-24T18:45:44Z", "reason": "Error", "startedAt": "2020-09-03T06:57:38Z" } }, "name": "kube-apiserver", "ready": true, "restartCount": 18, "started": true, "state": { "running": { "startedAt": "2020-09-24T18:47:46Z" } } } ], "hostIP": "192.168.7.50", "phase": "Running", "podIP": "192.168.7.50", "podIPs": [ { "ip": "192.168.7.50" } ], "qosClass": "Burstable", "startTime": "2020-09-24T18:47:45Z" } }

=================================================

kubectl get pods etcd-master -n kube-system -o json

{ "apiVersion": "v1", "kind": "Pod", "metadata": { "annotations": { "kubeadm.kubernetes.io/etcd.advertise-client-urls": "https://192.168.7.50:2379", "kubernetes.io/config.hash": "e67566359a338aa0f1499fd2f84979f6", "kubernetes.io/config.mirror": "e67566359a338aa0f1499fd2f84979f6", "kubernetes.io/config.seen": "2020-08-08T22:50:06.758517275Z", "kubernetes.io/config.source": "file" }, "creationTimestamp": "2020-08-08T22:50:07Z", "labels": { "component": "etcd", "tier": "control-plane" }, "managedFields": [ { "apiVersion": "v1", "fieldsType": "FieldsV1", "fieldsV1": { "f:metadata": { "f:annotations": { ".": {}, "f:kubeadm.kubernetes.io/etcd.advertise-client-urls": {}, "f:kubernetes.io/config.hash": {}, "f:kubernetes.io/config.mirror": {}, "f:kubernetes.io/config.seen": {}, "f:kubernetes.io/config.source": {} }, "f:labels": { ".": {}, "f:component": {}, "f:tier": {} }, "f:ownerReferences": { ".": {}, "k:{\"uid\":\"77eb0e7c-51a6-45b9-89f6-8993f3d19fce\"}": { ".": {}, "f:apiVersion": {}, "f:controller": {}, "f:kind": {}, "f:name": {}, "f:uid": {} } } }, "f:spec": { "f:containers": { "k:{\"name\":\"etcd\"}": { ".": {}, "f:command": {}, "f:image": {}, "f:imagePullPolicy": {}, "f:livenessProbe": { ".": {}, "f:failureThreshold": {}, "f:httpGet": { ".": {}, "f:host": {}, "f:path": {}, "f:port": {}, "f:scheme": {} }, "f:initialDelaySeconds": {}, "f:periodSeconds": {}, "f:successThreshold": {}, "f:timeoutSeconds": {} }, "f:name": {}, "f:resources": {}, "f:terminationMessagePath": {}, "f:terminationMessagePolicy": {}, "f:volumeMounts": { ".": {}, "k:{\"mountPath\":\"/etc/kubernetes/pki/etcd\"}": { ".": {}, "f:mountPath": {}, "f:name": {} }, "k:{\"mountPath\":\"/var/lib/etcd\"}": { ".": {}, "f:mountPath": {}, "f:name": {} } } } }, "f:dnsPolicy": {}, "f:enableServiceLinks": {}, "f:hostNetwork": {}, "f:nodeName": {}, "f:priorityClassName": {}, "f:restartPolicy": {}, "f:schedulerName": {}, "f:securityContext": {}, "f:terminationGracePeriodSeconds": {}, "f:tolerations": {}, "f:volumes": { ".": {}, "k:{\"name\":\"etcd-certs\"}": { ".": {}, "f:hostPath": { ".": {}, "f:path": {}, "f:type": {} }, "f:name": {} }, "k:{\"name\":\"etcd-data\"}": { ".": {}, "f:hostPath": { ".": {}, "f:path": {}, "f:type": {} }, "f:name": {} } } }, "f:status": { "f:conditions": { ".": {}, "k:{\"type\":\"ContainersReady\"}": { ".": {}, "f:lastProbeTime": {}, "f:lastTransitionTime": {}, "f:status": {}, "f:type": {} }, "k:{\"type\":\"Initialized\"}": { ".": {}, "f:lastProbeTime": {}, "f:lastTransitionTime": {}, "f:status": {}, "f:type": {} }, "k:{\"type\":\"PodScheduled\"}": { ".": {}, "f:lastProbeTime": {}, "f:lastTransitionTime": {}, "f:status": {}, "f:type": {} }, "k:{\"type\":\"Ready\"}": { ".": {}, "f:lastProbeTime": {}, "f:lastTransitionTime": {}, "f:status": {}, "f:type": {} } }, "f:containerStatuses": {}, "f:hostIP": {}, "f:phase": {}, "f:podIP": {}, "f:podIPs": { ".": {}, "k:{\"ip\":\"192.168.7.50\"}": { ".": {}, "f:ip": {} } }, "f:startTime": {} } }, "manager": "kubelet", "operation": "Update", "time": "2020-09-24T18:50:46Z" } ], "name": "etcd-master", "namespace": "kube-system", "ownerReferences": [ { "apiVersion": "v1", "controller": true, "kind": "Node", "name": "master", "uid": "77eb0e7c-51a6-45b9-89f6-8993f3d19fce" } ], "resourceVersion": "760979", "selfLink": "/api/v1/namespaces/kube-system/pods/etcd-master", "uid": "9ec68991-d4e0-4807-9a3d-5c86b8274f51" }, "spec": { "containers": [ { "command": [ "etcd", "--advertise-client-urls=https://192.168.7.50:2379", "--cert-file=/etc/kubernetes/pki/etcd/server.crt", "--client-cert-auth=true", "--data-dir=/var/lib/etcd", "--initial-advertise-peer-urls=https://192.168.7.50:2380", "--initial-cluster=master=https://192.168.7.50:2380", "--key-file=/etc/kubernetes/pki/etcd/server.key", "--listen-client-urls=https://127.0.0.1:2379,https://192.168.7.50:2379", "--listen-metrics-urls=http://127.0.0.1:2381", "--listen-peer-urls=https://192.168.7.50:2380", "--name=master", "--peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt", "--peer-client-cert-auth=true", "--peer-key-file=/etc/kubernetes/pki/etcd/peer.key", "--peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt", "--snapshot-count=10000", "--trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt" ], "image": "k8s.gcr.io/etcd:3.4.3-0", "imagePullPolicy": "IfNotPresent", "livenessProbe": { "failureThreshold": 8, "httpGet": { "host": "127.0.0.1", "path": "/health", "port": 2381, "scheme": "HTTP" }, "initialDelaySeconds": 15, "periodSeconds": 10, "successThreshold": 1, "timeoutSeconds": 15 }, "name": "etcd", "resources": {}, "terminationMessagePath": "/dev/termination-log", "terminationMessagePolicy": "File", "volumeMounts": [ { "mountPath": "/var/lib/etcd", "name": "etcd-data" }, { "mountPath": "/etc/kubernetes/pki/etcd", "name": "etcd-certs" } ] } ], "dnsPolicy": "ClusterFirst", "enableServiceLinks": true, "hostNetwork": true, "nodeName": "master", "priority": 2000000000, "priorityClassName": "system-cluster-critical", "restartPolicy": "Always", "schedulerName": "default-scheduler", "securityContext": {}, "terminationGracePeriodSeconds": 30, "tolerations": [ { "effect": "NoExecute", "operator": "Exists" } ], "volumes": [ { "hostPath": { "path": "/etc/kubernetes/pki/etcd", "type": "DirectoryOrCreate" }, "name": "etcd-certs" }, { "hostPath": { "path": "/var/lib/etcd", "type": "DirectoryOrCreate" }, "name": "etcd-data" } ] }, "status": { "conditions": [ { "lastProbeTime": null, "lastTransitionTime": "2020-09-24T18:47:45Z", "status": "True", "type": "Initialized" }, { "lastProbeTime": null, "lastTransitionTime": "2020-09-24T18:48:28Z", "status": "True", "type": "Ready" }, { "lastProbeTime": null, "lastTransitionTime": "2020-09-24T18:48:28Z", "status": "True", "type": "ContainersReady" }, { "lastProbeTime": null, "lastTransitionTime": "2020-09-24T18:47:45Z", "status": "True", "type": "PodScheduled" } ], "containerStatuses": [ { "containerID": "docker://969abdf672f41e2d3034be63ccb519be8e8547d95f3b992a72293e05fd9ed032", "image": "k8s.gcr.io/etcd:3.4.3-0", "imageID": "docker-pullable://k8s.gcr.io/etcd@sha256:4afb99b4690b418ffc2ceb67e1a17376457e441c1f09ab55447f0aaf992fa646", "lastState": { "terminated": { "containerID": "docker://ea98165145f6501200303af2e358ff30865c9a9591d50476e8712c4ef8e23f97", "exitCode": 255, "finishedAt": "2020-09-24T18:45:44Z", "reason": "Error", "startedAt": "2020-09-03T06:57:38Z" } }, "name": "etcd", "ready": true, "restartCount": 16, "started": true, "state": { "running": { "startedAt": "2020-09-24T18:47:46Z" } } } ], "hostIP": "192.168.7.50", "phase": "Running", "podIP": "192.168.7.50", "podIPs": [ { "ip": "192.168.7.50" } ], "qosClass": "BestEffort", "startTime": "2020-09-24T18:47:45Z" } }

[devdattakulkarni]

@mohsaman Sorry about bit of a delay in getting back on this.

I tried connections on some Pods in kube-system namespace on both minikube and GKE cluster, but I am not seeing above issue. Here are some sample outputs:

Minikube:

Devs-MacBook:kubeplus devdatta$ kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE coredns-66bff467f8-52ktl 1/1 Running 6 129d etcd-minikube 1/1 Running 5 83d kube-apiserver-minikube 1/1 Running 5 83d kube-controller-manager-minikube 1/1 Running 6 129d kube-proxy-qfzl9 1/1 Running 6 129d kube-scheduler-minikube 1/1 Running 6 129d storage-provisioner 1/1 Running 10 129d Devs-MacBook:kubeplus devdatta$ kubectl connections Pod etcd-minikube kube-system Discovering node - Level: 1, Kind:Pod, instance:etcd-minikube Discovering node - Level: 2, Kind:ServiceAccount, instance:default Discovering node - Level: 3, Kind:Pod, instance:coredns-66bff467f8-52ktl Discovering node - Level: 4, Kind:Service, instance:kube-dns Discovering node - Level: 4, Kind:ReplicaSet, instance:coredns-66bff467f8 Discovering node - Level: 5, Kind:Deployment, instance:coredns ^C Devs-MacBook:kubeplus devdatta$ kubectl connections Pod kube-scheduler kube-system Resource kube-scheduler of kind Pod in namespace kube-system does not exist. Devs-MacBook:kubeplus devdatta$ kubectl connections Pod kube-scheduler-minikube kube-system Discovering node - Level: 1, Kind:Pod, instance:kube-scheduler-minikube Discovering node - Level: 2, Kind:ServiceAccount, instance:default Discovering node - Level: 3, Kind:Pod, instance:coredns-66bff467f8-52ktl Discovering node - Level: 4, Kind:Service, instance:kube-dns Devs-MacBook:kubeplus devdatta$ kubectl connections Pod storage-provisioner kube-system Discovering node - Level: 1, Kind:Pod, instance:storage-provisioner Discovering node - Level: 2, Kind:ServiceAccount, instance:default Discovering node - Level: 3, Kind:Pod, instance:coredns-66bff467f8-52ktl Discovering node - Level: 4, Kind:Service, instance:kube-dns ^C GKE: C Devs-MacBook:kubeplus devdatta$ kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE event-exporter-v0.3.0-5cd6ccb7f7-ts9hr 2/2 Running 0 40d fluentd-gcp-scaler-bfd6cf8dd-tq9pb 1/1 Running 0 40d fluentd-gcp-v3.1.1-fcb9h 2/2 Running 0 40d heapster-gke-85795475ff-qckl2 3/3 Running 0 40d kube-dns-5c446b66bd-6dft9 4/4 Running 0 40d kube-dns-autoscaler-8687c64fc-kl5m4 1/1 Running 0 40d kube-proxy-gke-cluster-2-default-pool-b3a56a0d-3dxx 1/1 Running 0 40d l7-default-backend-8f479dd9-24dfd 1/1 Running 0 40d metrics-server-v0.3.3-fdc67d4b6-xv9r6 2/2 Running 0 40d prometheus-to-sd-zzmpz 2/2 Running 0 40d stackdriver-metadata-agent-cluster-level-66dcdc87f4-8scdp 2/2 Running 0 40d tiller-deploy-598f58dd45-pjl9q 1/1 Running 0 40d Devs-MacBook:kubeplus devdatta$ kubectl connections Pod event-exporter-v0.3.0-5cd6ccb7f7-ts9hr kube-system Discovering node - Level: 1, Kind:Pod, instance:event-exporter-v0.3.0-5cd6ccb7f7-ts9hr Discovering node - Level: 2, Kind:ServiceAccount, instance:default Discovering node - Level: 3, Kind:Pod, instance:fluentd-gcp-scaler-bfd6cf8dd-tq9pb Discovering node - Level: 4, Kind:ReplicaSet, instance:fluentd-gcp-scaler-bfd6cf8dd ^C Devs-MacBook:kubeplus devdatta$ kubectl connections Pod fluentd-gcp-scaler-bfd6cf8dd-tq9pb kube-system Discovering node - Level: 1, Kind:Pod, instance:fluentd-gcp-scaler-bfd6cf8dd-tq9pb Discovering node - Level: 2, Kind:ServiceAccount, instance:default Discovering node - Level: 3, Kind:Pod, instance:event-exporter-v0.3.0-5cd6ccb7f7-ts9hr Discovering node - Level: 4, Kind:ReplicaSet, instance:event-exporter-v0.3.0-5cd6ccb7f7 ^C Will it be possible for you to check again and confirm if you are still seeing this issue? If so, please share the exact environment details (kubernetes/minikube version and kubectl version).

Please pull latest plugins code before you try again. We have added support for PersistentVolume and ServiceAccount now.

[devdattakulkarni]

@mohsaman I was able to replicate this issue on Minikube. Turns out that Pods in the kube-system namespace are owned by "Node". KubePlus does yet support Node kind. We will put a fix for this very soon.

[devdattakulkarni]

Fixed in #644

[devdattakulkarni]

@mohsaman Thanks for reporting this issue. We have now fixed it. Please try the latest plugin and let us know if this is still a problem.

I am going to close this issue. If the problem exists, please submit a new issue.

[devdattakulkarni]

@mohsaman Hi! We are starting KubePlus community meetings next week. They will be on Tuesdays at 10.30 a.m. US Central time. The meetings will be on #kubeplus Slack channel in the CNCF workspace. I hope you are able to join us for these meetings as we look forward to building KubePlus further with community inputs and involvement.