Openshift 4.4 Install: Workers not seen by cluster

[darkdatter]

Version

$ openshift-install version
./openshift-install 4.4.6
built from commit 99e1dc030910ccd241e5d2563f27725c0d3117b0
release image mylocal.registry.com:5000/ocp4/openshift4@sha256:7613d8f7db639147b91b16b54b24cfa351c3cbde6aa7b7bf1b9c80c260efad06

$ cat /etc/os-release
NAME="Red Hat Enterprise Linux CoreOS"
VERSION="44.81.202005250830-0"
VERSION_ID="4.4"
OPENSHIFT_VERSION="4.4"
RHEL_VERSION="8.2"
PRETTY_NAME="Red Hat Enterprise Linux CoreOS 44.81.202005250830-0 (Ootpa)"
ID="rhcos"
ID_LIKE="rhel fedora"
ANSI_COLOR="0;31"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="OpenShift Container Platform"
REDHAT_BUGZILLA_PRODUCT_VERSION="4.4"
REDHAT_SUPPORT_PRODUCT="OpenShift Container Platform"
REDHAT_SUPPORT_PRODUCT_VERSION="4.4"
OSTREE_VERSION='44.81.202005250830-0'

Platform:

UPI Restricted Network, bare metal Install

What happened?

After running the openshift-installer, my Cluster/Master nodes appear to function normally, but my Worker nodes are not even detected by the cluster. I did check to make sure that there were no pending CSRs. Here are the logs that appear on both Workers:

Jun 02 22:07:37 worker1.my.domain.com hyperkube[1362]: E0602 22:07:37.753555    1362 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 22:07:37 worker1.my.domain.com hyperkube[1362]: E0602 22:07:37.754002    1362 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 22:07:37 worker1.my.domain.com hyperkube[1362]: E0602 22:07:37.775113    1362 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 22:07:37 worker1.my.domain.com hyperkube[1362]: E0602 22:07:37.875399    1362 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 22:07:37 worker1.my.domain.com hyperkube[1362]: E0602 22:07:37.975572    1362 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 22:07:38 worker1.my.domain.com hyperkube[1362]: E0602 22:07:38.075737    1362 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 22:07:38 worker1.my.domain.com hyperkube[1362]: I0602 22:07:38.159460    1362 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 22:07:38 worker1.my.domain.com hyperkube[1362]: I0602 22:07:38.160471    1362 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 22:07:38 worker1.my.domain.com hyperkube[1362]: E0602 22:07:38.160974    1362 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 22:07:38 worker1.my.domain.com hyperkube[1362]: E0602 22:07:38.161318    1362 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 22:07:38 worker1.my.domain.com hyperkube[1362]: I0602 22:07:38.161491    1362 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 22:07:38 worker1.my.domain.com hyperkube[1362]: E0602 22:07:38.162446    1362 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 22:07:38 worker1.my.domain.com hyperkube[1362]: E0602 22:07:38.175868    1362 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 22:07:38 worker1.my.domain.com hyperkube[1362]: E0602 22:07:38.276064    1362 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 22:07:38 worker1.my.domain.com hyperkube[1362]: I0602 22:07:38.319552    1362 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 22:07:38 worker1.my.domain.com hyperkube[1362]: E0602 22:07:38.376398    1362 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 22:07:38 worker1.my.domain.com hyperkube[1362]: E0602 22:07:38.476529    1362 kubelet.go:2274] node "worker1.my.domain.com" not found

What you expected to happen?

I expected my Worker nodes to be cluster-ready after the completion of the openshift-installer.

How to reproduce it (as minimally and precisely as possible)?

• Create UPI: 3 Masters, 2 workers, 2 Load balancers, DNS & HTTP servers
• Follow steps per RH install guide for Restricted network bare metal UPI: https://docs.openshift.com/container-platform/4.4/installing/installing_bare_metal/installing-restricted-networks-bare-metal.html

Anything else we need to know?

• Restricted network, bare metal UPI install
• Registry cloned locally and running on bastion host
• RHCOS v4.4.6 on all cluster machines

References

Unsure

[abhinavdahiya]

can you attach the complete kubelet service log from the workers that are failing to join? @darkdatter

[darkdatter]

Hi @abhinavdahiya Here are the first 1k lines, the remainder of the service logs just basically repeats with the original log snippet from above:

-- Logs begin at Tue 2020-06-02 20:35:15 UTC, end at Wed 2020-06-03 15:48:14 UTC. --
Jun 02 20:37:30 worker1.my.domain.com systemd[1]: Starting Kubernetes Kubelet...
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451734    1374 flags.go:33] FLAG: --add-dir-header="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451898    1374 flags.go:33] FLAG: --address="0.0.0.0"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451903    1374 flags.go:33] FLAG: --allowed-unsafe-sysctls="[]"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451909    1374 flags.go:33] FLAG: --alsologtostderr="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451912    1374 flags.go:33] FLAG: --anonymous-auth="true"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451916    1374 flags.go:33] FLAG: --application-metrics-count-limit="100"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451920    1374 flags.go:33] FLAG: --authentication-token-webhook="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451923    1374 flags.go:33] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451927    1374 flags.go:33] FLAG: --authorization-mode="AlwaysAllow"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451932    1374 flags.go:33] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451935    1374 flags.go:33] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451938    1374 flags.go:33] FLAG: --azure-container-registry-config=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451941    1374 flags.go:33] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451945    1374 flags.go:33] FLAG: --bootstrap-checkpoint-path=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451948    1374 flags.go:33] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451951    1374 flags.go:33] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451955    1374 flags.go:33] FLAG: --cgroup-driver="cgroupfs"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451963    1374 flags.go:33] FLAG: --cgroup-root=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451966    1374 flags.go:33] FLAG: --cgroups-per-qos="true"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451970    1374 flags.go:33] FLAG: --chaos-chance="0"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451975    1374 flags.go:33] FLAG: --client-ca-file=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451978    1374 flags.go:33] FLAG: --cloud-config=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451981    1374 flags.go:33] FLAG: --cloud-provider=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451984    1374 flags.go:33] FLAG: --cluster-dns="[]"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451991    1374 flags.go:33] FLAG: --cluster-domain=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451994    1374 flags.go:33] FLAG: --cni-bin-dir="/opt/cni/bin"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.451997    1374 flags.go:33] FLAG: --cni-cache-dir="/var/lib/cni/cache"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452001    1374 flags.go:33] FLAG: --cni-conf-dir="/etc/cni/net.d"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452004    1374 flags.go:33] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452008    1374 flags.go:33] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452011    1374 flags.go:33] FLAG: --container-log-max-files="5"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452015    1374 flags.go:33] FLAG: --container-log-max-size="10Mi"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452018    1374 flags.go:33] FLAG: --container-runtime="remote"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452025    1374 flags.go:33] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452029    1374 flags.go:33] FLAG: --containerd="/run/containerd/containerd.sock"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452032    1374 flags.go:33] FLAG: --contention-profiling="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452037    1374 flags.go:33] FLAG: --cpu-cfs-quota="true"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452040    1374 flags.go:33] FLAG: --cpu-cfs-quota-period="100ms"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452043    1374 flags.go:33] FLAG: --cpu-manager-policy="none"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452046    1374 flags.go:33] FLAG: --cpu-manager-reconcile-period="10s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452049    1374 flags.go:33] FLAG: --docker="unix:///var/run/docker.sock"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452052    1374 flags.go:33] FLAG: --docker-endpoint="unix:///var/run/docker.sock"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452055    1374 flags.go:33] FLAG: --docker-env-metadata-whitelist=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452058    1374 flags.go:33] FLAG: --docker-only="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452060    1374 flags.go:33] FLAG: --docker-root="/var/lib/docker"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452063    1374 flags.go:33] FLAG: --docker-tls="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452066    1374 flags.go:33] FLAG: --docker-tls-ca="ca.pem"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452069    1374 flags.go:33] FLAG: --docker-tls-cert="cert.pem"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452072    1374 flags.go:33] FLAG: --docker-tls-key="key.pem"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452078    1374 flags.go:33] FLAG: --dynamic-config-dir=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452082    1374 flags.go:33] FLAG: --enable-cadvisor-json-endpoints="true"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452085    1374 flags.go:33] FLAG: --enable-controller-attach-detach="true"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452088    1374 flags.go:33] FLAG: --enable-debugging-handlers="true"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452091    1374 flags.go:33] FLAG: --enable-load-reader="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452093    1374 flags.go:33] FLAG: --enable-server="true"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452096    1374 flags.go:33] FLAG: --enforce-node-allocatable="[pods]"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452101    1374 flags.go:33] FLAG: --event-burst="10"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452104    1374 flags.go:33] FLAG: --event-qps="5"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452106    1374 flags.go:33] FLAG: --event-storage-age-limit="default=0"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452109    1374 flags.go:33] FLAG: --event-storage-event-limit="default=0"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452112    1374 flags.go:33] FLAG: --eviction-hard="imagefs.available<15%,memory.available<100Mi,nodefs.available<10%,nodefs.inodesFree<5%"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452123    1374 flags.go:33] FLAG: --eviction-max-pod-grace-period="0"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452126    1374 flags.go:33] FLAG: --eviction-minimum-reclaim=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452131    1374 flags.go:33] FLAG: --eviction-pressure-transition-period="5m0s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452134    1374 flags.go:33] FLAG: --eviction-soft=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452140    1374 flags.go:33] FLAG: --eviction-soft-grace-period=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452144    1374 flags.go:33] FLAG: --exit-on-lock-contention="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452147    1374 flags.go:33] FLAG: --experimental-allocatable-ignore-eviction="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452149    1374 flags.go:33] FLAG: --experimental-bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452153    1374 flags.go:33] FLAG: --experimental-check-node-capabilities-before-mount="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452156    1374 flags.go:33] FLAG: --experimental-dockershim="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452158    1374 flags.go:33] FLAG: --experimental-dockershim-root-directory="/var/lib/dockershim"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452161    1374 flags.go:33] FLAG: --experimental-kernel-memcg-notification="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452164    1374 flags.go:33] FLAG: --experimental-mounter-path=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452169    1374 flags.go:33] FLAG: --fail-swap-on="true"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452172    1374 flags.go:33] FLAG: --feature-gates=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452176    1374 flags.go:33] FLAG: --file-check-frequency="20s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452179    1374 flags.go:33] FLAG: --global-housekeeping-interval="1m0s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452182    1374 flags.go:33] FLAG: --hairpin-mode="promiscuous-bridge"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452185    1374 flags.go:33] FLAG: --healthz-bind-address="127.0.0.1"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452188    1374 flags.go:33] FLAG: --healthz-port="10248"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452195    1374 flags.go:33] FLAG: --help="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452198    1374 flags.go:33] FLAG: --hostname-override=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452201    1374 flags.go:33] FLAG: --housekeeping-interval="10s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452203    1374 flags.go:33] FLAG: --http-check-frequency="20s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452206    1374 flags.go:33] FLAG: --image-gc-high-threshold="85"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452210    1374 flags.go:33] FLAG: --image-gc-low-threshold="80"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452212    1374 flags.go:33] FLAG: --image-pull-progress-deadline="1m0s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452215    1374 flags.go:33] FLAG: --image-service-endpoint=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452218    1374 flags.go:33] FLAG: --iptables-drop-bit="15"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452221    1374 flags.go:33] FLAG: --iptables-masquerade-bit="14"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452223    1374 flags.go:33] FLAG: --keep-terminated-pod-volumes="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452226    1374 flags.go:33] FLAG: --kube-api-burst="10"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452229    1374 flags.go:33] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452232    1374 flags.go:33] FLAG: --kube-api-qps="5"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452235    1374 flags.go:33] FLAG: --kube-reserved=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452238    1374 flags.go:33] FLAG: --kube-reserved-cgroup=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452245    1374 flags.go:33] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452249    1374 flags.go:33] FLAG: --kubelet-cgroups=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452252    1374 flags.go:33] FLAG: --lock-file=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452254    1374 flags.go:33] FLAG: --log-backtrace-at=":0"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452258    1374 flags.go:33] FLAG: --log-cadvisor-usage="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452261    1374 flags.go:33] FLAG: --log-dir=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452264    1374 flags.go:33] FLAG: --log-file=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452266    1374 flags.go:33] FLAG: --log-file-max-size="1800"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452269    1374 flags.go:33] FLAG: --log-flush-frequency="5s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452273    1374 flags.go:33] FLAG: --logtostderr="true"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452276    1374 flags.go:33] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452279    1374 flags.go:33] FLAG: --make-iptables-util-chains="true"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452282    1374 flags.go:33] FLAG: --manifest-url=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452285    1374 flags.go:33] FLAG: --manifest-url-header=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452290    1374 flags.go:33] FLAG: --master-service-namespace="default"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452293    1374 flags.go:33] FLAG: --max-open-files="1000000"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452303    1374 flags.go:33] FLAG: --max-pods="110"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452306    1374 flags.go:33] FLAG: --maximum-dead-containers="-1"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452309    1374 flags.go:33] FLAG: --maximum-dead-containers-per-container="1"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452312    1374 flags.go:33] FLAG: --minimum-container-ttl-duration="6m0s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452315    1374 flags.go:33] FLAG: --minimum-image-ttl-duration="2m0s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452318    1374 flags.go:33] FLAG: --network-plugin=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452320    1374 flags.go:33] FLAG: --network-plugin-mtu="0"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452323    1374 flags.go:33] FLAG: --node-ip=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452326    1374 flags.go:33] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhcos"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452332    1374 flags.go:33] FLAG: --node-status-max-images="50"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452335    1374 flags.go:33] FLAG: --node-status-update-frequency="10s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452338    1374 flags.go:33] FLAG: --non-masquerade-cidr="10.0.0.0/8"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452341    1374 flags.go:33] FLAG: --oom-score-adj="-999"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452344    1374 flags.go:33] FLAG: --pod-cidr=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452347    1374 flags.go:33] FLAG: --pod-infra-container-image="k8s.gcr.io/pause:3.1"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452350    1374 flags.go:33] FLAG: --pod-manifest-path=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452356    1374 flags.go:33] FLAG: --pod-max-pids="-1"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452359    1374 flags.go:33] FLAG: --pods-per-core="0"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452362    1374 flags.go:33] FLAG: --port="10250"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452364    1374 flags.go:33] FLAG: --protect-kernel-defaults="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452367    1374 flags.go:33] FLAG: --provider-id=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452370    1374 flags.go:33] FLAG: --qos-reserved=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452373    1374 flags.go:33] FLAG: --read-only-port="10255"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452376    1374 flags.go:33] FLAG: --really-crash-for-testing="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452379    1374 flags.go:33] FLAG: --redirect-container-streaming="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452382    1374 flags.go:33] FLAG: --register-node="true"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452385    1374 flags.go:33] FLAG: --register-schedulable="true"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452388    1374 flags.go:33] FLAG: --register-with-taints=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452407    1374 flags.go:33] FLAG: --registry-burst="10"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452410    1374 flags.go:33] FLAG: --registry-qps="5"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452413    1374 flags.go:33] FLAG: --reserved-cpus=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452416    1374 flags.go:33] FLAG: --resolv-conf="/etc/resolv.conf"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452423    1374 flags.go:33] FLAG: --root-dir="/var/lib/kubelet"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452426    1374 flags.go:33] FLAG: --rotate-certificates="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452428    1374 flags.go:33] FLAG: --rotate-server-certificates="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452431    1374 flags.go:33] FLAG: --runonce="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452434    1374 flags.go:33] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452437    1374 flags.go:33] FLAG: --runtime-request-timeout="2m0s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452440    1374 flags.go:33] FLAG: --seccomp-profile-root="/var/lib/kubelet/seccomp"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452446    1374 flags.go:33] FLAG: --serialize-image-pulls="true"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452449    1374 flags.go:33] FLAG: --skip-headers="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452452    1374 flags.go:33] FLAG: --skip-log-headers="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452454    1374 flags.go:33] FLAG: --stderrthreshold="2"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452457    1374 flags.go:33] FLAG: --storage-driver-buffer-duration="1m0s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452460    1374 flags.go:33] FLAG: --storage-driver-db="cadvisor"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452463    1374 flags.go:33] FLAG: --storage-driver-host="localhost:8086"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452466    1374 flags.go:33] FLAG: --storage-driver-password="root"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452469    1374 flags.go:33] FLAG: --storage-driver-secure="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452475    1374 flags.go:33] FLAG: --storage-driver-table="stats"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452478    1374 flags.go:33] FLAG: --storage-driver-user="root"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452481    1374 flags.go:33] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452484    1374 flags.go:33] FLAG: --sync-frequency="1m0s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452487    1374 flags.go:33] FLAG: --system-cgroups=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452490    1374 flags.go:33] FLAG: --system-reserved=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452493    1374 flags.go:33] FLAG: --system-reserved-cgroup=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452496    1374 flags.go:33] FLAG: --tls-cert-file=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452498    1374 flags.go:33] FLAG: --tls-cipher-suites="[]"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452504    1374 flags.go:33] FLAG: --tls-min-version=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452507    1374 flags.go:33] FLAG: --tls-private-key-file=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452509    1374 flags.go:33] FLAG: --topology-manager-policy="none"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452512    1374 flags.go:33] FLAG: --v="3"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452515    1374 flags.go:33] FLAG: --version="false"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452523    1374 flags.go:33] FLAG: --vmodule=""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452526    1374 flags.go:33] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.452533    1374 flags.go:33] FLAG: --volume-stats-agg-period="1m0s"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.453053    1374 feature_gate.go:244] feature gates: &{map[]}
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.471752    1374 feature_gate.go:244] feature gates: &{map[LegacyNodeRoleBehavior:false NodeDisruptionExclusion:true RotateKubeletServerCertificate:true SCTPSupport:true ServiceNodeExclusion:true SupportPodPidsLimit:true]}
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.471817    1374 feature_gate.go:244] feature gates: &{map[LegacyNodeRoleBehavior:false NodeDisruptionExclusion:true RotateKubeletServerCertificate:true SCTPSupport:true ServiceNodeExclusion:true SupportPodPidsLimit:true]}
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.500360    1374 mount_linux.go:168] Detected OS with systemd
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.502419    1374 server.go:424] Version: v1.17.1
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.502538    1374 feature_gate.go:244] feature gates: &{map[LegacyNodeRoleBehavior:false NodeDisruptionExclusion:true RotateKubeletServerCertificate:true SCTPSupport:true ServiceNodeExclusion:true SupportPodPidsLimit:true]}
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.502632    1374 feature_gate.go:244] feature gates: &{map[LegacyNodeRoleBehavior:false NodeDisruptionExclusion:true RotateKubeletServerCertificate:true SCTPSupport:true ServiceNodeExclusion:true SupportPodPidsLimit:true]}
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.513124    1374 plugins.go:100] No cloud provider specified.
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.514122    1374 server.go:540] No cloud provider specified: "" from the config file: ""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.514148    1374 server.go:830] Client rotation is on, will bootstrap in background
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.521311    1374 bootstrap.go:100] Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.521646    1374 server.go:857] Starting client certificate rotation.
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.522472    1374 certificate_manager.go:285] Certificate rotation is enabled.
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.539565    1374 dynamic_cafile_content.go:129] Loaded a new CA Bundle and Verifier for "client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.540849    1374 certificate_manager.go:412] Rotating certificates
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.617645    1374 dynamic_cafile_content.go:167] Starting client-ca-bundle::/etc/kubernetes/kubelet-ca.crt
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.619550    1374 manager.go:146] cAdvisor running in container: "/sys/fs/cgroup/cpu,cpuacct/system.slice/kubelet.service"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.670045    1374 reflector.go:150] Starting reflector *v1beta1.CertificateSigningRequest (0s) from k8s.io/client-go/tools/watch/informerwatcher.go:146
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.670097    1374 reflector.go:185] Listing and watching *v1beta1.CertificateSigningRequest from k8s.io/client-go/tools/watch/informerwatcher.go:146
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.685115    1374 fs.go:125] Filesystem UUIDs: map[00000000-0000-4000-a000-000000000002:/dev/sda4 1ccc3ac6-b59d-46bb-9850-229cd8b4e007:/dev/dm-0 676b3306-2850-478c-a817-3f723d49377d:/dev/sda1 D802-CD71:/dev/sda2]
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.685144    1374 fs.go:126] Filesystem partitions: map[/dev/mapper/coreos-luks-root-nocrypt:{mountpoint:/var major:253 minor:0 fsType:xfs blockSize:0} /dev/sda1:{mountpoint:/boot major:8 minor:1 fsType:ext4 blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /sys/fs/cgroup:{mountpoint:/sys/fs/cgroup major:0 minor:25 fsType:tmpfs blockSize:0}]
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.686539    1374 manager.go:193] Machine: {NumCores:2 CpuFrequency:3192607 MemoryCapacity:8360095744 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:0987df58225047b1b206a5a507c29f52 SystemUUID:77751a42-f94c-c344-c1be-39b86b7ff169 BootID:3627bd68-7aad-4b48-9513-5210b63aa522 Filesystems:[{Device:/dev/sda1 DeviceMajor:8 DeviceMinor:1 Capacity:381549568 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:4180045824 Type:vfs Inodes:1020519 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:4180045824 Type:vfs Inodes:1020519 HasInodes:true} {Device:/sys/fs/cgroup DeviceMajor:0 DeviceMinor:25 Capacity:4180045824 Type:vfs Inodes:1020519 HasInodes:true} {Device:/dev/mapper/coreos-luks-root-nocrypt DeviceMajor:253 DeviceMinor:0 Capacity:68154273792 Type:vfs Inodes:33283520 HasInodes:true}] DiskMap:map[253:0:{Name:dm-0 Major:253 Minor:0 Size:68164763136 Scheduler:none} 8:0:{Name:sda Major:8 Minor:0 Size:68719476736 Scheduler:mq-deadline}] NetworkDevices:[{Name:ens192 MacAddress:00:50:56:2c:0a:68 Speed:10000 Mtu:1500}] Topology:[{Id:0 Memory:8360095744 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0] Caches:[]} {Id:1 Threads:[1] Caches:[]}] Caches:[{Size:26214400 Type:Unified Level:3}]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.686811    1374 manager.go:199] Version: {KernelVersion:4.18.0-147.8.1.el8_1.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 44.81.202005250830-0 (Ootpa) DockerVersion:Unknown DockerAPIVersion:Unknown CadvisorVersion: CadvisorRevision:}
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.725675    1374 container_manager_linux.go:265] container manager verified user specified cgroup-root exists: []
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.725703    1374 container_manager_linux.go:270] Creating Container Manager object based on Node Config: {RuntimeCgroupsName:/system.slice/crio.service SystemCgroupsName:/system.slice KubeletCgroupsName: ContainerRuntime:remote CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:systemd KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: ReservedSystemCPUs: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[cpu:{i:{value:500 scale:-3} d:{Dec:<nil>} s:500m Format:DecimalSI} ephemeral-storage:{i:{value:1073741824 scale:0} d:{Dec:<nil>} s:1Gi Format:BinarySI} memory:{i:{value:1073741824 scale:0} d:{Dec:<nil>} s:1Gi Format:BinarySI}] HardEvictionThresholds:[{Signal:memory.available Operator:LessThan Value:{Quantity:100Mi Percentage:0} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.1} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.inodesFree Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>} {Signal:imagefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.15} GracePeriod:0s MinReclaim:<nil>}]} QOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalCPUManagerReconcilePeriod:10s ExperimentalPodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms ExperimentalTopologyManagerPolicy:none}
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.725894    1374 fake_topology_manager.go:29] [fake topologymanager] NewFakeManager
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.725902    1374 container_manager_linux.go:305] Creating device plugin manager: true
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.725912    1374 manager.go:126] Creating Device Plugin manager at /var/lib/kubelet/device-plugins/kubelet.sock
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.735068    1374 fake_topology_manager.go:39] [fake topologymanager] AddHintProvider HintProvider:  &{kubelet.sock /var/lib/kubelet/device-plugins/ map[] {0 0} <nil> {{} [0 0 0]} 0x1b64c30 0x72dc9c8 0x1b65500 map[] map[] map[] map[] map[] 0xc0005bcd50 [0] 0x72dc9c8}
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.735116    1374 state_mem.go:36] [cpumanager] initializing new in-memory state store
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.736507    1374 fake_topology_manager.go:39] [fake topologymanager] AddHintProvider HintProvider:  &{{0 0} 0x72dc9c8 10000000000 0xc00097d920 <nil> <nil> <nil> <nil> map[cpu:{{500 -3} {<nil>}  DecimalSI} memory:{{1178599424 0} {<nil>}  BinarySI}] 0x72dc9c8}
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.736587    1374 server.go:1064] Using root directory: /var/lib/kubelet
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.753959    1374 kubelet.go:286] Adding pod path: /etc/kubernetes/manifests
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.772709    1374 file.go:68] Watching path "/etc/kubernetes/manifests"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.772735    1374 kubelet.go:311] Watching apiserver
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.774037    1374 reflector.go:150] Starting reflector *v1.Pod (0s) from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.774109    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.774461    1374 reflector.go:150] Starting reflector *v1.Service (0s) from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.774525    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.788800    1374 reflector.go:150] Starting reflector *v1.Node (0s) from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.788861    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: E0602 20:37:33.794174    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: E0602 20:37:33.794235    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: E0602 20:37:33.794271    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.896727    1374 remote_runtime.go:51] Connecting to runtime service /var/run/crio/crio.sock
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: W0602 20:37:33.896776    1374 util_unix.go:103] Using "/var/run/crio/crio.sock" as endpoint is deprecated, please consider using full url format "unix:///var/run/crio/crio.sock".
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.904724    1374 remote_runtime.go:59] parsed scheme: ""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.904744    1374 remote_runtime.go:59] scheme "" not registered, fallback to default scheme
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.904815    1374 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/var/run/crio/crio.sock 0  <nil>}] <nil>}
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.904836    1374 clientconn.go:577] ClientConn switching balancer to "pick_first"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.904862    1374 remote_image.go:41] Connecting to image service /var/run/crio/crio.sock
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: W0602 20:37:33.904872    1374 util_unix.go:103] Using "/var/run/crio/crio.sock" as endpoint is deprecated, please consider using full url format "unix:///var/run/crio/crio.sock".
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.904883    1374 remote_image.go:50] parsed scheme: ""
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.904888    1374 remote_image.go:50] scheme "" not registered, fallback to default scheme
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.904896    1374 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/var/run/crio/crio.sock 0  <nil>}] <nil>}
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.904900    1374 clientconn.go:577] ClientConn switching balancer to "pick_first"
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.904951    1374 balancer_conn_wrappers.go:127] pickfirstBalancer: HandleSubConnStateChange: 0xc00091c6a0, CONNECTING
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.974972    1374 balancer_conn_wrappers.go:127] pickfirstBalancer: HandleSubConnStateChange: 0xc000213ab0, CONNECTING
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.975358    1374 balancer_conn_wrappers.go:127] pickfirstBalancer: HandleSubConnStateChange: 0xc000213ab0, READY
Jun 02 20:37:33 worker1.my.domain.com hyperkube[1374]: I0602 20:37:33.975436    1374 balancer_conn_wrappers.go:127] pickfirstBalancer: HandleSubConnStateChange: 0xc00091c6a0, READY
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.453489    1374 aws_credentials.go:77] while getting AWS credentials NoCredentialProviders: no valid providers in chain. Deprecated.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]:         For verbose messaging see aws.Config.CredentialsChainVerboseErrors
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.541922    1374 kuberuntime_manager.go:211] Container runtime cri-o initialized, version: 1.17.4-12.dev.rhaos4.4.git2be4d9c.el8, apiVersion: v1alpha1
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.569814    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/aws-ebs"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.569834    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/gce-pd"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.569841    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/cinder"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.569846    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/azure-disk"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.569852    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/azure-file"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.569858    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/vsphere-volume"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.569867    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/empty-dir"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.569872    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/git-repo"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.569880    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/host-path"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.569886    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/nfs"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.569895    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/secret"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.569901    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/iscsi"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.569907    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/glusterfs"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.569915    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/rbd"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.569921    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/quobyte"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.569926    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/cephfs"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.569932    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/downward-api"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.571047    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/fc"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.571063    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/flocker"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.571070    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/configmap"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.571076    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/projected"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.571093    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/portworx-volume"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.571102    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/scaleio"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.571109    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/local-volume"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.571114    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/storageos"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.580425    1374 plugins.go:629] Loaded volume plugin "kubernetes.io/csi"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.580755    1374 server.go:1122] Started kubelet
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.593090    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'Starting' Starting kubelet.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.593205    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:34 worker1.my.domain.com systemd[1]: Started Kubernetes Kubelet.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.647697    1374 kubelet.go:1302] Image garbage collection failed once. Stats initialization may not have completed yet: failed to get imageFs info: unable to find data in memory cache
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.647845    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d4894fa8919e", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03a29c859e, ext:2770892031, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03a29c859e, ext:2770892031, loc:(*time.Location)(0x72be680)}}, Count:1, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.648512    1374 server.go:144] Starting to listen on 0.0.0.0:10250
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.663667    1374 server.go:383] Adding debug handlers to kubelet server.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.673825    1374 certificate_manager.go:285] Certificate rotation is enabled.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.674199    1374 fs_resource_analyzer.go:64] Starting FS ResourceAnalyzer
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.677436    1374 volume_manager.go:263] The desired_state_of_world populator starts
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.677452    1374 volume_manager.go:265] Starting Kubelet Volume Manager
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.677598    1374 reflector.go:150] Starting reflector *v1beta1.CSIDriver (0s) from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.677613    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.689876    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.690086    1374 desired_state_of_world_populator.go:138] Desired state populator starts to run
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.694562    1374 controller.go:135] failed to ensure node lease exists, will retry in 200ms, error: leases.coordination.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.697892    1374 kubelet.go:2194] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.698464    1374 factory.go:157] Registering CRI-O factory
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.698654    1374 factory.go:54] Registering systemd factory
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.698785    1374 factory.go:101] Registering Raw factory
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.698891    1374 manager.go:1158] Started watching for new ooms in manager
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.699842    1374 factory.go:173] Using factory "raw" for container "/"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.700107    1374 manager.go:950] Added container: "/" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.700259    1374 manager.go:272] Starting recovery of all containers
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.705321    1374 factory.go:173] Using factory "raw" for container "/system.slice/crio.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.705584    1374 manager.go:950] Added container: "/system.slice/crio.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.705838    1374 factory.go:173] Using factory "raw" for container "/system.slice/systemd-hostnamed.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.706106    1374 manager.go:950] Added container: "/system.slice/systemd-hostnamed.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.706240    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/dev-hugepages.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.706255    1374 factory.go:170] Factory "raw" can handle container "/user.slice", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.706261    1374 factory.go:173] Using factory "raw" for container "/system.slice/vgauthd.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.706476    1374 manager.go:950] Added container: "/system.slice/vgauthd.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.706748    1374 factory.go:173] Using factory "raw" for container "/system.slice/chronyd.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.707003    1374 manager.go:950] Added container: "/system.slice/chronyd.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.707213    1374 factory.go:173] Using factory "raw" for container "/system.slice/kubelet.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.707465    1374 manager.go:950] Added container: "/system.slice/kubelet.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.707996    1374 factory.go:173] Using factory "raw" for container "/system.slice/dbus.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.708204    1374 manager.go:950] Added container: "/system.slice/dbus.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.708372    1374 factory.go:173] Using factory "raw" for container "/system.slice/system-getty.slice/getty@tty1.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.708580    1374 manager.go:950] Added container: "/system.slice/system-getty.slice/getty@tty1.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.708769    1374 factory.go:173] Using factory "raw" for container "/system.slice"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.710586    1374 manager.go:950] Added container: "/system.slice" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.710741    1374 factory.go:173] Using factory "raw" for container "/system.slice/auditd.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.711387    1374 manager.go:950] Added container: "/system.slice/auditd.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.711788    1374 container.go:467] Start housekeeping for container "/system.slice"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.712254    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/boot-efi.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.712268    1374 factory.go:173] Using factory "raw" for container "/system.slice/sssd.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.712448    1374 manager.go:950] Added container: "/system.slice/sssd.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.712587    1374 factory.go:173] Using factory "raw" for container "/system.slice/system-systemd\\x2dfsck.slice"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.712759    1374 manager.go:950] Added container: "/system.slice/system-systemd\\x2dfsck.slice" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.712894    1374 factory.go:173] Using factory "raw" for container "/system.slice/system-afterburn\\x2dsshkeys.slice"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.713102    1374 manager.go:950] Added container: "/system.slice/system-afterburn\\x2dsshkeys.slice" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.713256    1374 factory.go:173] Using factory "raw" for container "/system.slice/rpc-statd.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.713441    1374 manager.go:950] Added container: "/system.slice/rpc-statd.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.713583    1374 factory.go:173] Using factory "raw" for container "/system.slice/system-getty.slice"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.713758    1374 manager.go:950] Added container: "/system.slice/system-getty.slice" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.713899    1374 factory.go:173] Using factory "raw" for container "/system.slice/NetworkManager-dispatcher.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.714073    1374 manager.go:950] Added container: "/system.slice/NetworkManager-dispatcher.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.714213    1374 factory.go:173] Using factory "raw" for container "/system.slice/systemd-udevd.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.714448    1374 manager.go:950] Added container: "/system.slice/systemd-udevd.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.714579    1374 factory.go:173] Using factory "raw" for container "/system.slice/system-sshd\\x2dkeygen.slice"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.714731    1374 manager.go:950] Added container: "/system.slice/system-sshd\\x2dkeygen.slice" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.714861    1374 factory.go:173] Using factory "raw" for container "/system.slice/NetworkManager.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715007    1374 manager.go:950] Added container: "/system.slice/NetworkManager.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715124    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/sys-kernel-debug.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715135    1374 factory.go:173] Using factory "raw" for container "/system.slice/sshd.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715233    1374 container.go:467] Start housekeeping for container "/"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715291    1374 manager.go:950] Added container: "/system.slice/sshd.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715343    1374 container.go:467] Start housekeeping for container "/system.slice/systemd-hostnamed.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715387    1374 container.go:467] Start housekeeping for container "/system.slice/vgauthd.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715445    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/var.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715458    1374 factory.go:173] Using factory "raw" for container "/system.slice/irqbalance.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715450    1374 container.go:467] Start housekeeping for container "/system.slice/chronyd.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715541    1374 container.go:467] Start housekeeping for container "/system.slice/kubelet.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715582    1374 container.go:467] Start housekeeping for container "/system.slice/dbus.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715612    1374 manager.go:950] Added container: "/system.slice/irqbalance.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715652    1374 container.go:467] Start housekeeping for container "/system.slice/rpc-statd.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715666    1374 container.go:467] Start housekeeping for container "/system.slice/system-getty.slice"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715674    1374 container.go:467] Start housekeeping for container "/system.slice/NetworkManager-dispatcher.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715682    1374 container.go:467] Start housekeeping for container "/system.slice/systemd-udevd.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715688    1374 container.go:467] Start housekeeping for container "/system.slice/system-sshd\\x2dkeygen.slice"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715696    1374 container.go:467] Start housekeeping for container "/system.slice/NetworkManager.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715292    1374 container.go:467] Start housekeeping for container "/system.slice/crio.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715628    1374 container.go:467] Start housekeeping for container "/system.slice/auditd.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715635    1374 container.go:467] Start housekeeping for container "/system.slice/sssd.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715640    1374 container.go:467] Start housekeeping for container "/system.slice/system-systemd\\x2dfsck.slice"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715645    1374 container.go:467] Start housekeeping for container "/system.slice/system-afterburn\\x2dsshkeys.slice"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.716077    1374 factory.go:173] Using factory "raw" for container "/system.slice/systemd-journald.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.720441    1374 container.go:467] Start housekeeping for container "/system.slice/irqbalance.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.720468    1374 container.go:467] Start housekeeping for container "/system.slice/system-getty.slice/getty@tty1.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.715622    1374 container.go:467] Start housekeeping for container "/system.slice/sshd.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.720651    1374 manager.go:950] Added container: "/system.slice/systemd-journald.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.720823    1374 factory.go:173] Using factory "raw" for container "/system.slice/rpcbind.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.721014    1374 manager.go:950] Added container: "/system.slice/rpcbind.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.721163    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/sys-kernel-config.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.721174    1374 factory.go:173] Using factory "raw" for container "/system.slice/systemd-logind.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.721444    1374 manager.go:950] Added container: "/system.slice/systemd-logind.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.721650    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/boot.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.721722    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/dev-mqueue.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.721768    1374 factory.go:173] Using factory "raw" for container "/system.slice/vmtoolsd.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.721975    1374 manager.go:950] Added container: "/system.slice/vmtoolsd.service" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.722198    1374 manager.go:277] Recovery completed
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.731187    1374 container.go:467] Start housekeeping for container "/system.slice/systemd-journald.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.731217    1374 container.go:467] Start housekeeping for container "/system.slice/rpcbind.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.731226    1374 container.go:467] Start housekeeping for container "/system.slice/systemd-logind.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.731233    1374 container.go:467] Start housekeeping for container "/system.slice/vmtoolsd.service"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.741681    1374 factory.go:170] Factory "raw" can handle container "/user.slice", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.741703    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/sys-kernel-debug.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.741710    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/sys-kernel-config.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.741715    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/boot.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.741719    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/boot-efi.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.741724    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/dev-mqueue.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.741729    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/dev-hugepages.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.741733    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/var.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.741920    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/boot-efi.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742098    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/boot.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742165    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/var.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742227    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/boot-efi.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742291    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/boot.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742346    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/dev-hugepages.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742406    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/dev-mqueue.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742459    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/sys-kernel-config.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742499    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/sys-kernel-debug.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742537    1374 factory.go:170] Factory "raw" can handle container "/user.slice", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742574    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/var.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742614    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/dev-hugepages.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742651    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/boot-efi.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742687    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/boot.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742722    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/dev-mqueue.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742756    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/sys-kernel-config.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742791    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/sys-kernel-debug.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742826    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/var.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742866    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/boot.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742908    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/boot-efi.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742944    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/dev-mqueue.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.742979    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/dev-hugepages.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.743017    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/sys-kernel-config.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.743057    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/sys-kernel-debug.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.743094    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/var.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.743135    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/boot-efi.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.743172    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/boot.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.743206    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/dev-hugepages.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.743248    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/dev-mqueue.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.743292    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/sys-kernel-config.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.743329    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/sys-kernel-debug.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.743368    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/var.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.743415    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/boot-efi.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.743462    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/boot.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.743499    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/dev-hugepages.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.743538    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/dev-mqueue.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.743574    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/sys-kernel-config.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.743609    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/sys-kernel-debug.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.743648    1374 factory.go:170] Factory "systemd" can handle container "/system.slice/var.mount", but ignoring.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.744766    1374 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.748367    1374 kubelet_node_status.go:486] Recording NodeHasSufficientMemory event message for node worker1.my.domain.com
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.748397    1374 kubelet_node_status.go:486] Recording NodeHasNoDiskPressure event message for node worker1.my.domain.com
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.748412    1374 kubelet_node_status.go:486] Recording NodeHasSufficientPID event message for node worker1.my.domain.com
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.748440    1374 cpu_manager.go:173] [cpumanager] starting with none policy
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.748443    1374 cpu_manager.go:174] [cpumanager] reconciling every 10s
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.749564    1374 policy_none.go:43] [cpumanager] none policy: Start
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.749838    1374 container_manager_linux.go:414] Updating kernel flag: vm/overcommit_memory, expected value: 1, actual value: 0
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.749965    1374 container_manager_linux.go:414] Updating kernel flag: kernel/panic, expected value: 10, actual value: 0
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.754488    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasSufficientMemory' Node worker1.my.domain.com status is now: NodeHasSufficientMemory
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.754514    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasNoDiskPressure' Node worker1.my.domain.com status is now: NodeHasNoDiskPressure
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.754521    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasSufficientPID' Node worker1.my.domain.com status is now: NodeHasSufficientPID
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.754760    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a77caf", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node worker1.my.domain.com status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9b70af, ext:2938593297, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9b70af, ext:2938593297, loc:(*time.Location)(0x72be680)}}, Count:1, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.756203    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a7d936", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node worker1.my.domain.com status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9bcd36, ext:2938616975, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9bcd36, ext:2938616975, loc:(*time.Location)(0x72be680)}}, Count:1, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.757006    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a7f754", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node worker1.my.domain.com status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9beb54, ext:2938624692, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9beb54, ext:2938624692, loc:(*time.Location)(0x72be680)}}, Count:1, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.759105    1374 factory.go:173] Using factory "raw" for container "/kubepods.slice"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.759359    1374 manager.go:950] Added container: "/kubepods.slice" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.759548    1374 container.go:467] Start housekeeping for container "/kubepods.slice"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.766292    1374 factory.go:173] Using factory "raw" for container "/kubepods.slice/kubepods-burstable.slice"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.766474    1374 manager.go:950] Added container: "/kubepods.slice/kubepods-burstable.slice" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.766614    1374 container.go:467] Start housekeeping for container "/kubepods.slice/kubepods-burstable.slice"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.772991    1374 factory.go:173] Using factory "raw" for container "/kubepods.slice/kubepods-besteffort.slice"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.773180    1374 manager.go:950] Added container: "/kubepods.slice/kubepods-besteffort.slice" (aliases: [], namespace: "")
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.773330    1374 container.go:467] Start housekeeping for container "/kubepods.slice/kubepods-besteffort.slice"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.776645    1374 manager.go:226] Starting Device Plugin manager
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: W0602 20:37:34.776686    1374 manager.go:577] Failed to retrieve checkpoint for "kubelet_internal_checkpoint": checkpoint is not found
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.778528    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeAllocatableEnforced' Updated Node Allocatable limit across pods
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.779161    1374 manager.go:268] Serving device plugin registration server on "/var/lib/kubelet/device-plugins/kubelet.sock"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.779971    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.780102    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d4895b70d53b", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeAllocatableEnforced", Message:"Updated Node Allocatable limit across pods", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ae64c93b, ext:2968565918, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ae64c93b, ext:2968565918, loc:(*time.Location)(0x72be680)}}, Count:1, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events is forbidden: User "system:anonymous" cannot create resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.780180    1374 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.780480    1374 container_manager_linux.go:880] Found 132 PIDs in root, 129 of them are not to be moved
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.780493    1374 container_manager_linux.go:887] Moving non-kernel processes: [949 978 1405]
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.788180    1374 plugin_watcher.go:54] Plugin Watcher Start at /var/lib/kubelet/plugins_registry
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.788240    1374 plugin_manager.go:112] The desired_state_of_world populator (plugin watcher) starts
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.788246    1374 plugin_manager.go:114] Starting Kubelet Plugin Manager
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.788290    1374 eviction_manager.go:238] eviction manager: synchronize housekeeping
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.788183    1374 kubelet_node_status.go:486] Recording NodeHasSufficientMemory event message for node worker1.my.domain.com
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.791331    1374 kubelet_node_status.go:486] Recording NodeHasNoDiskPressure event message for node worker1.my.domain.com
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.791340    1374 kubelet_node_status.go:486] Recording NodeHasSufficientPID event message for node worker1.my.domain.com
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.791363    1374 kubelet_node_status.go:70] Attempting to register node worker1.my.domain.com
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.792159    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasSufficientMemory' Node worker1.my.domain.com status is now: NodeHasSufficientMemory
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.792236    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasNoDiskPressure' Node worker1.my.domain.com status is now: NodeHasNoDiskPressure
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.792314    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasSufficientPID' Node worker1.my.domain.com status is now: NodeHasSufficientPID
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.814799    1374 container_manager_linux.go:880] Found 130 PIDs in root, 130 of them are not to be moved
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.817949    1374 eviction_manager.go:255] eviction manager: failed to get summary stats: failed to get node info: node "worker1.my.domain.com" not found
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.818137    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a77caf", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node worker1.my.domain.com status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9b70af, ext:2938593297, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03af2aa59c, ext:2981532918, loc:(*time.Location)(0x72be680)}}, Count:2, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a77caf" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.818594    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.818861    1374 kubelet_node_status.go:92] Unable to register node "worker1.my.domain.com" with API server: nodes is forbidden: User "system:anonymous" cannot create resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.819718    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a7d936", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node worker1.my.domain.com status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9bcd36, ext:2938616975, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03af2ad28c, ext:2981544421, loc:(*time.Location)(0x72be680)}}, Count:2, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a7d936" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.819785    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.819818    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.819823    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.820815    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.821064    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a7f754", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node worker1.my.domain.com status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9beb54, ext:2938624692, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03af2af0b1, ext:2981552146, loc:(*time.Location)(0x72be680)}}, Count:2, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a7f754" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.821300    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.880062    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.883189    1374 kubelet_network_linux.go:118] Using `--random-fully` in the MASQUERADE rule for iptables
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.896062    1374 controller.go:135] failed to ensure node lease exists, will retry in 400ms, error: leases.coordination.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.926516    1374 status_manager.go:157] Starting to sync pod status with apiserver
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.926539    1374 kubelet.go:1827] Starting kubelet main sync loop.
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.926569    1374 kubelet.go:1851] skipping pod synchronization - PLEG is not healthy: pleg has yet to be successful
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.926823    1374 reflector.go:150] Starting reflector *v1beta1.RuntimeClass (0s) from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: I0602 20:37:34.926841    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.939580    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:34 worker1.my.domain.com hyperkube[1374]: E0602 20:37:34.980307    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.019942    1374 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.021602    1374 kubelet_node_status.go:486] Recording NodeHasSufficientMemory event message for node worker1.my.domain.com
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.021627    1374 kubelet_node_status.go:486] Recording NodeHasNoDiskPressure event message for node worker1.my.domain.com
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.021634    1374 kubelet_node_status.go:486] Recording NodeHasSufficientPID event message for node worker1.my.domain.com
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.021651    1374 kubelet_node_status.go:70] Attempting to register node worker1.my.domain.com
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.021824    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasSufficientMemory' Node worker1.my.domain.com status is now: NodeHasSufficientMemory
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.021841    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasNoDiskPressure' Node worker1.my.domain.com status is now: NodeHasNoDiskPressure
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.021849    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasSufficientPID' Node worker1.my.domain.com status is now: NodeHasSufficientPID
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.022692    1374 kubelet_node_status.go:92] Unable to register node "worker1.my.domain.com" with API server: nodes is forbidden: User "system:anonymous" cannot create resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.022862    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a77caf", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node worker1.my.domain.com status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9b70af, ext:2938593297, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03c149e03f, ext:3211827609, loc:(*time.Location)(0x72be680)}}, Count:3, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a77caf" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.023582    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a7d936", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node worker1.my.domain.com status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9bcd36, ext:2938616975, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03c14a0ebb, ext:3211839508, loc:(*time.Location)(0x72be680)}}, Count:3, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a7d936" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.024387    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a7f754", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node worker1.my.domain.com status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9beb54, ext:2938624692, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03c14a2756, ext:3211845807, loc:(*time.Location)(0x72be680)}}, Count:3, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a7f754" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.026709    1374 kubelet.go:1913] SyncLoop (ADD, "file"): ""
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.080445    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.180610    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.280743    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.297682    1374 controller.go:135] failed to ensure node lease exists, will retry in 800ms, error: leases.coordination.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.380906    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.422821    1374 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.424428    1374 kubelet_node_status.go:486] Recording NodeHasSufficientMemory event message for node worker1.my.domain.com
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.424461    1374 kubelet_node_status.go:486] Recording NodeHasNoDiskPressure event message for node worker1.my.domain.com
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.424469    1374 kubelet_node_status.go:486] Recording NodeHasSufficientPID event message for node worker1.my.domain.com
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.424484    1374 kubelet_node_status.go:70] Attempting to register node worker1.my.domain.com
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.424682    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasSufficientMemory' Node worker1.my.domain.com status is now: NodeHasSufficientMemory
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.424720    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasNoDiskPressure' Node worker1.my.domain.com status is now: NodeHasNoDiskPressure
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.424728    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasSufficientPID' Node worker1.my.domain.com status is now: NodeHasSufficientPID
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.425731    1374 kubelet_node_status.go:92] Unable to register node "worker1.my.domain.com" with API server: nodes is forbidden: User "system:anonymous" cannot create resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.425768    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a77caf", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node worker1.my.domain.com status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9b70af, ext:2938593297, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03d94cab9c, ext:3614663926, loc:(*time.Location)(0x72be680)}}, Count:4, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a77caf" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.426555    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a7d936", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node worker1.my.domain.com status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9bcd36, ext:2938616975, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03d94cd5e7, ext:3614674752, loc:(*time.Location)(0x72be680)}}, Count:4, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a7d936" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.427374    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a7f754", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node worker1.my.domain.com status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9beb54, ext:2938624692, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03d94cee19, ext:3614680955, loc:(*time.Location)(0x72be680)}}, Count:4, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a7f754" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.481089    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.581251    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.594293    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.681415    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.690002    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.691294    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.781551    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.820147    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.821280    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.821627    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.822203    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.822346    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.823110    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.881694    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: I0602 20:37:35.939755    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.941176    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:35 worker1.my.domain.com hyperkube[1374]: E0602 20:37:35.981836    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.081990    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.099044    1374 controller.go:135] failed to ensure node lease exists, will retry in 1.6s, error: leases.coordination.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.182157    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: I0602 20:37:36.225904    1374 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: I0602 20:37:36.227410    1374 kubelet_node_status.go:486] Recording NodeHasSufficientMemory event message for node worker1.my.domain.com
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: I0602 20:37:36.227439    1374 kubelet_node_status.go:486] Recording NodeHasNoDiskPressure event message for node worker1.my.domain.com
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: I0602 20:37:36.227448    1374 kubelet_node_status.go:486] Recording NodeHasSufficientPID event message for node worker1.my.domain.com
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: I0602 20:37:36.227465    1374 kubelet_node_status.go:70] Attempting to register node worker1.my.domain.com
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: I0602 20:37:36.227647    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasSufficientMemory' Node worker1.my.domain.com status is now: NodeHasSufficientMemory
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: I0602 20:37:36.227665    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasNoDiskPressure' Node worker1.my.domain.com status is now: NodeHasNoDiskPressure
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: I0602 20:37:36.227674    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasSufficientPID' Node worker1.my.domain.com status is now: NodeHasSufficientPID
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.228674    1374 kubelet_node_status.go:92] Unable to register node "worker1.my.domain.com" with API server: nodes is forbidden: User "system:anonymous" cannot create resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.229059    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a77caf", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node worker1.my.domain.com status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9b70af, ext:2938593297, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc040d8e4b6b, ext:4417638086, loc:(*time.Location)(0x72be680)}}, Count:5, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a77caf" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.234457    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a7d936", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node worker1.my.domain.com status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9bcd36, ext:2938616975, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc040d8e8378, ext:4417652434, loc:(*time.Location)(0x72be680)}}, Count:5, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a7d936" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.236069    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a7f754", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node worker1.my.domain.com status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9beb54, ext:2938624692, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc040d8e9fe6, ext:4417659720, loc:(*time.Location)(0x72be680)}}, Count:5, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a7f754" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.282288    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.382451    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.482650    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.582830    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: I0602 20:37:36.594292    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.682966    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: I0602 20:37:36.691434    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.692645    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.783126    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: I0602 20:37:36.821757    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: I0602 20:37:36.822809    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.823035    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.823770    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: I0602 20:37:36.823887    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.824609    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.883279    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: I0602 20:37:36.941299    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.942407    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:36 worker1.my.domain.com hyperkube[1374]: E0602 20:37:36.983409    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.083540    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.183691    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.283840    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.383985    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.484133    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.584291    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: I0602 20:37:37.594422    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.684448    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: I0602 20:37:37.692813    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.694424    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.699932    1374 controller.go:135] failed to ensure node lease exists, will retry in 3.2s, error: leases.coordination.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.784628    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: I0602 20:37:37.823371    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: I0602 20:37:37.824218    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.824660    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.824997    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: I0602 20:37:37.825304    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.826044    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: I0602 20:37:37.828774    1374 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: I0602 20:37:37.830118    1374 kubelet_node_status.go:486] Recording NodeHasSufficientMemory event message for node worker1.my.domain.com
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: I0602 20:37:37.830247    1374 kubelet_node_status.go:486] Recording NodeHasNoDiskPressure event message for node worker1.my.domain.com
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: I0602 20:37:37.830261    1374 kubelet_node_status.go:486] Recording NodeHasSufficientPID event message for node worker1.my.domain.com
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: I0602 20:37:37.830278    1374 kubelet_node_status.go:70] Attempting to register node worker1.my.domain.com
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: I0602 20:37:37.830453    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasSufficientMemory' Node worker1.my.domain.com status is now: NodeHasSufficientMemory
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: I0602 20:37:37.830471    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasNoDiskPressure' Node worker1.my.domain.com status is now: NodeHasNoDiskPressure
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: I0602 20:37:37.830478    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasSufficientPID' Node worker1.my.domain.com status is now: NodeHasSufficientPID
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.831310    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a77caf", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node worker1.my.domain.com status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9b70af, ext:2938593297, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc04717c5577, ext:6020440801, loc:(*time.Location)(0x72be680)}}, Count:6, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a77caf" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.831362    1374 kubelet_node_status.go:92] Unable to register node "worker1.my.domain.com" with API server: nodes is forbidden: User "system:anonymous" cannot create resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.832136    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a7d936", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node worker1.my.domain.com status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9bcd36, ext:2938616975, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc04717cbae9, ext:6020466757, loc:(*time.Location)(0x72be680)}}, Count:6, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a7d936" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.832931    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a7f754", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node worker1.my.domain.com status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9beb54, ext:2938624692, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc04717cd6b5, ext:6020473877, loc:(*time.Location)(0x72be680)}}, Count:6, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a7f754" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.884770    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: I0602 20:37:37.942536    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.943659    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:37 worker1.my.domain.com hyperkube[1374]: E0602 20:37:37.984901    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: E0602 20:37:38.085034    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: E0602 20:37:38.185247    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: E0602 20:37:38.285425    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: E0602 20:37:38.385642    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: E0602 20:37:38.485807    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: E0602 20:37:38.586002    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: I0602 20:37:38.594408    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: E0602 20:37:38.686171    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: I0602 20:37:38.694532    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: E0602 20:37:38.695767    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: E0602 20:37:38.786453    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: I0602 20:37:38.824763    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: I0602 20:37:38.825855    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: E0602 20:37:38.826101    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: E0602 20:37:38.826658    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: I0602 20:37:38.826923    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: E0602 20:37:38.827683    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: E0602 20:37:38.886599    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: I0602 20:37:38.943810    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: E0602 20:37:38.945042    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:38 worker1.my.domain.com hyperkube[1374]: E0602 20:37:38.986744    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: E0602 20:37:39.086945    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: E0602 20:37:39.187119    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: E0602 20:37:39.287328    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: E0602 20:37:39.387588    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: E0602 20:37:39.487824    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: E0602 20:37:39.588048    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: I0602 20:37:39.594329    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: E0602 20:37:39.688219    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: I0602 20:37:39.695876    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: E0602 20:37:39.697181    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: E0602 20:37:39.788299    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: E0602 20:37:39.788809    1374 kubelet.go:2194] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: I0602 20:37:39.826213    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: I0602 20:37:39.827323    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: E0602 20:37:39.827544    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: I0602 20:37:39.828443    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: E0602 20:37:39.828493    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: E0602 20:37:39.829322    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: E0602 20:37:39.888425    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: I0602 20:37:39.945227    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: E0602 20:37:39.946624    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:39 worker1.my.domain.com hyperkube[1374]: E0602 20:37:39.988616    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: E0602 20:37:40.088764    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: E0602 20:37:40.188977    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: E0602 20:37:40.289125    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: E0602 20:37:40.389347    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: E0602 20:37:40.489581    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: I0602 20:37:40.529746    1374 manager.go:1007] Destroyed container: "/system.slice/NetworkManager-dispatcher.service" (aliases: [], namespace: "")
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: E0602 20:37:40.589733    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: I0602 20:37:40.594362    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: E0602 20:37:40.689975    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: I0602 20:37:40.697346    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: E0602 20:37:40.698969    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: E0602 20:37:40.790199    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: I0602 20:37:40.827673    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: I0602 20:37:40.828668    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: E0602 20:37:40.829169    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: E0602 20:37:40.829404    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: I0602 20:37:40.829746    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: E0602 20:37:40.830723    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: E0602 20:37:40.890535    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: E0602 20:37:40.901074    1374 controller.go:135] failed to ensure node lease exists, will retry in 6.4s, error: leases.coordination.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: I0602 20:37:40.946748    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: E0602 20:37:40.947808    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:40 worker1.my.domain.com hyperkube[1374]: E0602 20:37:40.990750    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: I0602 20:37:41.031666    1374 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: I0602 20:37:41.033569    1374 kubelet_node_status.go:486] Recording NodeHasSufficientMemory event message for node worker1.my.domain.com
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: I0602 20:37:41.033591    1374 kubelet_node_status.go:486] Recording NodeHasNoDiskPressure event message for node worker1.my.domain.com
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: I0602 20:37:41.033597    1374 kubelet_node_status.go:486] Recording NodeHasSufficientPID event message for node worker1.my.domain.com
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: I0602 20:37:41.033612    1374 kubelet_node_status.go:70] Attempting to register node worker1.my.domain.com
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: I0602 20:37:41.033781    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasSufficientMemory' Node worker1.my.domain.com status is now: NodeHasSufficientMemory
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: I0602 20:37:41.033799    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasNoDiskPressure' Node worker1.my.domain.com status is now: NodeHasNoDiskPressure
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: I0602 20:37:41.033810    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasSufficientPID' Node worker1.my.domain.com status is now: NodeHasSufficientPID
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.034989    1374 kubelet_node_status.go:92] Unable to register node "worker1.my.domain.com" with API server: nodes is forbidden: User "system:anonymous" cannot create resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.035317    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a77caf", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node worker1.my.domain.com status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9b70af, ext:2938593297, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc0542007088, ext:9223792114, loc:(*time.Location)(0x72be680)}}, Count:7, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a77caf" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.036117    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a7d936", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node worker1.my.domain.com status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9bcd36, ext:2938616975, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc0542009d1e, ext:9223803526, loc:(*time.Location)(0x72be680)}}, Count:7, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a7d936" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.037072    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a7f754", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node worker1.my.domain.com status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9beb54, ext:2938624692, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc054200b372, ext:9223809236, loc:(*time.Location)(0x72be680)}}, Count:7, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a7f754" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.090895    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.191035    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.291208    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.391379    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.491589    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.591785    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: I0602 20:37:41.594715    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.691942    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: I0602 20:37:41.699100    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.700323    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.792138    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: I0602 20:37:41.829275    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: I0602 20:37:41.830428    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.830754    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.831275    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: I0602 20:37:41.831539    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.832312    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.892311    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: I0602 20:37:41.947936    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.949158    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:41 worker1.my.domain.com hyperkube[1374]: E0602 20:37:41.992478    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: E0602 20:37:42.092710    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: E0602 20:37:42.192900    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: E0602 20:37:42.293060    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: E0602 20:37:42.393289    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: E0602 20:37:42.493460    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: E0602 20:37:42.593568    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: I0602 20:37:42.594572    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: E0602 20:37:42.693770    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: I0602 20:37:42.700442    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: E0602 20:37:42.701802    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: E0602 20:37:42.794001    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: I0602 20:37:42.830880    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: I0602 20:37:42.832006    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: E0602 20:37:42.832276    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: I0602 20:37:42.833028    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: E0602 20:37:42.833213    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: E0602 20:37:42.834028    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: E0602 20:37:42.894378    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: I0602 20:37:42.949289    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: E0602 20:37:42.950686    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:42 worker1.my.domain.com hyperkube[1374]: E0602 20:37:42.994586    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: E0602 20:37:43.094718    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: E0602 20:37:43.194944    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: E0602 20:37:43.295157    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: E0602 20:37:43.395864    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: E0602 20:37:43.496071    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: I0602 20:37:43.594415    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: E0602 20:37:43.596273    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: E0602 20:37:43.696501    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: I0602 20:37:43.701917    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: E0602 20:37:43.703178    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: E0602 20:37:43.796810    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: I0602 20:37:43.832485    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: I0602 20:37:43.833441    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: E0602 20:37:43.833848    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: E0602 20:37:43.834214    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: I0602 20:37:43.834500    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: E0602 20:37:43.835440    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: E0602 20:37:43.897051    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: I0602 20:37:43.950814    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: E0602 20:37:43.952340    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:43 worker1.my.domain.com hyperkube[1374]: E0602 20:37:43.997296    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: E0602 20:37:44.097448    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: E0602 20:37:44.197616    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: E0602 20:37:44.297773    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: E0602 20:37:44.398029    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: E0602 20:37:44.498239    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: I0602 20:37:44.594758    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: E0602 20:37:44.598404    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: E0602 20:37:44.698630    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: I0602 20:37:44.703353    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: E0602 20:37:44.706833    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: E0602 20:37:44.789340    1374 kubelet.go:2194] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: E0602 20:37:44.798778    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: I0602 20:37:44.818121    1374 eviction_manager.go:238] eviction manager: synchronize housekeeping
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: E0602 20:37:44.818193    1374 eviction_manager.go:255] eviction manager: failed to get summary stats: failed to get node info: node "worker1.my.domain.com" not found
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: I0602 20:37:44.834014    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: I0602 20:37:44.835452    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: E0602 20:37:44.835797    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: I0602 20:37:44.836160    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: E0602 20:37:44.836342    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: E0602 20:37:44.837133    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: E0602 20:37:44.898924    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: I0602 20:37:44.952567    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: E0602 20:37:44.954065    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:44 worker1.my.domain.com hyperkube[1374]: E0602 20:37:44.999064    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: E0602 20:37:45.099198    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: E0602 20:37:45.199329    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: E0602 20:37:45.299473    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: E0602 20:37:45.399637    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: E0602 20:37:45.499889    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: I0602 20:37:45.594724    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: E0602 20:37:45.600759    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: E0602 20:37:45.700881    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: I0602 20:37:45.706982    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: E0602 20:37:45.708676    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: E0602 20:37:45.801018    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: I0602 20:37:45.835958    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: I0602 20:37:45.837070    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: E0602 20:37:45.837477    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: E0602 20:37:45.838442    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: I0602 20:37:45.838468    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: E0602 20:37:45.839439    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: E0602 20:37:45.901194    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: I0602 20:37:45.954193    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:45 worker1.my.domain.com hyperkube[1374]: E0602 20:37:45.955500    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: E0602 20:37:46.001348    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: E0602 20:37:46.101489    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: E0602 20:37:46.201713    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: E0602 20:37:46.301914    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: E0602 20:37:46.402103    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: E0602 20:37:46.502315    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: I0602 20:37:46.594530    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: E0602 20:37:46.602460    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: E0602 20:37:46.702610    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: I0602 20:37:46.708798    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: E0602 20:37:46.710163    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: E0602 20:37:46.802789    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: I0602 20:37:46.837597    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: I0602 20:37:46.838662    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: E0602 20:37:46.839019    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: E0602 20:37:46.839413    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: I0602 20:37:46.839736    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: E0602 20:37:46.840462    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: E0602 20:37:46.903007    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: I0602 20:37:46.955685    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:46 worker1.my.domain.com hyperkube[1374]: E0602 20:37:46.956921    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.003191    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.103439    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.203748    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.302825    1374 controller.go:135] failed to ensure node lease exists, will retry in 7s, error: leases.coordination.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.303932    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.404208    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: I0602 20:37:47.435186    1374 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: I0602 20:37:47.436822    1374 kubelet_node_status.go:486] Recording NodeHasSufficientMemory event message for node worker1.my.domain.com
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: I0602 20:37:47.436847    1374 kubelet_node_status.go:486] Recording NodeHasNoDiskPressure event message for node worker1.my.domain.com
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: I0602 20:37:47.436853    1374 kubelet_node_status.go:486] Recording NodeHasSufficientPID event message for node worker1.my.domain.com
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: I0602 20:37:47.436871    1374 kubelet_node_status.go:70] Attempting to register node worker1.my.domain.com
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: I0602 20:37:47.437308    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasSufficientMemory' Node worker1.my.domain.com status is now: NodeHasSufficientMemory
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: I0602 20:37:47.437330    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasNoDiskPressure' Node worker1.my.domain.com status is now: NodeHasNoDiskPressure
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: I0602 20:37:47.437338    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasSufficientPID' Node worker1.my.domain.com status is now: NodeHasSufficientPID
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.438221    1374 kubelet_node_status.go:92] Unable to register node "worker1.my.domain.com" with API server: nodes is forbidden: User "system:anonymous" cannot create resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.438233    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a77caf", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node worker1.my.domain.com status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9b70af, ext:2938593297, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc06da09a28a, ext:15627047908, loc:(*time.Location)(0x72be680)}}, Count:8, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a77caf" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.439152    1374 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"worker1.my.domain.com.1614d48959a7d936", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node worker1.my.domain.com status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"worker1.my.domain.com"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc03ac9bcd36, ext:2938616975, loc:(*time.Location)(0x72be680)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfadcc06da09cf54, ext:15627059374, loc:(*time.Location)(0x72be680)}}, Count:8, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "worker1.my.domain.com.1614d48959a7d936" is forbidden: User "system:anonymous" cannot patch resource "events" in API group "" in the namespace "default"' (will not retry!)
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.504423    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: I0602 20:37:47.594947    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.604604    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.704866    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: I0602 20:37:47.710316    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.711844    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.805090    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: I0602 20:37:47.839163    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: I0602 20:37:47.840272    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.840496    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.841013    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: I0602 20:37:47.841330    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.842506    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.905321    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: I0602 20:37:47.957079    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:47 worker1.my.domain.com hyperkube[1374]: E0602 20:37:47.958337    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: E0602 20:37:48.005481    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: E0602 20:37:48.105612    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: E0602 20:37:48.205763    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: E0602 20:37:48.305898    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: E0602 20:37:48.406090    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: E0602 20:37:48.506306    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: I0602 20:37:48.594309    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: E0602 20:37:48.606521    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: E0602 20:37:48.706726    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: I0602 20:37:48.712034    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: E0602 20:37:48.713554    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: E0602 20:37:48.806913    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: I0602 20:37:48.840625    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: I0602 20:37:48.841688    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: E0602 20:37:48.841984    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: E0602 20:37:48.842596    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: I0602 20:37:48.842746    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: E0602 20:37:48.843789    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: E0602 20:37:48.907064    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: I0602 20:37:48.958471    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:48 worker1.my.domain.com hyperkube[1374]: E0602 20:37:48.959735    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: E0602 20:37:49.007213    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: E0602 20:37:49.107431    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: E0602 20:37:49.207622    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: E0602 20:37:49.307818    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: E0602 20:37:49.408035    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: E0602 20:37:49.508219    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: I0602 20:37:49.597372    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: E0602 20:37:49.608344    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: E0602 20:37:49.708461    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: I0602 20:37:49.713777    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: E0602 20:37:49.715069    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: E0602 20:37:49.790020    1374 kubelet.go:2194] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: E0602 20:37:49.808604    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: I0602 20:37:49.842140    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: I0602 20:37:49.843207    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: E0602 20:37:49.843751    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: E0602 20:37:49.844140    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: I0602 20:37:49.844235    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: E0602 20:37:49.845080    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: E0602 20:37:49.908736    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: I0602 20:37:49.959881    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:49 worker1.my.domain.com hyperkube[1374]: E0602 20:37:49.965593    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: E0602 20:37:50.008923    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: E0602 20:37:50.109058    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: E0602 20:37:50.209203    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: E0602 20:37:50.309339    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: E0602 20:37:50.409492    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: E0602 20:37:50.509651    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: I0602 20:37:50.594568    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: E0602 20:37:50.609866    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: E0602 20:37:50.710111    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: I0602 20:37:50.715237    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: E0602 20:37:50.716637    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: E0602 20:37:50.810330    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: I0602 20:37:50.843860    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: I0602 20:37:50.845193    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: E0602 20:37:50.845492    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: E0602 20:37:50.845938    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: I0602 20:37:50.846288    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: E0602 20:37:50.846952    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: E0602 20:37:50.910524    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: I0602 20:37:50.965760    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:50 worker1.my.domain.com hyperkube[1374]: E0602 20:37:50.967328    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: E0602 20:37:51.010757    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: E0602 20:37:51.110977    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: E0602 20:37:51.211143    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: E0602 20:37:51.312157    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: E0602 20:37:51.412382    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: E0602 20:37:51.512616    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: I0602 20:37:51.594677    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: E0602 20:37:51.612754    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: E0602 20:37:51.712904    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: I0602 20:37:51.716745    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: E0602 20:37:51.718107    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: E0602 20:37:51.813044    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: I0602 20:37:51.845612    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: I0602 20:37:51.846703    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: E0602 20:37:51.846998    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: E0602 20:37:51.847506    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: I0602 20:37:51.847797    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: E0602 20:37:51.848774    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: E0602 20:37:51.913275    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: I0602 20:37:51.967451    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:51 worker1.my.domain.com hyperkube[1374]: E0602 20:37:51.968749    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: E0602 20:37:52.013427    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: E0602 20:37:52.113568    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: E0602 20:37:52.213766    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: E0602 20:37:52.314026    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: E0602 20:37:52.414280    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: E0602 20:37:52.514493    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: I0602 20:37:52.594504    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: E0602 20:37:52.614716    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: E0602 20:37:52.714943    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: I0602 20:37:52.718259    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: E0602 20:37:52.719490    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: E0602 20:37:52.815186    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: I0602 20:37:52.847158    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: I0602 20:37:52.848232    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: E0602 20:37:52.848499    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: E0602 20:37:52.849007    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: I0602 20:37:52.849332    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: E0602 20:37:52.850135    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: E0602 20:37:52.915464    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: I0602 20:37:52.968886    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:52 worker1.my.domain.com hyperkube[1374]: E0602 20:37:52.970301    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: E0602 20:37:53.015614    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: E0602 20:37:53.115810    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: E0602 20:37:53.215960    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: E0602 20:37:53.316121    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: E0602 20:37:53.416350    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: E0602 20:37:53.516532    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: I0602 20:37:53.594498    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: E0602 20:37:53.616656    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: E0602 20:37:53.716851    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: I0602 20:37:53.719622    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: E0602 20:37:53.720937    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: E0602 20:37:53.817021    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: I0602 20:37:53.848647    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: I0602 20:37:53.849732    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: E0602 20:37:53.850717    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: I0602 20:37:53.850791    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: E0602 20:37:53.850917    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: E0602 20:37:53.851807    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: E0602 20:37:53.917188    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: I0602 20:37:53.970460    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:53 worker1.my.domain.com hyperkube[1374]: E0602 20:37:53.972256    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.017312    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.117609    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.217744    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.304430    1374 controller.go:135] failed to ensure node lease exists, will retry in 7s, error: leases.coordination.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.317898    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.418049    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: I0602 20:37:54.438368    1374 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: I0602 20:37:54.440031    1374 kubelet_node_status.go:486] Recording NodeHasSufficientMemory event message for node worker1.my.domain.com
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: I0602 20:37:54.440051    1374 kubelet_node_status.go:486] Recording NodeHasNoDiskPressure event message for node worker1.my.domain.com
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: I0602 20:37:54.440068    1374 kubelet_node_status.go:486] Recording NodeHasSufficientPID event message for node worker1.my.domain.com
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: I0602 20:37:54.440086    1374 kubelet_node_status.go:70] Attempting to register node worker1.my.domain.com
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: I0602 20:37:54.440530    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasSufficientMemory' Node worker1.my.domain.com status is now: NodeHasSufficientMemory
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: I0602 20:37:54.440553    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasNoDiskPressure' Node worker1.my.domain.com status is now: NodeHasNoDiskPressure
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: I0602 20:37:54.440562    1374 event.go:281] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"worker1.my.domain.com", UID:"worker1.my.domain.com", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeHasSufficientPID' Node worker1.my.domain.com status is now: NodeHasSufficientPID
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.441561    1374 kubelet_node_status.go:92] Unable to register node "worker1.my.domain.com" with API server: nodes is forbidden: User "system:anonymous" cannot create resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.518246    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: I0602 20:37:54.594692    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.618404    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.718621    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: I0602 20:37:54.721088    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.722478    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.790601    1374 kubelet.go:2194] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: I0602 20:37:54.818337    1374 eviction_manager.go:238] eviction manager: synchronize housekeeping
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.818401    1374 eviction_manager.go:255] eviction manager: failed to get summary stats: failed to get node info: node "worker1.my.domain.com" not found
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.818750    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: I0602 20:37:54.850902    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: I0602 20:37:54.851944    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.852463    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.852745    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: I0602 20:37:54.853020    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.853781    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.918900    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: I0602 20:37:54.972443    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:54 worker1.my.domain.com hyperkube[1374]: E0602 20:37:54.974279    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: E0602 20:37:55.019068    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: E0602 20:37:55.119425    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: E0602 20:37:55.219605    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: E0602 20:37:55.319790    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: E0602 20:37:55.419963    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: E0602 20:37:55.520173    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: I0602 20:37:55.594557    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: E0602 20:37:55.620353    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: E0602 20:37:55.720498    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: I0602 20:37:55.722612    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: E0602 20:37:55.724427    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: E0602 20:37:55.820712    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: I0602 20:37:55.852626    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: I0602 20:37:55.853703    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: E0602 20:37:55.854154    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: E0602 20:37:55.854557    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: I0602 20:37:55.854723    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: E0602 20:37:55.855742    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: E0602 20:37:55.920959    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: I0602 20:37:55.974419    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:55 worker1.my.domain.com hyperkube[1374]: E0602 20:37:55.975707    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: E0602 20:37:56.021176    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: E0602 20:37:56.121348    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: E0602 20:37:56.221599    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: E0602 20:37:56.321817    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: E0602 20:37:56.421970    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: E0602 20:37:56.522183    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: I0602 20:37:56.594326    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: E0602 20:37:56.622347    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: E0602 20:37:56.722549    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: I0602 20:37:56.724602    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: E0602 20:37:56.726198    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: E0602 20:37:56.822742    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: I0602 20:37:56.854334    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: I0602 20:37:56.855450    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: E0602 20:37:56.855932    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: E0602 20:37:56.856324    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: I0602 20:37:56.856517    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: E0602 20:37:56.857235    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: E0602 20:37:56.922943    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: I0602 20:37:56.975853    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:56 worker1.my.domain.com hyperkube[1374]: E0602 20:37:56.977408    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: E0602 20:37:57.023081    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: E0602 20:37:57.123220    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: E0602 20:37:57.223367    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: E0602 20:37:57.323559    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: E0602 20:37:57.423791    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: E0602 20:37:57.523946    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: I0602 20:37:57.594522    1374 csi_plugin.go:947] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: E0602 20:37:57.624132    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: E0602 20:37:57.724262    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: I0602 20:37:57.726373    1374 reflector.go:185] Listing and watching *v1beta1.CSIDriver from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: E0602 20:37:57.727895    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: E0602 20:37:57.824451    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: I0602 20:37:57.856083    1374 reflector.go:185] Listing and watching *v1.Node from k8s.io/kubernetes/pkg/kubelet/kubelet.go:458
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: I0602 20:37:57.857129    1374 reflector.go:185] Listing and watching *v1.Pod from k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: E0602 20:37:57.857854    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: nodes "worker1.my.domain.com" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: E0602 20:37:57.858309    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: I0602 20:37:57.858328    1374 reflector.go:185] Listing and watching *v1.Service from k8s.io/kubernetes/pkg/kubelet/kubelet.go:449
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: E0602 20:37:57.859163    1374 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: E0602 20:37:57.924674    1374 kubelet.go:2274] node "worker1.my.domain.com" not found
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: I0602 20:37:57.977521    1374 reflector.go:185] Listing and watching *v1beta1.RuntimeClass from k8s.io/client-go/informers/factory.go:135
Jun 02 20:37:57 worker1.my.domain.com hyperkube[1374]: E0602 20:37:57.978752    1374 reflector.go:153] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Jun 02 20:37:58 worker1.my.domain.com hyperkube[1374]: E0602 20:37:58.024821    1374 kubelet.go:2274] node "worker1.my.domain.com" not found

[abhinavdahiya]

If possible can you attach the complete log as file? @darkdatter

[darkdatter]

@abhinavdahiya Here is the kubelet-related logs for the entire day of the install. The "---log duplicate removed from here---" was put in place to de-dupelicate "node worker1... not found" entries in order to reduce file size. full_kubectl.zip

[rphillips]

We need to debug into why the kubelet is erroring with the following message:

Failed to contact API server when waiting for CSINode publishing

It's likely to be a networking, load balancer, or cert issue. I suggest starting in that order and ruling out networking/firewalls/etc first. Raising the kubelet log level may yield some additional information.

[darkdatter]

@rphillips

I can ping the api and api-int hostnames from my workers, all cluster machines are on the same network, and my masters are operational so wouldn't that mean my load balancers, firewalls, and general networking are ok?

When testing, if I run:

$ echo $KUBECONFIG

There is just blank space. And also:

$ kubectl get ns
error: no configuration has been provided

$ kubectl config view
apiVersion: v1
clusters: null
contexts: null
current-context: ""
kind: Config
preferences: {}
users: null

I do see what looks like an actual config @ /etc/kubernetes/kubelet.conf Is any of this above an issue? Can I give anything a "push" here by manually specifying my config?

[eselvam]

It seems after master install you need to remove bootstrap node then the worker node will get its config from one of the master based on your loadbalancer config. I got this details from on of the form. Please give a try and post the update

[darkdatter]

@eselvam I removed the bootstrap node per the RedHat install guide. What I did notice however is that the bootstrap machine created dozens of CSRs that seemed to come into the queue late. After approving these, more came in...then eventually the worker's CSRs. Once these were all approved I watched the CSR queue to ensure additional ones did not come in...the workers joined the cluster! Closing the issue, thank you all.

[eselvam]

could you please elaborate the CSR part? we have to sign the csr after the boot strap node removal or before.

[darkdatter]

@eselvam Sure. After the boostrapping completed, I removed the boostrap machine from the load balancer config. I then restarted my load balancer 2, then the 'master' load balancer 1 so the new configs would go into effect. I then checked the CSR queue and approved all the available CSRs. As I did this, more would come into the visible queue (e.g. the worker CSRs). Once no new CSRs were visible, I checked 'oc get nodes' and my workers were then in the cluster and ready.

[eselvam]

Thanks. Quick help. Is the procedure same for OKD as well with FCOS? Kindly share details.

[wayne-cec]

same problem with OKD 4.5. After approved all CSRs, this problem was resolved.

[LeslieKuo]

Hit the same error with OCP 4.3.

curvet-inf:~ # oc version
Client Version: 4.3.13
Server Version: 4.3.13
Kubernetes Version: v1.16.2
curvet-inf:~ # cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.6 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.6"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.6 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.6:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.6
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.6"

Approve all the csr and wait a minutes the worker nodes can be seen now.

# oc get csr -o name | xargs oc adm certificate approve

[yogeesh-venkanna]

Hi @darkdatter,

I see this issue is more than an year old. I still have this issue while trying to install the cluster at openshift V 4.8.9. Using bigip load balancers.

Master installation was successful but workers are unable to join the cluster.

I see the below errors: Jan 10 16:34:19 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:19.743656 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:19 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:19.844441 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:19 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:19.945150 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:20 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:20.045848 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:20 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:20.146788 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:20 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:20.247557 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:20 yv01-ops01.saastb01.okd hyperkube[1879]: I0110 16:34:20.295316 1879 csi_plugin.go:1031] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "yv01-ops01.saastb01.okd" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jan 10 16:34:20 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:20.347781 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:20 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:20.448374 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:20 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:20.549158 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:20 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:20.650047 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:20 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:20.750771 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:20 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:20.851624 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:20 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:20.952695 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:21 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:21.053055 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:21 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:21.053665 1879 kubelet.go:2223] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Jan 10 16:34:21 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:21.153779 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:21 yv01-ops01.saastb01.okd hyperkube[1879]: I0110 16:34:21.212878 1879 csi_plugin.go:1031] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "yv01-ops01.saastb01.okd" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jan 10 16:34:21 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:21.254016 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:21 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:21.354372 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:21 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:21.455245 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:21 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:21.555946 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:21 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:21.656820 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:21 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:21.757464 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:21 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:21.858431 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:21 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:21.959138 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:22 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:22.060104 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:22 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:22.160897 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:22 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:22.261612 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:22 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:22.362314 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found" Jan 10 16:34:22 yv01-ops01.saastb01.okd hyperkube[1879]: E0110 16:34:22.463181 1879 kubelet.go:2303] "Error getting node" err="node \"yv01-ops01.saastb01.okd\" not found"

Can you let me know if I am missing any configuration on LB?

Regards Y V

[staebler]

@yogeesh-venkanna If you are having problems that are not addressed by the resolution described in this issue, then please open a new issue rather than trying to resurrect this issue.