search

kubernetes / minikube

Run Kubernetes locally
https://minikube.sigs.k8s.io/
Apache License 2.0
28.8k stars 4.82k forks source link

Error with minikube start #8561

Closed dayeol closed 3 years ago

dayeol commented 4 years ago

Steps to reproduce the issue:

Install kubectl via package manager

sudo apt-get update && sudo apt-get install -y apt-transport-https gnupg2
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubectl

Install minikube via binary option

 curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
 sudo install minikube-linux-amd64 /usr/local/bin/minikube

set proxy environment variables

export http_proxy=< my proxy >
export https_proxy=< my proxy >
export no_proxy=localhost,127.0.0.1,10.0.0.0/8,192.168.0.0/16,172.16.0.0/12
export HTTP_PROXY=< my proxy >
export HTTPS_PROXY=< my proxy >
export NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,192.168.0.0/16,172.16.0.0/12

set KUBECONFIG environment variable

export KUBECONFIG=~/.kube/config

Start minikube

minikube start --driver=docker

Full output of failed command: 

* minikube v1.11.0 on Ubuntu 18.04
  - KUBECONFIG=/home/dayeol/.kube/config
* Using the docker driver based on user configuration
* Starting control plane node minikube in cluster minikube
* Creating docker container (CPUs=2, Memory=7900MB) ...
* Found network options:
  - HTTP_PROXY=http://proxy-us.intel.com:911
  - HTTPS_PROXY=http://proxy-us.intel.com:912
  - NO_PROXY=localhost,.intel.com,127.0.0.0/8,10.0.0.0/8,::1,192.168.0.0/16,172.17.0.0/12
  - http_proxy=http://proxy-us.intel.com:911
  - https_proxy=http://proxy-us.intel.com:912
  - no_proxy=localhost,.intel.com,127.0.0.0/8,10.0.0.0/8,::1,192.168.0.0/16,172.17.0.0/12
* Preparing Kubernetes v1.18.3 on Docker 19.03.2 ...
  - env HTTP_PROXY=http://proxy-us.intel.com:911
  - env HTTPS_PROXY=http://proxy-us.intel.com:912
  - env NO_PROXY=localhost,.intel.com,127.0.0.0/8,10.0.0.0/8,::1,192.168.0.0/16,172.17.0.0/12
  - kubeadm.pod-network-cidr=10.244.0.0/16
! initialization failed, will try again: run: /bin/bash -c "sudo env PATH=/var/lib/minikube/binaries/v1.18.3:$PATH kubeadm init --config /var/tmp/minikube/kubeadm.yaml  --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,D$
rAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-contr$
ller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.18.3
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 5.3.0-1020-gke
DOCKER_VERSION: 19.03.2
DOCKER_GRAPH_DRIVER: overlay2
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [minikube localhost] and IPs [172.17.0.3 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [minikube localhost] and IPs [172.17.0.3 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 16.501294 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.18" in namespace kube-system with the configuration for the kubelets in the cluster
[kubelet-check] Initial timeout of 40s passed.

stderr:
W0625 20:48:10.973884    3191 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
        [WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.3.0-1020-gke\n", err: exit status 1
        [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
W0625 20:48:11.846166    3191 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
W0625 20:48:11.847233    3191 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
error execution phase upload-config/kubelet: Error writing Crisocket information for the control-plane node: timed out waiting for the condition
To see the stack trace of this error execute with --v=5 or higher

*
* minikube is exiting due to an error. If the above message is not useful, open an issue:
  - https://github.com/kubernetes/minikube/issues/new/choose

Full output of minikube start command used, if not already included:

Included above

Optional: Full output of minikube logs command:

* ==> Docker <== * -- Logs begin at Thu 2020-06-25 20:45:30 UTC, end at Thu 2020-06-25 21:05:51 UTC. -- * Jun 25 20:45:31 minikube dockerd[113]: time="2020-06-25T20:45:31.071234209Z" level=warning msg="Your kernel does not support cgroup rt period" * Jun 25 20:45:31 minikube dockerd[113]: time="2020-06-25T20:45:31.071241902Z" level=warning msg="Your kernel does not support cgroup rt runtime" * Jun 25 20:45:31 minikube dockerd[113]: time="2020-06-25T20:45:31.071247997Z" level=warning msg="Your kernel does not support cgroup blkio weight" * Jun 25 20:45:31 minikube dockerd[113]: time="2020-06-25T20:45:31.071257635Z" level=warning msg="Your kernel does not support cgroup blkio weight_device" * Jun 25 20:45:31 minikube dockerd[113]: time="2020-06-25T20:45:31.071398962Z" level=info msg="Loading containers: start." * Jun 25 20:45:31 minikube dockerd[113]: time="2020-06-25T20:45:31.162864718Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.18.0.0/16. Daemon option --bip can be used to set a preferred IP address" * Jun 25 20:45:31 minikube dockerd[113]: time="2020-06-25T20:45:31.190779993Z" level=info msg="Loading containers: done." * Jun 25 20:45:46 minikube dockerd[113]: time="2020-06-25T20:45:46.399574747Z" level=info msg="Docker daemon" commit=6a30dfca03 graphdriver(s)=overlay2 version=19.03.2 * Jun 25 20:45:46 minikube dockerd[113]: time="2020-06-25T20:45:46.399666404Z" level=info msg="Daemon has completed initialization" * Jun 25 20:45:46 minikube dockerd[113]: time="2020-06-25T20:45:46.651988828Z" level=info msg="API listen on /run/docker.sock" * Jun 25 20:45:46 minikube systemd[1]: Started Docker Application Container Engine. * Jun 25 20:45:47 minikube systemd[1]: docker.service: Current command vanished from the unit file, execution of the command list won't be resumed. * Jun 25 20:45:47 minikube systemd[1]: Stopping Docker Application Container Engine... * Jun 25 20:45:47 minikube dockerd[113]: time="2020-06-25T20:45:47.665799419Z" level=info msg="Processing signal 'terminated'" * Jun 25 20:45:47 minikube dockerd[113]: time="2020-06-25T20:45:47.666465972Z" level=info msg="stopping event stream following graceful shutdown" error="" module=libcontainerd namespace=moby * Jun 25 20:45:47 minikube dockerd[113]: time="2020-06-25T20:45:47.666862380Z" level=info msg="Daemon shutdown complete" * Jun 25 20:45:47 minikube dockerd[113]: time="2020-06-25T20:45:47.666886146Z" level=info msg="stopping event stream following graceful shutdown" error="context canceled" module=libcontainerd namespace=plugins.moby * Jun 25 20:45:47 minikube systemd[1]: docker.service: Succeeded. * Jun 25 20:45:47 minikube systemd[1]: Stopped Docker Application Container Engine. * Jun 25 20:45:47 minikube systemd[1]: Starting Docker Application Container Engine... * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.699560132Z" level=info msg="Starting up" * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.700801400Z" level=info msg="parsed scheme: \"unix\"" module=grpc * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.700816651Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.700829068Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 }] }" module=grpc * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.700835111Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.700894962Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc000712730, CONNECTING" module=grpc * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.700920570Z" level=info msg="blockingPicker: the picked transport is not ready, loop back to repick" module=grpc * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.701112574Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc000712730, READY" module=grpc * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.701566203Z" level=info msg="parsed scheme: \"unix\"" module=grpc * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.701579475Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.701590375Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 }] }" module=grpc * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.701597235Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.701634609Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc000712bf0, CONNECTING" module=grpc * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.701809459Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc000712bf0, READY" module=grpc * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.703691973Z" level=info msg="[graphdriver] using prior storage driver: overlay2" * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.710150325Z" level=warning msg="Your kernel does not support swap memory limit" * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.710166178Z" level=warning msg="Your kernel does not support cgroup rt period" * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.710170740Z" level=warning msg="Your kernel does not support cgroup rt runtime" * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.710174383Z" level=warning msg="Your kernel does not support cgroup blkio weight" * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.710177711Z" level=warning msg="Your kernel does not support cgroup blkio weight_device" * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.710250915Z" level=info msg="Loading containers: start." * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.771035269Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.18.0.0/16. Daemon option --bip can be used to set a preferred IP address" * Jun 25 20:45:47 minikube dockerd[351]: time="2020-06-25T20:45:47.796346423Z" level=info msg="Loading containers: done." * Jun 25 20:45:48 minikube dockerd[351]: time="2020-06-25T20:45:48.027058994Z" level=info msg="Docker daemon" commit=6a30dfca03 graphdriver(s)=overlay2 version=19.03.2 * Jun 25 20:45:48 minikube dockerd[351]: time="2020-06-25T20:45:48.027107139Z" level=info msg="Daemon has completed initialization" * Jun 25 20:45:48 minikube dockerd[351]: time="2020-06-25T20:45:48.197881039Z" level=info msg="API listen on /var/run/docker.sock" * Jun 25 20:45:48 minikube dockerd[351]: time="2020-06-25T20:45:48.197892261Z" level=info msg="API listen on [::]:2376" * Jun 25 20:45:48 minikube systemd[1]: Started Docker Application Container Engine. * Jun 25 20:48:08 minikube dockerd[351]: time="2020-06-25T20:48:08.043053950Z" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" * Jun 25 20:48:08 minikube dockerd[351]: time="2020-06-25T20:48:08.043172894Z" level=warning msg="44252030ba9ae8e4af679bd9915dd401a42a6a8d49adf54ad5e10ffaaf8b1cba cleanup: failed to unmount IPC: umount /var/lib/docker/containers/44252030ba9ae8e4af679bd9915dd401a42a6a8d49adf54ad5e10ffaaf8b1cba/mounts/shm, flags: 0x2: no such file or directory" * Jun 25 20:48:08 minikube dockerd[351]: time="2020-06-25T20:48:08.295297619Z" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" * Jun 25 20:48:08 minikube dockerd[351]: time="2020-06-25T20:48:08.295356407Z" level=warning msg="b794ff12f00a9ac90974514c1eb9c8e4954c3fcfa8b2ef1d823e442ff6b00ccb cleanup: failed to unmount IPC: umount /var/lib/docker/containers/b794ff12f00a9ac90974514c1eb9c8e4954c3fcfa8b2ef1d823e442ff6b00ccb/mounts/shm, flags: 0x2: no such file or directory" * Jun 25 20:48:08 minikube dockerd[351]: time="2020-06-25T20:48:08.659072749Z" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" * Jun 25 20:48:08 minikube dockerd[351]: time="2020-06-25T20:48:08.659153624Z" level=warning msg="6d42e479a5f6901c6648df68e1a01f15cd9309facdad24b6034066c7655c3486 cleanup: failed to unmount IPC: umount /var/lib/docker/containers/6d42e479a5f6901c6648df68e1a01f15cd9309facdad24b6034066c7655c3486/mounts/shm, flags: 0x2: no such file or directory" * Jun 25 20:48:08 minikube dockerd[351]: time="2020-06-25T20:48:08.999931431Z" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" * Jun 25 20:48:09 minikube dockerd[351]: time="2020-06-25T20:48:09.000069351Z" level=warning msg="aa4dfa7ab5cccb59bc4b3a89506e1fdec4424c4d1079eb82b93990f7d4118d31 cleanup: failed to unmount IPC: umount /var/lib/docker/containers/aa4dfa7ab5cccb59bc4b3a89506e1fdec4424c4d1079eb82b93990f7d4118d31/mounts/shm, flags: 0x2: no such file or directory" * Jun 25 20:48:09 minikube dockerd[351]: time="2020-06-25T20:48:09.365839556Z" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" * Jun 25 20:48:09 minikube dockerd[351]: time="2020-06-25T20:48:09.769742938Z" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" * Jun 25 20:48:10 minikube dockerd[351]: time="2020-06-25T20:48:10.177416106Z" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" * Jun 25 20:48:10 minikube dockerd[351]: time="2020-06-25T20:48:10.583296808Z" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete" * * ==> container status <== * CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID * 5ec0d08e6af6f 76216c34ed0c7 17 minutes ago Running kube-scheduler 0 408dbf13a7ceb * 372b75cece7ae 303ce5db0e90d 17 minutes ago Running etcd 0 fbe11b45715bc * 35dc28c2f6a7c 7e28efa976bd1 17 minutes ago Running kube-apiserver 0 1703890e5da32 * 62853ed46a8cc da26705ccb4b5 17 minutes ago Running kube-controller-manager 0 270ab5338159a * * ==> describe nodes <== * No resources found in default namespace. * * ==> dmesg <== * [ +0.000009] testing the buffer * [ +0.000010] testing the buffer * [ +10.465056] testing the buffer * [ +0.000066] testing the buffer * [ +0.000030] testing the buffer * [ +0.000031] testing the buffer * [ +0.000743] testing the buffer * [ +0.000011] testing the buffer * [ +0.000008] testing the buffer * [ +0.000009] testing the buffer * [ +0.000008] testing the buffer * [ +0.000017] testing the buffer * [Jun25 20:46] testing the buffer * [ +0.000099] testing the buffer * [Jun25 20:47] testing the buffer * [ +0.000092] testing the buffer * [Jun25 20:48] testing the buffer * [ +0.000040] testing the buffer * [ +0.000899] testing the buffer * [ +0.000015] testing the buffer * [ +0.000011] testing the buffer * [ +0.000013] testing the buffer * [ +0.000011] testing the buffer * [ +0.000024] testing the buffer * [ +0.000013] testing the buffer * [ +0.000754] testing the buffer * [Jun25 20:49] testing the buffer * [ +0.000088] testing the buffer * [Jun25 20:50] testing the buffer * [ +0.000066] testing the buffer * [Jun25 20:51] testing the buffer * [ +0.000038] testing the buffer * [Jun25 20:52] testing the buffer * [ +0.000095] testing the buffer * [Jun25 20:53] testing the buffer * [ +0.000072] testing the buffer * [Jun25 20:54] testing the buffer * [ +0.000067] testing the buffer * [Jun25 20:55] testing the buffer * [ +0.000099] testing the buffer * [Jun25 20:56] testing the buffer * [ +0.000032] testing the buffer * [Jun25 20:57] testing the buffer * [ +0.000099] testing the buffer * [Jun25 20:58] testing the buffer * [ +0.000077] testing the buffer * [Jun25 20:59] testing the buffer * [ +0.000104] testing the buffer * [Jun25 21:00] testing the buffer * [ +0.000035] testing the buffer * [Jun25 21:01] testing the buffer * [ +0.000099] testing the buffer * [Jun25 21:02] testing the buffer * [ +0.000064] testing the buffer * [Jun25 21:03] testing the buffer * [ +0.000032] testing the buffer * [Jun25 21:04] testing the buffer * [ +0.000064] testing the buffer * [Jun25 21:05] testing the buffer * [ +0.000064] testing the buffer * * ==> etcd [372b75cece7a] <== * 2020-06-25 20:48:23.185887 I | etcdmain: Go OS/Arch: linux/amd64 * 2020-06-25 20:48:23.185890 I | etcdmain: setting maximum number of CPUs to 12, total number of available CPUs is 12 * [WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead * 2020-06-25 20:48:23.185930 I | embed: peerTLS: cert = /var/lib/minikube/certs/etcd/peer.crt, key = /var/lib/minikube/certs/etcd/peer.key, trusted-ca = /var/lib/minikube/certs/etcd/ca.crt, client-cert-auth = true, crl-file = * 2020-06-25 20:48:23.186282 I | embed: name = minikube * 2020-06-25 20:48:23.186288 I | embed: data dir = /var/lib/minikube/etcd * 2020-06-25 20:48:23.186291 I | embed: member dir = /var/lib/minikube/etcd/member * 2020-06-25 20:48:23.186293 I | embed: heartbeat = 100ms * 2020-06-25 20:48:23.186295 I | embed: election = 1000ms * 2020-06-25 20:48:23.186298 I | embed: snapshot count = 10000 * 2020-06-25 20:48:23.186307 I | embed: advertise client URLs = https://172.17.0.3:2379 * 2020-06-25 20:48:23.307121 I | etcdserver: starting member b273bc7741bcb020 in cluster 86482fea2286a1d2 * raft2020/06/25 20:48:23 INFO: b273bc7741bcb020 switched to configuration voters=() * raft2020/06/25 20:48:23 INFO: b273bc7741bcb020 became follower at term 0 * raft2020/06/25 20:48:23 INFO: newRaft b273bc7741bcb020 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0] * raft2020/06/25 20:48:23 INFO: b273bc7741bcb020 became follower at term 1 * raft2020/06/25 20:48:23 INFO: b273bc7741bcb020 switched to configuration voters=(12858828581462913056) * 2020-06-25 20:48:23.333435 W | auth: simple token is not cryptographically signed * 2020-06-25 20:48:23.335923 I | etcdserver: starting server... [version: 3.4.3, cluster version: to_be_decided] * 2020-06-25 20:48:23.336052 I | etcdserver: b273bc7741bcb020 as single-node; fast-forwarding 9 ticks (election ticks 10) * raft2020/06/25 20:48:23 INFO: b273bc7741bcb020 switched to configuration voters=(12858828581462913056) * 2020-06-25 20:48:23.336309 I | etcdserver/membership: added member b273bc7741bcb020 [https://172.17.0.3:2380] to cluster 86482fea2286a1d2 * 2020-06-25 20:48:23.337281 I | embed: ClientTLS: cert = /var/lib/minikube/certs/etcd/server.crt, key = /var/lib/minikube/certs/etcd/server.key, trusted-ca = /var/lib/minikube/certs/etcd/ca.crt, client-cert-auth = true, crl-file = * 2020-06-25 20:48:23.337330 I | embed: listening for peers on 172.17.0.3:2380 * 2020-06-25 20:48:23.337370 I | embed: listening for metrics on http://127.0.0.1:2381 * raft2020/06/25 20:48:23 INFO: b273bc7741bcb020 is starting a new election at term 1 * raft2020/06/25 20:48:23 INFO: b273bc7741bcb020 became candidate at term 2 * raft2020/06/25 20:48:23 INFO: b273bc7741bcb020 received MsgVoteResp from b273bc7741bcb020 at term 2 * raft2020/06/25 20:48:23 INFO: b273bc7741bcb020 became leader at term 2 * raft2020/06/25 20:48:23 INFO: raft.node: b273bc7741bcb020 elected leader b273bc7741bcb020 at term 2 * 2020-06-25 20:48:23.407714 I | etcdserver: setting up the initial cluster version to 3.4 * 2020-06-25 20:48:23.408261 N | etcdserver/membership: set the initial cluster version to 3.4 * 2020-06-25 20:48:23.408313 I | etcdserver/api: enabled capabilities for version 3.4 * 2020-06-25 20:48:23.408324 I | embed: ready to serve client requests * 2020-06-25 20:48:23.408341 I | etcdserver: published {Name:minikube ClientURLs:[https://172.17.0.3:2379]} to cluster 86482fea2286a1d2 * 2020-06-25 20:48:23.408351 I | embed: ready to serve client requests * 2020-06-25 20:48:23.409639 I | embed: serving client requests on 127.0.0.1:2379 * 2020-06-25 20:48:23.409888 I | embed: serving client requests on 172.17.0.3:2379 * 2020-06-25 20:48:26.921177 W | etcdserver: read-only range request "key:\"/registry/clusterroles/system:node-bootstrapper\" " with result "range_response_count:0 size:4" took too long (129.385013ms) to execute * 2020-06-25 20:48:27.079430 W | etcdserver: read-only range request "key:\"/registry/clusterroles/system:kube-scheduler\" " with result "range_response_count:0 size:4" took too long (134.156777ms) to execute * 2020-06-25 20:48:27.079504 W | etcdserver: read-only range request "key:\"/registry/health\" " with result "range_response_count:0 size:4" took too long (101.895226ms) to execute * 2020-06-25 20:48:27.245759 W | etcdserver: read-only range request "key:\"/registry/clusterroles/system:controller:persistent-volume-binder\" " with result "range_response_count:0 size:4" took too long (139.742149ms) to execute * 2020-06-25 20:48:27.379339 W | etcdserver: read-only range request "key:\"/registry/clusterroles/system:controller:pvc-protection-controller\" " with result "range_response_count:0 size:4" took too long (114.344594ms) to execute * 2020-06-25 20:48:27.379375 W | etcdserver: read-only range request "key:\"/registry/health\" " with result "range_response_count:0 size:4" took too long (101.676308ms) to execute * 2020-06-25 20:48:27.546553 W | etcdserver: read-only range request "key:\"/registry/clusterrolebindings/system:controller:daemon-set-controller\" " with result "range_response_count:0 size:4" took too long (140.506842ms) to execute * 2020-06-25 20:48:27.680114 W | etcdserver: read-only range request "key:\"/registry/clusterrolebindings/system:controller:persistent-volume-binder\" " with result "range_response_count:0 size:4" took too long (115.645348ms) to execute * 2020-06-25 20:48:27.680138 W | etcdserver: read-only range request "key:\"/registry/health\" " with result "range_response_count:0 size:4" took too long (102.605524ms) to execute * 2020-06-25 20:48:27.849537 W | etcdserver: read-only range request "key:\"/registry/roles/kube-system/system:controller:cloud-provider\" " with result "range_response_count:0 size:5" took too long (141.791826ms) to execute * 2020-06-25 20:48:28.014355 W | etcdserver: read-only range request "key:\"/registry/resourcequotas/default/\" range_end:\"/registry/resourcequotas/default0\" " with result "range_response_count:0 size:5" took too long (131.229448ms) to execute * 2020-06-25 20:48:40.094515 W | etcdserver: read-only range request "key:\"/registry/serviceaccounts/kube-system/namespace-controller\" " with result "range_response_count:0 size:5" took too long (114.594173ms) to execute * 2020-06-25 20:48:43.507477 W | etcdserver: read-only range request "key:\"/registry/minions/minikube\" " with result "range_response_count:0 size:5" took too long (145.580455ms) to execute * 2020-06-25 20:49:25.602178 W | etcdserver: read-only range request "key:\"/registry/minions\" range_end:\"/registry/miniont\" count_only:true " with result "range_response_count:0 size:5" took too long (292.315061ms) to execute * 2020-06-25 20:49:25.602223 W | etcdserver: read-only range request "key:\"/registry/minions/minikube\" " with result "range_response_count:0 size:5" took too long (240.333396ms) to execute * 2020-06-25 20:49:25.602237 W | etcdserver: read-only range request "key:\"/registry/leases/kube-system/kube-controller-manager\" " with result "range_response_count:1 size:506" took too long (555.385278ms) to execute * 2020-06-25 20:58:24.058274 I | mvcc: store.index: compact 885 * 2020-06-25 20:58:24.059119 I | mvcc: finished scheduled compaction at 885 (took 620.428µs) * 2020-06-25 21:03:04.346377 W | etcdserver: read-only range request "key:\"/registry/leases/kube-system/kube-scheduler\" " with result "range_response_count:1 size:480" took too long (129.866429ms) to execute * 2020-06-25 21:03:24.061692 I | mvcc: store.index: compact 1516 * 2020-06-25 21:03:24.062437 I | mvcc: finished scheduled compaction at 1516 (took 552.22µs) * 2020-06-25 21:04:16.291020 W | etcdserver: read-only range request "key:\"/registry/leases/kube-system/kube-controller-manager\" " with result "range_response_count:1 size:507" took too long (186.548721ms) to execute * * ==> kernel <== * 21:05:51 up 7 days, 21:00, 0 users, load average: 1.14, 1.15, 1.21 * Linux minikube 5.3.0-1020-gke #22~18.04.1-Ubuntu SMP Tue May 12 07:56:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux * PRETTY_NAME="Ubuntu 19.10" * * ==> kube-apiserver [35dc28c2f6a7] <== * W0625 20:48:24.560769 1 genericapiserver.go:409] Skipping API batch/v2alpha1 because it has no resources. * W0625 20:48:24.565779 1 genericapiserver.go:409] Skipping API discovery.k8s.io/v1alpha1 because it has no resources. * W0625 20:48:24.571701 1 genericapiserver.go:409] Skipping API node.k8s.io/v1alpha1 because it has no resources. * W0625 20:48:24.580946 1 genericapiserver.go:409] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources. * W0625 20:48:24.582858 1 genericapiserver.go:409] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources. * W0625 20:48:24.590423 1 genericapiserver.go:409] Skipping API storage.k8s.io/v1alpha1 because it has no resources. * W0625 20:48:24.601527 1 genericapiserver.go:409] Skipping API apps/v1beta2 because it has no resources. * W0625 20:48:24.601537 1 genericapiserver.go:409] Skipping API apps/v1beta1 because it has no resources. * I0625 20:48:24.607474 1 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook. * I0625 20:48:24.607482 1 plugins.go:161] Loaded 10 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota. * I0625 20:48:24.608468 1 client.go:361] parsed scheme: "endpoint" * I0625 20:48:24.608481 1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379 0 }] * I0625 20:48:24.612761 1 client.go:361] parsed scheme: "endpoint" * I0625 20:48:24.612798 1 endpoint.go:68] ccResolverWrapper: sending new addresses to cc: [{https://127.0.0.1:2379 0 }] * I0625 20:48:25.761111 1 dynamic_cafile_content.go:167] Starting request-header::/var/lib/minikube/certs/front-proxy-ca.crt * I0625 20:48:25.761133 1 dynamic_cafile_content.go:167] Starting client-ca-bundle::/var/lib/minikube/certs/ca.crt * I0625 20:48:25.761243 1 dynamic_serving_content.go:130] Starting serving-cert::/var/lib/minikube/certs/apiserver.crt::/var/lib/minikube/certs/apiserver.key * I0625 20:48:25.761500 1 secure_serving.go:178] Serving securely on [::]:8443 * I0625 20:48:25.761519 1 controller.go:81] Starting OpenAPI AggregationController * I0625 20:48:25.761564 1 apiservice_controller.go:94] Starting APIServiceRegistrationController * I0625 20:48:25.761574 1 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller * I0625 20:48:25.761613 1 tlsconfig.go:240] Starting DynamicServingCertificateController * I0625 20:48:25.761638 1 crd_finalizer.go:266] Starting CRDFinalizer * I0625 20:48:25.761642 1 available_controller.go:387] Starting AvailableConditionController * I0625 20:48:25.761649 1 cache.go:32] Waiting for caches to sync for AvailableConditionController controller * I0625 20:48:25.761660 1 establishing_controller.go:76] Starting EstablishingController * I0625 20:48:25.761667 1 nonstructuralschema_controller.go:186] Starting NonStructuralSchemaConditionController * I0625 20:48:25.761672 1 controller.go:86] Starting OpenAPI controller * I0625 20:48:25.761682 1 customresource_discovery_controller.go:209] Starting DiscoveryController * I0625 20:48:25.761685 1 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController * I0625 20:48:25.761694 1 naming_controller.go:291] Starting NamingConditionController * I0625 20:48:25.761710 1 autoregister_controller.go:141] Starting autoregister controller * I0625 20:48:25.761717 1 cache.go:32] Waiting for caches to sync for autoregister controller * I0625 20:48:25.761722 1 crdregistration_controller.go:111] Starting crd-autoregister controller * I0625 20:48:25.761734 1 shared_informer.go:223] Waiting for caches to sync for crd-autoregister * I0625 20:48:25.761856 1 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller * I0625 20:48:25.761866 1 shared_informer.go:223] Waiting for caches to sync for cluster_authentication_trust_controller * I0625 20:48:25.761889 1 dynamic_cafile_content.go:167] Starting client-ca-bundle::/var/lib/minikube/certs/ca.crt * I0625 20:48:25.761912 1 dynamic_cafile_content.go:167] Starting request-header::/var/lib/minikube/certs/front-proxy-ca.crt * E0625 20:48:25.762617 1 controller.go:152] Unable to remove old endpoints from kubernetes service: StorageError: key not found, Code: 1, Key: /registry/masterleases/172.17.0.3, ResourceVersion: 0, AdditionalErrorMsg: * I0625 20:48:25.861714 1 cache.go:39] Caches are synced for APIServiceRegistrationController controller * I0625 20:48:25.861765 1 cache.go:39] Caches are synced for AvailableConditionController controller * I0625 20:48:25.861773 1 cache.go:39] Caches are synced for autoregister controller * I0625 20:48:25.861806 1 shared_informer.go:230] Caches are synced for crd-autoregister * I0625 20:48:25.861949 1 shared_informer.go:230] Caches are synced for cluster_authentication_trust_controller * I0625 20:48:26.761117 1 controller.go:130] OpenAPI AggregationController: action for item : Nothing (removed from the queue). * I0625 20:48:26.761130 1 controller.go:130] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue). * I0625 20:48:26.763995 1 storage_scheduling.go:134] created PriorityClass system-node-critical with value 2000001000 * I0625 20:48:26.765210 1 storage_scheduling.go:134] created PriorityClass system-cluster-critical with value 2000000000 * I0625 20:48:26.765219 1 storage_scheduling.go:143] all system priority classes are created successfully or already exist. * I0625 20:48:27.703408 1 controller.go:606] quota admission added evaluator for: roles.rbac.authorization.k8s.io * I0625 20:48:27.862942 1 controller.go:606] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io * W0625 20:48:28.017194 1 lease.go:224] Resetting endpoints for master service "kubernetes" to [172.17.0.3] * I0625 20:48:28.017631 1 controller.go:606] quota admission added evaluator for: endpoints * I0625 20:48:28.019039 1 controller.go:606] quota admission added evaluator for: endpointslices.discovery.k8s.io * I0625 20:48:28.768362 1 controller.go:606] quota admission added evaluator for: leases.coordination.k8s.io * I0625 20:48:29.041477 1 controller.go:606] quota admission added evaluator for: serviceaccounts * I0625 20:49:25.602538 1 trace.go:116] Trace[289263807]: "Get" url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:kube-controller-manager/v1.18.3 (linux/amd64) kubernetes/2e7996e/leader-election,client:172.17.0.3 (started: 2020-06-25 20:49:25.046681754 +0000 UTC m=+62.356444465) (total time: 555.808177ms): * Trace[289263807]: [555.751395ms] [555.745595ms] About to write a response * W0625 21:04:20.419563 1 watcher.go:199] watch chan error: etcdserver: mvcc: required revision has been compacted * * ==> kube-controller-manager [62853ed46a8c] <== * I0625 20:48:42.749090 1 controllermanager.go:533] Started "pv-protection" * I0625 20:48:42.749115 1 pv_protection_controller.go:83] Starting PV protection controller * I0625 20:48:42.749119 1 shared_informer.go:223] Waiting for caches to sync for PV protection * I0625 20:48:42.999110 1 controllermanager.go:533] Started "daemonset" * I0625 20:48:42.999123 1 daemon_controller.go:257] Starting daemon sets controller * I0625 20:48:42.999140 1 shared_informer.go:223] Waiting for caches to sync for daemon sets * I0625 20:48:43.699081 1 controllermanager.go:533] Started "horizontalpodautoscaling" * I0625 20:48:43.699114 1 horizontal.go:169] Starting HPA controller * I0625 20:48:43.699118 1 shared_informer.go:223] Waiting for caches to sync for HPA * I0625 20:48:44.099034 1 controllermanager.go:533] Started "disruption" * I0625 20:48:44.099061 1 disruption.go:331] Starting disruption controller * I0625 20:48:44.099065 1 shared_informer.go:223] Waiting for caches to sync for disruption * I0625 20:48:44.349192 1 controllermanager.go:533] Started "tokencleaner" * I0625 20:48:44.349247 1 tokencleaner.go:118] Starting token cleaner controller * I0625 20:48:44.349254 1 shared_informer.go:223] Waiting for caches to sync for token_cleaner * I0625 20:48:44.349258 1 shared_informer.go:230] Caches are synced for token_cleaner * I0625 20:48:44.499044 1 node_lifecycle_controller.go:78] Sending events to api server * E0625 20:48:44.499059 1 core.go:229] failed to start cloud node lifecycle controller: no cloud provider provided * W0625 20:48:44.499063 1 controllermanager.go:525] Skipping "cloud-node-lifecycle" * I0625 20:48:44.749752 1 controllermanager.go:533] Started "persistentvolume-binder" * I0625 20:48:44.750157 1 shared_informer.go:223] Waiting for caches to sync for resource quota * I0625 20:48:44.750180 1 pv_controller_base.go:295] Starting persistent volume controller * I0625 20:48:44.750189 1 shared_informer.go:223] Waiting for caches to sync for persistent volume * I0625 20:48:44.752541 1 shared_informer.go:223] Waiting for caches to sync for garbage collector * I0625 20:48:44.765373 1 shared_informer.go:230] Caches are synced for endpoint_slice * I0625 20:48:44.770156 1 shared_informer.go:230] Caches are synced for GC * I0625 20:48:44.779705 1 shared_informer.go:230] Caches are synced for bootstrap_signer * I0625 20:48:44.799181 1 shared_informer.go:230] Caches are synced for disruption * I0625 20:48:44.799187 1 disruption.go:339] Sending events to api server. * I0625 20:48:44.799239 1 shared_informer.go:230] Caches are synced for daemon sets * I0625 20:48:44.799240 1 shared_informer.go:230] Caches are synced for ReplicaSet * I0625 20:48:44.799238 1 shared_informer.go:230] Caches are synced for endpoint * I0625 20:48:44.799282 1 shared_informer.go:230] Caches are synced for ClusterRoleAggregator * I0625 20:48:44.799287 1 shared_informer.go:230] Caches are synced for TTL * E0625 20:48:44.808446 1 clusterroleaggregation_controller.go:181] edit failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "edit": the object has been modified; please apply your changes to the latest version and try again * I0625 20:48:44.811949 1 shared_informer.go:230] Caches are synced for deployment * I0625 20:48:44.845679 1 shared_informer.go:230] Caches are synced for certificate-csrsigning * I0625 20:48:44.847795 1 shared_informer.go:230] Caches are synced for certificate-csrapproving * I0625 20:48:44.848226 1 shared_informer.go:230] Caches are synced for node * I0625 20:48:44.848243 1 range_allocator.go:172] Starting range CIDR allocator * I0625 20:48:44.848247 1 shared_informer.go:223] Waiting for caches to sync for cidrallocator * I0625 20:48:44.848250 1 shared_informer.go:230] Caches are synced for cidrallocator * I0625 20:48:44.849232 1 shared_informer.go:230] Caches are synced for PV protection * I0625 20:48:44.849347 1 shared_informer.go:230] Caches are synced for ReplicationController * I0625 20:48:44.850735 1 shared_informer.go:230] Caches are synced for taint * I0625 20:48:44.850778 1 taint_manager.go:187] Starting NoExecuteTaintManager * I0625 20:48:44.899286 1 shared_informer.go:230] Caches are synced for HPA * I0625 20:48:44.904827 1 shared_informer.go:230] Caches are synced for namespace * I0625 20:48:44.949258 1 shared_informer.go:230] Caches are synced for service account * I0625 20:48:45.173144 1 shared_informer.go:230] Caches are synced for job * I0625 20:48:45.258533 1 shared_informer.go:230] Caches are synced for attach detach * I0625 20:48:45.262574 1 shared_informer.go:230] Caches are synced for expand * I0625 20:48:45.299435 1 shared_informer.go:230] Caches are synced for stateful set * I0625 20:48:45.349287 1 shared_informer.go:230] Caches are synced for PVC protection * I0625 20:48:45.350308 1 shared_informer.go:230] Caches are synced for persistent volume * I0625 20:48:45.352644 1 shared_informer.go:230] Caches are synced for garbage collector * I0625 20:48:45.427387 1 shared_informer.go:230] Caches are synced for resource quota * I0625 20:48:45.439996 1 shared_informer.go:230] Caches are synced for garbage collector * I0625 20:48:45.440004 1 garbagecollector.go:142] Garbage collector: all resource monitors have synced. Proceeding to collect garbage * I0625 20:48:45.450277 1 shared_informer.go:230] Caches are synced for resource quota * * ==> kube-scheduler [5ec0d08e6af6] <== * I0625 20:48:23.219285 1 registry.go:150] Registering EvenPodsSpread predicate and priority function * I0625 20:48:23.219309 1 registry.go:150] Registering EvenPodsSpread predicate and priority function * I0625 20:48:23.516186 1 serving.go:313] Generated self-signed cert in-memory * W0625 20:48:25.769553 1 authentication.go:349] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA' * W0625 20:48:25.769569 1 authentication.go:297] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system" * W0625 20:48:25.769583 1 authentication.go:298] Continuing without authentication configuration. This may treat all requests as anonymous. * W0625 20:48:25.769588 1 authentication.go:299] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false * I0625 20:48:25.798754 1 registry.go:150] Registering EvenPodsSpread predicate and priority function * I0625 20:48:25.798766 1 registry.go:150] Registering EvenPodsSpread predicate and priority function * W0625 20:48:25.799533 1 authorization.go:47] Authorization is disabled * W0625 20:48:25.799540 1 authentication.go:40] Authentication is disabled * I0625 20:48:25.799545 1 deprecated_insecure_serving.go:51] Serving healthz insecurely on [::]:10251 * I0625 20:48:25.800220 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file * I0625 20:48:25.800236 1 shared_informer.go:223] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file * I0625 20:48:25.800411 1 secure_serving.go:178] Serving securely on 127.0.0.1:10259 * I0625 20:48:25.800452 1 tlsconfig.go:240] Starting DynamicServingCertificateController * E0625 20:48:25.821433 1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope * E0625 20:48:25.821450 1 reflector.go:178] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system" * E0625 20:48:25.821822 1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope * E0625 20:48:25.821842 1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope * E0625 20:48:25.821877 1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope * E0625 20:48:25.821883 1 reflector.go:178] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:233: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope * E0625 20:48:25.822106 1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope * E0625 20:48:25.822112 1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope * E0625 20:48:25.822176 1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope * E0625 20:48:26.730496 1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope * E0625 20:48:26.763616 1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope * E0625 20:48:26.848352 1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope * E0625 20:48:26.933497 1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope * E0625 20:48:27.050452 1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope * E0625 20:48:27.210213 1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope * E0625 20:48:27.287959 1 reflector.go:178] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:233: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope * E0625 20:48:27.291912 1 reflector.go:178] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system" * I0625 20:48:29.400403 1 shared_informer.go:230] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file * I0625 20:48:29.801165 1 leaderelection.go:242] attempting to acquire leader lease kube-system/kube-scheduler... * I0625 20:48:29.805192 1 leaderelection.go:252] successfully acquired lease kube-system/kube-scheduler * * ==> kubelet <== * -- Logs begin at Thu 2020-06-25 20:45:30 UTC, end at Thu 2020-06-25 21:05:51 UTC. -- * Jun 25 21:05:45 minikube kubelet[3449]: E0625 21:05:45.980491 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:46 minikube kubelet[3449]: E0625 21:05:46.080619 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:46 minikube kubelet[3449]: E0625 21:05:46.180729 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:46 minikube kubelet[3449]: E0625 21:05:46.280867 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:46 minikube kubelet[3449]: E0625 21:05:46.381011 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:46 minikube kubelet[3449]: E0625 21:05:46.481140 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:46 minikube kubelet[3449]: E0625 21:05:46.581276 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:46 minikube kubelet[3449]: E0625 21:05:46.681394 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:46 minikube kubelet[3449]: E0625 21:05:46.781524 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:46 minikube kubelet[3449]: E0625 21:05:46.881657 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:46 minikube kubelet[3449]: E0625 21:05:46.981819 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:47 minikube kubelet[3449]: E0625 21:05:47.081903 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:47 minikube kubelet[3449]: E0625 21:05:47.182014 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:47 minikube kubelet[3449]: E0625 21:05:47.282097 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:47 minikube kubelet[3449]: E0625 21:05:47.382184 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:47 minikube kubelet[3449]: E0625 21:05:47.482365 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:47 minikube kubelet[3449]: E0625 21:05:47.582475 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:47 minikube kubelet[3449]: E0625 21:05:47.682580 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:47 minikube kubelet[3449]: E0625 21:05:47.782729 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:47 minikube kubelet[3449]: E0625 21:05:47.882894 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:47 minikube kubelet[3449]: E0625 21:05:47.983037 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:48 minikube kubelet[3449]: E0625 21:05:48.083185 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:48 minikube kubelet[3449]: E0625 21:05:48.183330 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:48 minikube kubelet[3449]: E0625 21:05:48.283456 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:48 minikube kubelet[3449]: E0625 21:05:48.383581 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:48 minikube kubelet[3449]: E0625 21:05:48.483674 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:48 minikube kubelet[3449]: E0625 21:05:48.583802 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:48 minikube kubelet[3449]: E0625 21:05:48.683916 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:48 minikube kubelet[3449]: E0625 21:05:48.784081 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:48 minikube kubelet[3449]: E0625 21:05:48.785073 3449 eviction_manager.go:255] eviction manager: failed to get summary stats: failed to get node info: node "minikube" not found * Jun 25 21:05:48 minikube kubelet[3449]: E0625 21:05:48.884207 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:48 minikube kubelet[3449]: E0625 21:05:48.984363 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:49 minikube kubelet[3449]: E0625 21:05:49.084505 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:49 minikube kubelet[3449]: E0625 21:05:49.184595 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:49 minikube kubelet[3449]: E0625 21:05:49.284756 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:49 minikube kubelet[3449]: E0625 21:05:49.384887 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:49 minikube kubelet[3449]: E0625 21:05:49.485066 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:49 minikube kubelet[3449]: E0625 21:05:49.585209 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:49 minikube kubelet[3449]: E0625 21:05:49.685352 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:49 minikube kubelet[3449]: E0625 21:05:49.785503 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:49 minikube kubelet[3449]: E0625 21:05:49.885654 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:49 minikube kubelet[3449]: E0625 21:05:49.985830 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:50 minikube kubelet[3449]: E0625 21:05:50.085954 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:50 minikube kubelet[3449]: I0625 21:05:50.090379 3449 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach * Jun 25 21:05:50 minikube kubelet[3449]: I0625 21:05:50.118540 3449 kubelet_node_status.go:70] Attempting to register node minikube * Jun 25 21:05:50 minikube kubelet[3449]: E0625 21:05:50.186085 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:50 minikube kubelet[3449]: E0625 21:05:50.286190 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:50 minikube kubelet[3449]: E0625 21:05:50.386349 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:50 minikube kubelet[3449]: E0625 21:05:50.486492 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:50 minikube kubelet[3449]: E0625 21:05:50.586649 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:50 minikube kubelet[3449]: E0625 21:05:50.686766 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:50 minikube kubelet[3449]: E0625 21:05:50.786886 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:50 minikube kubelet[3449]: E0625 21:05:50.887080 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:50 minikube kubelet[3449]: E0625 21:05:50.987245 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:51 minikube kubelet[3449]: E0625 21:05:51.087380 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:51 minikube kubelet[3449]: E0625 21:05:51.187506 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:51 minikube kubelet[3449]: E0625 21:05:51.287709 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:51 minikube kubelet[3449]: E0625 21:05:51.387799 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:51 minikube kubelet[3449]: E0625 21:05:51.487906 3449 kubelet.go:2267] node "minikube" not found * Jun 25 21:05:51 minikube kubelet[3449]: E0625 21:05:51.588058 3449 kubelet.go:2267] node "minikube" not found
dayeol commented 4 years ago

Interestingly, after I ran minikube update-context, I was able to see these outputs for minikube status and kubectl version

Does this mean minikube start was actually successful?

$ minikube status
minikube
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.4", GitCommit:"c96aede7b5205121079932896c4ad89bb93260af", GitTreeState:"clean", BuildDate:"2020-06-17T11:41:22Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.3", GitCommit:"2e7996e3e2712684bc73f0dec0200d64eec7fe40", GitTreeState:"clean", BuildDate:"2020-05-20T12:43:34Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
dayeol commented 4 years ago

Update: I downgraded minikube to 1.8.1 and it works fine.

sharifelgamal commented 4 years ago

It would seem like the minikube cluster did start up successfully but something else in our start flow failed, which cause the kubectl context to not get updated. Looking at the logs, it seems like there were permissions issues making apiserver calls.

Could you rerun minikube start --driver=docker -v=5 --alsologtostderr with 1.11 for better logging?

priyawadhwa commented 3 years ago

Hey @dayeol I'm going to close this issue as it hasn't seen any activity in some time. If you're still seeing this issue, please:

  1. Try upgrading to our latest minikube version (1.12.0) and see if that fixes it
  2. Otherwise ,provide the output of minikube start --driver=docker -v=5 --alsologtostderr
  3. Reopen this issue by including /reopen in your comment

Thank you for sharing your experience!