search

kubernetes / minikube

Run Kubernetes locally
https://minikube.sigs.k8s.io/
Apache License 2.0
29.36k stars 4.88k forks source link

minikube start hangs on Starting cluster coponents #2886

Closed bfalor closed 6 years ago

bfalor commented 6 years ago

BUG REPORT

Environment: macOS Sierra 10.12.6

Minikube version: v0.27.0

What happened:minikube got stuck on Starting cluster components

What you expected to happen:minikube to start

How to reproduce it (as minimally and precisely as possible): minikube start

Anything else do we need to know: minikube starts if I specify --bootstrapper=localkube but then minikube console doesn't point to the proper end point, and minikube status says cluster is stopped. This works perfectly on my coworkers machine who followed the exact same steps to install and run minikube.

akjamie commented 6 years ago

got same issue, i tried on windows 10 & macos, it hangs for more than 8 hours, i hadto stop it manually

adelowo commented 6 years ago

I ran into this and had to use localkube as the bootstrapper instead ( https://github.com/kubernetes/minikube/issues/2765#issuecomment-386773015 ) 

minikube start --bootstrapper=localkube

I am running High Sierra (10.13.2) but shouldn't matter since this seemed to fix the issue for a lot of people

bfalor commented 6 years ago

minikube start --bootstrapper=localkube doesn't start a cluster

bfalor commented 6 years ago

minikube start --vm-driver=hyperkit --bootstrapper=localkube Starting local Kubernetes v1.10.0 cluster... Starting VM... Downloading Minikube ISO 150.53 MB / 150.53 MB [============================================] 100.00% 0s Getting VM IP address... WARNING: The localkube bootstrapper is now deprecated and support for it will be removed in a future release. Please consider switching to the kubeadm bootstrapper, which is intended to replace the localkube bootstrapper. To disable this message, run [minikube config set ShowBootstrapperDeprecationNotification false] Moving files into cluster... Downloading localkube binary 173.54 MB / 173.54 MB [============================================] 100.00% 0s 0 B / 65 B [----------------------------------------------------------] 0.00% 65 B / 65 B [======================================================] 100.00% 0sSetting up certs... Connecting to cluster... Setting up kubeconfig... Starting cluster components... Kubectl is now configured to use the cluster. Loading cached images from config file. minikube status minikube: Running cluster: Stopped kubectl: Correctly Configured: pointing to minikube-vm at 192.168.64.10

greglanthier commented 6 years ago

Is it possible the kubeadm bootstrapper is having problems with k8s v1.10. on nix platforms?

I ask this because I've been seeing the same symptoms on some test machines I've be working with for the past couple of days and when I downgraded to K8s v1.9.6 (i.e.: minikube --kubernetes-version v1.9.6 ...) the minikube cluster started up without complaint.

@bfalor: are you able to try starting minikube with the --kubernetes-version v1.9.6 or similar? If you get a working Minikube cluster using v1.9.x but can't with v1.10.x that might be a useful data point in tracking down the problem.

On my end I haven't pinned down a specific suspicious line in a log file pointing to a likely cause but I'm happy to supply logs/telemetry. I'd need pointers to which are the most interesting.

At first glance it looks an awful lot like the api server isn't starting up for some reason. The root cause is likely something simple or obvious - but I haven't stumbled onto it yet.

[root@test-vault ~]# journalctl -fu kubelet
-- Logs begin at Tue 2018-06-12 18:32:14 PDT. --
Jun 12 18:43:47 test-vault kubelet[26436]: Flag --cadvisor-port has been deprecated, The default will change to 0 (disabled) in 1.12, and the cadvisor port will be removed entirely in 1.13
Jun 12 18:43:47 test-vault kubelet[26436]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 12 18:43:47 test-vault kubelet[26436]: Flag --pod-manifest-path has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 12 18:43:47 test-vault kubelet[26436]: Flag --authorization-mode has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 12 18:43:47 test-vault kubelet[26436]: Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 12 18:43:47 test-vault kubelet[26436]: I0612 18:43:47.309017   26436 feature_gate.go:226] feature gates: &{{} map[]}
Jun 12 18:43:48 test-vault kubelet[26436]: F0612 18:43:48.351313   26436 server.go:218] unable to load client CA file /var/lib/localkube/certs/ca.crt: open /var/lib/localkube/certs/ca.crt: no such file or directory
Jun 12 18:43:48 test-vault systemd[1]: kubelet.service: main process exited, code=exited, status=255/n/a
Jun 12 18:43:48 test-vault systemd[1]: Unit kubelet.service entered failed state.
Jun 12 18:43:48 test-vault systemd[1]: kubelet.service failed.
Jun 12 18:43:58 test-vault systemd[1]: kubelet.service holdoff time over, scheduling restart.
Jun 12 18:43:58 test-vault systemd[1]: Started kubelet: The Kubernetes Node Agent.
Jun 12 18:43:58 test-vault systemd[1]: Starting kubelet: The Kubernetes Node Agent...
Jun 12 18:43:58 test-vault kubelet[26517]: Flag --allow-privileged has been deprecated, will be removed in a future version
Jun 12 18:43:58 test-vault kubelet[26517]: Flag --cluster-dns has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 12 18:43:58 test-vault kubelet[26517]: Flag --cluster-domain has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 12 18:43:58 test-vault kubelet[26517]: Flag --client-ca-file has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 12 18:43:58 test-vault kubelet[26517]: Flag --cadvisor-port has been deprecated, The default will change to 0 (disabled) in 1.12, and the cadvisor port will be removed entirely in 1.13
Jun 12 18:43:58 test-vault kubelet[26517]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 12 18:43:58 test-vault kubelet[26517]: Flag --pod-manifest-path has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 12 18:43:58 test-vault kubelet[26517]: Flag --authorization-mode has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 12 18:43:58 test-vault kubelet[26517]: Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.698782   26517 feature_gate.go:226] feature gates: &{{} map[]}
Jun 12 18:43:58 test-vault kubelet[26517]: W0612 18:43:58.706805   26517 cni.go:171] Unable to update cni config: No networks found in /etc/cni/net.d
Jun 12 18:43:58 test-vault kubelet[26517]: W0612 18:43:58.714501   26517 hostport_manager.go:68] The binary conntrack is not installed, this can cause failures in network connection cleanup.
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.714572   26517 server.go:376] Version: v1.10.0
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.714639   26517 feature_gate.go:226] feature gates: &{{} map[]}
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.714797   26517 plugins.go:89] No cloud provider specified.
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.813763   26517 server.go:613] --cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.814326   26517 container_manager_linux.go:242] container manager verified user specified cgroup-root exists: /
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.814362   26517 container_manager_linux.go:247] Creating Container Manager object based on Node Config: {RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:docker CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:memory.available Operator:LessThan Value:{Quantity:100Mi Percentage:0} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.1} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.inodesFree Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>} {Signal:imagefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.15} GracePeriod:0s MinReclaim:<nil>}]} ExperimentalQOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalCPUManagerReconcilePeriod:10s ExperimentalPodPidsLimit:-1 EnforceCPULimits:true}
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.814546   26517 container_manager_linux.go:266] Creating device plugin manager: true
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.814669   26517 state_mem.go:36] [cpumanager] initializing new in-memory state store
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.814797   26517 state_file.go:82] [cpumanager] state file: created new state file "/var/lib/kubelet/cpu_manager_state"
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.814884   26517 kubelet.go:272] Adding pod path: /etc/kubernetes/manifests
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.814938   26517 kubelet.go:297] Watching apiserver
Jun 12 18:43:58 test-vault kubelet[26517]: E0612 18:43:58.818644   26517 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://9.30.250.222:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 9.30.250.222:8443: getsockopt: connection refused
Jun 12 18:43:58 test-vault kubelet[26517]: E0612 18:43:58.818663   26517 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Service: Get https://9.30.250.222:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 9.30.250.222:8443: getsockopt: connection refused
Jun 12 18:43:58 test-vault kubelet[26517]: E0612 18:43:58.818749   26517 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list *v1.Node: Get https://9.30.250.222:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 9.30.250.222:8443: getsockopt: connection refused
Jun 12 18:43:58 test-vault kubelet[26517]: W0612 18:43:58.826793   26517 kubelet_network.go:139] Hairpin mode set to "promiscuous-bridge" but kubenet is not enabled, falling back to "hairpin-veth"
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.826866   26517 kubelet.go:556] Hairpin mode set to "hairpin-veth"
Jun 12 18:43:58 test-vault kubelet[26517]: W0612 18:43:58.827020   26517 cni.go:171] Unable to update cni config: No networks found in /etc/cni/net.d
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.827106   26517 client.go:75] Connecting to docker on unix:///var/run/docker.sock
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.827143   26517 client.go:104] Start docker client with request timeout=2m0s
Jun 12 18:43:58 test-vault kubelet[26517]: W0612 18:43:58.848702   26517 cni.go:171] Unable to update cni config: No networks found in /etc/cni/net.d
Jun 12 18:43:58 test-vault kubelet[26517]: W0612 18:43:58.863262   26517 hostport_manager.go:68] The binary conntrack is not installed, this can cause failures in network connection cleanup.
Jun 12 18:43:58 test-vault kubelet[26517]: W0612 18:43:58.863391   26517 cni.go:171] Unable to update cni config: No networks found in /etc/cni/net.d
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.863447   26517 docker_service.go:244] Docker cri networking managed by cni
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.910771   26517 docker_service.go:249] Docker Info: &{ID:DUZH:6KBS:PDRJ:WKYK:IHXM:WGGV:ATXO:UUOJ:GDMY:MJWW:C6M5:QSAW Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:0 Driver:devicemapper DriverStatus:[[Pool Name docker-253:0-402872253-pool] [Pool Blocksize 65.54kB] [Base Device Size 10.74GB] [Backing Filesystem xfs] [Udev Sync Supported true] [Data file /dev/loop0] [Metadata file /dev/loop1] [Data loop file /var/lib/docker/devicemapper/devicemapper/data] [Metadata loop file /var/lib/docker/devicemapper/devicemapper/metadata] [Data Space Used 11.73MB] [Data Space Total 107.4GB] [Data Space Available 107.4GB] [Metadata Space Used 581.6kB] [Metadata Space Total 2.147GB] [Metadata Space Available 2.147GB] [Thin Pool Minimum Free Space 10.74GB] [Deferred Removal Enabled true] [Deferred Deletion Enabled true] [Deferred Deleted Device Count 0] [Library Version 1.02.146-RHEL7 (2018-01-22)]] SystemStatus:[] Plugins:{Volume:[local] Network:[bridge host macvlan null overlay] Authorization:[] Log:[awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6tables:true Debug:false NFd:23 OomKillDisable:true NGoroutines:35 SystemTime:2018-06-12T18:43:58.876870844-07:00 LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:3.10.0-693.17.1.el7.x86_64 OperatingSystem:CentOS Linux 7 (Core) OSType:linux Architecture:x86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:0xc420166af0 NCPU:8 MemTotal:33568006144 GenericResources:[] DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:test-vault Labels:[provider=fyre] ExperimentalBuild:false ServerVersion:18.05.0-ce ClusterStore: ClusterAdvertise: Runtimes:map[runc:{Path:docker-runc Args:[]}] DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:[] Nodes
Jun 12 18:43:58 test-vault kubelet[26517]: :0 Managers:0 Cluster:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:773c489c9c1b21a6d78b5c538cd395416ec50f88 Expected:773c489c9c1b21a6d78b5c538cd395416ec50f88} RuncCommit:{ID:4fc53a81fb7c994640722ac585fa9ca548971871 Expected:4fc53a81fb7c994640722ac585fa9ca548971871} InitCommit:{ID:949e6fa Expected:949e6fa} SecurityOptions:[name=seccomp,profile=default]}
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.915246   26517 docker_service.go:262] Setting cgroupDriver to cgroupfs
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.968917   26517 remote_runtime.go:43] Connecting to runtime service unix:///var/run/dockershim.sock
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.973395   26517 kuberuntime_manager.go:186] Container runtime docker initialized, version: 18.05.0-ce, apiVersion: 1.37.0
Jun 12 18:43:58 test-vault kubelet[26517]: W0612 18:43:58.979153   26517 probe.go:215] Flexvolume plugin directory at /usr/libexec/kubernetes/kubelet-plugins/volume/exec/ does not exist. Recreating.
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.979755   26517 csi_plugin.go:61] kubernetes.io/csi: plugin initializing...
Jun 12 18:43:58 test-vault kubelet[26517]: E0612 18:43:58.988600   26517 kubelet.go:1277] Image garbage collection failed once. Stats initialization may not have completed yet: failed to get imageFs info: unable to find data for container /
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.989138   26517 server.go:129] Starting to listen on 0.0.0.0:10250
Jun 12 18:43:58 test-vault kubelet[26517]: E0612 18:43:58.989753   26517 event.go:209] Unable to write event: 'Post https://9.30.250.222:8443/api/v1/namespaces/default/events: dial tcp 9.30.250.222:8443: getsockopt: connection refused' (may retry after sleeping)
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.989801   26517 fs_resource_analyzer.go:66] Starting FS ResourceAnalyzer
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.989854   26517 status_manager.go:140] Starting to sync pod status with apiserver
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.989933   26517 kubelet.go:1777] Starting kubelet main sync loop.
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.989964   26517 kubelet.go:1794] skipping pod synchronization - [container runtime is down PLEG is not healthy: pleg was last seen active 2562047h47m16.854775807s ago; threshold is 3m0s]
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.990140   26517 server.go:944] Started kubelet
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.990393   26517 volume_manager.go:247] Starting Kubelet Volume Manager
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.990498   26517 desired_state_of_world_populator.go:129] Desired state populator starts to run
Jun 12 18:43:58 test-vault kubelet[26517]: I0612 18:43:58.991695   26517 server.go:299] Adding debug handlers to kubelet server.
Jun 12 18:43:59 test-vault kubelet[26517]: W0612 18:43:58.992140   26517 cni.go:171] Unable to update cni config: No networks found in /etc/cni/net.d
Jun 12 18:43:59 test-vault kubelet[26517]: E0612 18:43:58.992372   26517 kubelet.go:2125] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Jun 12 18:43:59 test-vault kubelet[26517]: E0612 18:43:59.050807   26517 factory.go:340] devicemapper filesystem stats will not be reported: usage of thin_ls is disabled to preserve iops
Jun 12 18:43:59 test-vault kubelet[26517]: I0612 18:43:59.091594   26517 kubelet.go:1794] skipping pod synchronization - [container runtime is down]
Jun 12 18:43:59 test-vault kubelet[26517]: I0612 18:43:59.091744   26517 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach
Jun 12 18:43:59 test-vault kubelet[26517]: I0612 18:43:59.099863   26517 kubelet_node_status.go:82] Attempting to register node minikube
Jun 12 18:43:59 test-vault kubelet[26517]: E0612 18:43:59.100465   26517 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://9.30.250.222:8443/api/v1/nodes: dial tcp 9.30.250.222:8443: getsockopt: connection refused
Jun 12 18:43:59 test-vault kubelet[26517]: I0612 18:43:59.291948   26517 kubelet.go:1794] skipping pod synchronization - [container runtime is down]
Jun 12 18:43:59 test-vault kubelet[26517]: I0612 18:43:59.300772   26517 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach
Jun 12 18:43:59 test-vault kubelet[26517]: I0612 18:43:59.316149   26517 kubelet_node_status.go:82] Attempting to register node minikube
Jun 12 18:43:59 test-vault kubelet[26517]: E0612 18:43:59.318109   26517 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://9.30.250.222:8443/api/v1/nodes: dial tcp 9.30.250.222:8443: getsockopt: connection refused
Jun 12 18:43:59 test-vault kubelet[26517]: I0612 18:43:59.692234   26517 kubelet.go:1794] skipping pod synchronization - [container runtime is down]
Jun 12 18:43:59 test-vault kubelet[26517]: I0612 18:43:59.718550   26517 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach
Jun 12 18:43:59 test-vault kubelet[26517]: I0612 18:43:59.732503   26517 kubelet_node_status.go:82] Attempting to register node minikube
Jun 12 18:43:59 test-vault kubelet[26517]: E0612 18:43:59.732956   26517 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://9.30.250.222:8443/api/v1/nodes: dial tcp 9.30.250.222:8443: getsockopt: connection refused
Jun 12 18:43:59 test-vault kubelet[26517]: E0612 18:43:59.819446   26517 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://9.30.250.222:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 9.30.250.222:8443: getsockopt: connection refused
Jun 12 18:43:59 test-vault kubelet[26517]: E0612 18:43:59.821623   26517 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Service: Get https://9.30.250.222:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 9.30.250.222:8443: getsockopt: connection refused
Jun 12 18:43:59 test-vault kubelet[26517]: E0612 18:43:59.823397   26517 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list *v1.Node: Get https://9.30.250.222:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 9.30.250.222:8443: getsockopt: connection refused
Jun 12 18:44:00 test-vault kubelet[26517]: I0612 18:44:00.492518   26517 kubelet.go:1794] skipping pod synchronization - [container runtime is down]
Jun 12 18:44:00 test-vault kubelet[26517]: I0612 18:44:00.533499   26517 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach
Jun 12 18:44:00 test-vault kubelet[26517]: I0612 18:44:00.546690   26517 kubelet_node_status.go:82] Attempting to register node minikube
Jun 12 18:44:00 test-vault kubelet[26517]: E0612 18:44:00.547304   26517 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://9.30.250.222:8443/api/v1/nodes: dial tcp 9.30.250.222:8443: getsockopt: connection refused

The Failed to list *v1.Service: Get https://.... messages continue on like that for 1000s of lines.

I should mention I'm running Minikube on Centos 7.4 and Ubuntu 18.04 virtual machines provisioned by docker-machine version 0.14.0. The VMs are running Docker v18.05.0-ce with minikube config set vm-driver none.

huguesalary commented 6 years ago

Running into the same issue with:

macos 10.13.5 minikube 0.28 kubernetes-version v1.9.6 hyperkit driver hyperkit v0.20180403-15-g753684

On my end, doing kubectl get all --all reveals that all the system pods are running, but, for some reason the storage provisionner and kubernetes-dashboards keep restarting

bfalor commented 6 years ago

This issue was due to my company's network certificates not being installed properly, minikube images would never pull because of that. Unplugging my ethernet cable resulted in a seemless minikube installation.

thebetterjort commented 6 years ago

I don't know if this issue should be closed. Many others have reported it and I'm not sure the fix is SSL certs. My corp does SSL inspection, but switching to public wifi still doesn't sovle.

724399396 commented 6 years ago

minikube start --kubernetes-version v1.9.4 solve my same issue. kubenetes version: 1.11.2 minikube version: 0.25.2

Berndinox commented 6 years ago

Ubuntu 18 - Fresh installed, Problem still there

BTW: me@host~ # minikube start --kubernetes-version v1.9.4 --vm-driver=none Starting local Kubernetes v1.9.4 cluster... Starting VM... Getting VM IP address... Kubernetes version downgrade is not supported. Using version: v1.10.0

gaohoward commented 6 years ago

I've got similar problem starting minikube v0.28.2 Ubuntu 16.04 virtualBox 5.2.18 I'm using wifi network connection

Here is console output: ]$ minikube start Starting local Kubernetes v1.10.0 cluster... Starting VM... Getting VM IP address... Moving files into cluster... Downloading kubeadm v1.10.0 Downloading kubelet v1.10.0 Finished Downloading kubelet v1.10.0 Finished Downloading kubeadm v1.10.0 Setting up certs... Connecting to cluster... Setting up kubeconfig... Starting cluster components... E0918 08:16:28.469751 32292 start.go:305] Error restarting cluster: restarting kube-proxy: waiting for kube-proxy to be up for configmap update: timed out waiting for the condition

An error has occurred. Would you like to opt in to sending anonymized crash information to minikube to help prevent future errors? To opt out of these messages, run the command: minikube config set WantReportErrorPrompt false

Please enter your response [Y/n]:

tstromberg commented 6 years ago

This bug report included no logs, so it's hard to tell what the root cause was. If you see something similar, please open a new bug report rather than adding to this, with the output of:

minikube logs

minikube ssh "sudo journalctl -u kubelet"

minikube ssh "sudo journalctl -xe"

Thanks!

SYip commented 6 years ago

Same issue in OSX.

minikube version: v0.28.2 kubectl version Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.3", GitCommit:"a4529464e4629c21224b3d52edfe0ea91b072862", GitTree State:"clean", BuildDate:"2018-09-10T11:44:36Z", GoVersion:"go1.11", Compiler:"gc", Platform:"darwin/amd64"}

minikube logs:

-- Logs begin at Thu 2018-09-27 02:57:43 UTC, end at Thu 2018-09-27 03:03:54 UTC. -- Sep 27 02:58:04 minikube systemd[1]: Started kubelet: The Kubernetes Node Agent. Sep 27 02:58:04 minikube kubelet[2621]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 27 02:58:04 minikube kubelet[2621]: Flag --cluster-dns has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 27 02:58:04 minikube kubelet[2621]: Flag --client-ca-file has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 27 02:58:04 minikube kubelet[2621]: Flag --authorization-mode has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 27 02:58:04 minikube kubelet[2621]: Flag --cadvisor-port has been deprecated, The default will change to 0 (disabled) in 1.12, and the cadvisor port will be removed entirely in 1.13 Sep 27 02:58:04 minikube kubelet[2621]: Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 27 02:58:04 minikube kubelet[2621]: Flag --pod-manifest-path has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 27 02:58:04 minikube kubelet[2621]: Flag --allow-privileged has been deprecated, will be removed in a future version Sep 27 02:58:04 minikube kubelet[2621]: Flag --cluster-domain has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 27 02:58:04 minikube kubelet[2621]: I0927 02:58:04.890537 2621 feature_gate.go:226] feature gates: &{{} map[]} Sep 27 02:58:04 minikube kubelet[2621]: W0927 02:58:04.909173 2621 hostport_manager.go:68] The binary conntrack is not installed, this can cause failures in network connection cleanup. Sep 27 02:58:04 minikube kubelet[2621]: I0927 02:58:04.909226 2621 server.go:376] Version: v1.10.0 Sep 27 02:58:04 minikube kubelet[2621]: I0927 02:58:04.909485 2621 feature_gate.go:226] feature gates: &{{} map[]} Sep 27 02:58:04 minikube kubelet[2621]: I0927 02:58:04.909604 2621 plugins.go:89] No cloud provider specified. Sep 27 02:58:04 minikube kubelet[2621]: F0927 02:58:04.909652 2621 server.go:233] failed to run Kubelet: unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubelet.conf: no such file or directory Sep 27 02:58:04 minikube systemd[1]: kubelet.service: Main process exited, code=exited, status=255/n/a Sep 27 02:58:04 minikube systemd[1]: kubelet.service: Failed with result 'exit-code'. Sep 27 02:58:15 minikube systemd[1]: kubelet.service: Service hold-off time over, scheduling restart. Sep 27 02:58:15 minikube systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 1. Sep 27 02:58:15 minikube systemd[1]: Stopped kubelet: The Kubernetes Node Agent. Sep 27 02:58:15 minikube systemd[1]: Started kubelet: The Kubernetes Node Agent. Sep 27 02:58:15 minikube kubelet[2693]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 27 02:58:15 minikube kubelet[2693]: Flag --cluster-dns has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 27 02:58:15 minikube kubelet[2693]: Flag --client-ca-file has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 27 02:58:15 minikube kubelet[2693]: Flag --authorization-mode has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 27 02:58:15 minikube kubelet[2693]: Flag --cadvisor-port has been deprecated, The default will change to 0 (disabled) in 1.12, and the cadvisor port will be removed entirely in 1.13 Sep 27 02:58:15 minikube kubelet[2693]: Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 27 02:58:15 minikube kubelet[2693]: Flag --pod-manifest-path has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 27 02:58:15 minikube kubelet[2693]: Flag --allow-privileged has been deprecated, will be removed in a future version Sep 27 02:58:15 minikube kubelet[2693]: Flag --cluster-domain has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.086722 2693 feature_gate.go:226] feature gates: &{{} map[]} Sep 27 02:58:15 minikube kubelet[2693]: W0927 02:58:15.112324 2693 hostport_manager.go:68] The binary conntrack is not installed, this can cause failures in network connection cleanup. Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.112753 2693 server.go:376] Version: v1.10.0 Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.114786 2693 feature_gate.go:226] feature gates: &{{} map[]} Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.115373 2693 plugins.go:89] No cloud provider specified. Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.297958 2693 server.go:613] --cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to / Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.298771 2693 container_manager_linux.go:242] container manager verified user specified cgroup-root exists: / Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.298872 2693 container_manager_linux.go:247] Creating Container Manager object based on Node Config: {RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:docker CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:nodefs.available Operator:LessThan Value:{Quantity: Percentage:0.1} GracePeriod:0s MinReclaim:} {Signal:nodefs.inodesFree Operator:LessThan Value:{Quantity: Percentage:0.05} GracePeriod:0s MinReclaim:} {Signal:imagefs.available Operator:LessThan Value:{Quantity: Percentage:0.15} GracePeriod:0s MinReclaim:} {Signal:memory.available Operator:LessThan Value:{Quantity:100Mi Percentage:0} GracePeriod:0s MinReclaim:}]} ExperimentalQOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalCPUManagerReconcilePeriod:10s ExperimentalPodPidsLimit:-1 EnforceCPULimits:true} Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.300496 2693 container_manager_linux.go:266] Creating device plugin manager: true Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.300649 2693 state_mem.go:36] [cpumanager] initializing new in-memory state store Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.303869 2693 state_mem.go:87] [cpumanager] updated default cpuset: "" Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.304261 2693 state_mem.go:95] [cpumanager] updated cpuset assignments: "map[]" Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.304498 2693 kubelet.go:272] Adding pod path: /etc/kubernetes/manifests Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.304598 2693 kubelet.go:297] Watching apiserver Sep 27 02:58:15 minikube kubelet[2693]: E0927 02:58:15.318270 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:15 minikube kubelet[2693]: E0927 02:58:15.318556 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:15 minikube kubelet[2693]: E0927 02:58:15.318846 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:15 minikube kubelet[2693]: W0927 02:58:15.343782 2693 kubelet_network.go:139] Hairpin mode set to "promiscuous-bridge" but kubenet is not enabled, falling back to "hairpin-veth" Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.344033 2693 kubelet.go:556] Hairpin mode set to "hairpin-veth" Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.345823 2693 client.go:75] Connecting to docker on unix:///var/run/docker.sock Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.346041 2693 client.go:104] Start docker client with request timeout=2m0s Sep 27 02:58:15 minikube kubelet[2693]: W0927 02:58:15.360333 2693 hostport_manager.go:68] The binary conntrack is not installed, this can cause failures in network connection cleanup. Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.372641 2693 docker_service.go:244] Docker cri networking managed by kubernetes.io/no-op Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.381353 2693 docker_service.go:249] Docker Info: &{ID:E243:PPML:4FZK:TE2E:TDOQ:7SGP:OUR2:EZUH:2NPJ:PISL:6NRO:POFJ Containers:16 ContainersRunning:0 ContainersPaused:0 ContainersStopped:16 Images:9 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true]] SystemStatus:[] Plugins:{Volume:[local] Network:[bridge host macvlan null overlay] Authorization:[] Log:[awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6tables:true Debug:false NFd:22 OomKillDisable:true NGoroutines:34 SystemTime:2018-09-27T02:58:15.376597366Z LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:4.15.0 OperatingSystem:Buildroot 2018.05 OSType:linux Architecture:x86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:0xc420132230 NCPU:2 MemTotal:1030586368 GenericResources:[] DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:minikube Labels:[provider=virtualbox] ExperimentalBuild:false ServerVersion:17.12.1-ce ClusterStore: ClusterAdvertise: Runtimes:map[runc:{Path:docker-runc Args:[]}] DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:[] Nodes:0 Managers:0 Cluster:} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:9b55aab90508bd389d7654c4baf173a981477d55 Expected:9b55aab90508bd389d7654c4baf173a981477d55} RuncCommit:{ID:9f9c96235cc97674e935002fc3d78361b696a69e Expected:9f9c96235cc97674e935002fc3d78361b696a69e} InitCommit:{ID:N/A Expected:} SecurityOptions:[name=seccomp,profile=default]} Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.381539 2693 docker_service.go:262] Setting cgroupDriver to cgroupfs Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.425255 2693 remote_runtime.go:43] Connecting to runtime service unix:///var/run/dockershim.sock Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.456249 2693 kuberuntime_manager.go:186] Container runtime docker initialized, version: 17.12.1-ce, apiVersion: 1.35.0 Sep 27 02:58:15 minikube kubelet[2693]: W0927 02:58:15.456558 2693 probe.go:215] Flexvolume plugin directory at /usr/libexec/kubernetes/kubelet-plugins/volume/exec/ does not exist. Recreating. Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.457027 2693 csi_plugin.go:61] kubernetes.io/csi: plugin initializing... Sep 27 02:58:15 minikube kubelet[2693]: E0927 02:58:15.474174 2693 kubelet.go:1277] Image garbage collection failed once. Stats initialization may not have completed yet: failed to get imageFs info: unable to find data for container / Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.474722 2693 fs_resource_analyzer.go:66] Starting FS ResourceAnalyzer Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.474841 2693 status_manager.go:140] Starting to sync pod status with apiserver Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.475055 2693 kubelet.go:1777] Starting kubelet main sync loop. Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.475127 2693 kubelet.go:1794] skipping pod synchronization - [container runtime is down PLEG is not healthy: pleg was last seen active 2562047h47m16.854775807s ago; threshold is 3m0s] Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.475309 2693 server.go:129] Starting to listen on 0.0.0.0:10250 Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.475989 2693 server.go:299] Adding debug handlers to kubelet server. Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.477447 2693 volume_manager.go:247] Starting Kubelet Volume Manager Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.479086 2693 server.go:944] Started kubelet Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.479220 2693 desired_state_of_world_populator.go:129] Desired state populator starts to run Sep 27 02:58:15 minikube kubelet[2693]: E0927 02:58:15.479444 2693 event.go:209] Unable to write event: 'Post https://localhost:8443/api/v1/namespaces/default/events: dial tcp 127.0.0.1:8443: getsockopt: connection refused' (may retry after sleeping) Sep 27 02:58:15 minikube kubelet[2693]: W0927 02:58:15.566920 2693 docker_sandbox.go:353] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kube-dns-6dcb57bcc8-q877v through plugin: invalid network status for Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.575361 2693 kubelet.go:1794] skipping pod synchronization - [container runtime is down] Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.578038 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.581606 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 02:58:15 minikube kubelet[2693]: E0927 02:58:15.584699 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:15 minikube kubelet[2693]: W0927 02:58:15.590102 2693 docker_sandbox.go:353] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kube-dns-6dcb57bcc8-gv2lc through plugin: invalid network status for Sep 27 02:58:15 minikube kubelet[2693]: W0927 02:58:15.601090 2693 docker_sandbox.go:353] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kube-dns-6dcb57bcc8-9pz8w through plugin: invalid network status for Sep 27 02:58:15 minikube kubelet[2693]: W0927 02:58:15.612691 2693 docker_sandbox.go:353] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kube-dns-6dcb57bcc8-hjhss through plugin: invalid network status for Sep 27 02:58:15 minikube kubelet[2693]: W0927 02:58:15.713767 2693 docker_sandbox.go:353] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kube-dns-6dcb57bcc8-njhzn through plugin: invalid network status for Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.776201 2693 kubelet.go:1794] skipping pod synchronization - [container runtime is down] Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.785099 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:15 minikube kubelet[2693]: I0927 02:58:15.788959 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 02:58:15 minikube kubelet[2693]: E0927 02:58:15.789562 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:15 minikube kubelet[2693]: W0927 02:58:15.824578 2693 docker_sandbox.go:353] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kube-dns-6dcb57bcc8-2d2xv through plugin: invalid network status for Sep 27 02:58:15 minikube kubelet[2693]: E0927 02:58:15.826489 2693 kuberuntime_manager.go:859] PodSandboxStatus of sandbox "579445bde652cdfd352a20d069f6bdf3fb4392f7b0aaf888cedac78670e7872a" for pod "kube-dns-6dcb57bcc8-2d2xv_kube-system(dd06fd2b-c200-11e8-aeb2-08002786b439)" error: rpc error: code = Unknown desc = Error: No such container: 579445bde652cdfd352a20d069f6bdf3fb4392f7b0aaf888cedac78670e7872a Sep 27 02:58:15 minikube kubelet[2693]: E0927 02:58:15.826642 2693 generic.go:241] PLEG: Ignoring events for pod kube-dns-6dcb57bcc8-2d2xv/kube-system: rpc error: code = Unknown desc = Error: No such container: 579445bde652cdfd352a20d069f6bdf3fb4392f7b0aaf888cedac78670e7872a Sep 27 02:58:16 minikube kubelet[2693]: I0927 02:58:16.177366 2693 kubelet.go:1794] skipping pod synchronization - [container runtime is down] Sep 27 02:58:16 minikube kubelet[2693]: I0927 02:58:16.189958 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:16 minikube kubelet[2693]: I0927 02:58:16.193224 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 02:58:16 minikube kubelet[2693]: E0927 02:58:16.194061 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:16 minikube kubelet[2693]: E0927 02:58:16.319305 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:16 minikube kubelet[2693]: E0927 02:58:16.324607 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:16 minikube kubelet[2693]: E0927 02:58:16.327385 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:16 minikube kubelet[2693]: W0927 02:58:16.837700 2693 docker_sandbox.go:353] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kube-dns-6dcb57bcc8-2d2xv through plugin: invalid network status for Sep 27 02:58:16 minikube kubelet[2693]: E0927 02:58:16.965616 2693 event.go:209] Unable to write event: 'Post https://localhost:8443/api/v1/namespaces/default/events: dial tcp 127.0.0.1:8443: getsockopt: connection refused' (may retry after sleeping) Sep 27 02:58:16 minikube kubelet[2693]: I0927 02:58:16.978212 2693 kubelet.go:1794] skipping pod synchronization - [container runtime is down] Sep 27 02:58:16 minikube kubelet[2693]: I0927 02:58:16.994235 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:16 minikube kubelet[2693]: I0927 02:58:16.997310 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 02:58:16 minikube kubelet[2693]: E0927 02:58:16.998214 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:17 minikube kubelet[2693]: E0927 02:58:17.321274 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:17 minikube kubelet[2693]: E0927 02:58:17.326093 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:17 minikube kubelet[2693]: E0927 02:58:17.329237 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:17 minikube kubelet[2693]: I0927 02:58:17.645733 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:17 minikube kubelet[2693]: I0927 02:58:17.651349 2693 cpu_manager.go:155] [cpumanager] starting with none policy Sep 27 02:58:17 minikube kubelet[2693]: I0927 02:58:17.651438 2693 cpu_manager.go:156] [cpumanager] reconciling every 10s Sep 27 02:58:17 minikube kubelet[2693]: I0927 02:58:17.651458 2693 policy_none.go:42] [cpumanager] none policy: Start Sep 27 02:58:17 minikube kubelet[2693]: Starting Device Plugin manager Sep 27 02:58:17 minikube kubelet[2693]: E0927 02:58:17.666499 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 02:58:18 minikube kubelet[2693]: E0927 02:58:18.325071 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:18 minikube kubelet[2693]: E0927 02:58:18.327502 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:18 minikube kubelet[2693]: E0927 02:58:18.330694 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:18 minikube kubelet[2693]: W0927 02:58:18.578401 2693 pod_container_deletor.go:77] Container "6ec4c4b6b36d3fc0e0dcd59a915f06bf820cf39bea3f5fbe4da898103a66cef6" not found in pod's containers Sep 27 02:58:18 minikube kubelet[2693]: W0927 02:58:18.578439 2693 pod_container_deletor.go:77] Container "f271c8a3431c8d7bcea96171a99203917746544a932b8ecd5caae0fedcf19cfc" not found in pod's containers Sep 27 02:58:18 minikube kubelet[2693]: W0927 02:58:18.578447 2693 pod_container_deletor.go:77] Container "ee707418b024ebb7f93239581e6f18aa94534165d32aeea4e074ab2a02c07c61" not found in pod's containers Sep 27 02:58:18 minikube kubelet[2693]: W0927 02:58:18.578460 2693 pod_container_deletor.go:77] Container "c0dc5a51d1e8defeea263b8b45b82fbe9ddf426e0f9213d68995eccde7c148d4" not found in pod's containers Sep 27 02:58:18 minikube kubelet[2693]: W0927 02:58:18.578471 2693 pod_container_deletor.go:77] Container "0cbfe4cfe9efa794d01d007bd40d5e9f765bb376effb2c518970ddc011e424d8" not found in pod's containers Sep 27 02:58:18 minikube kubelet[2693]: W0927 02:58:18.578482 2693 pod_container_deletor.go:77] Container "7f1bc0add3f208e0cf67e81bf0eb1ea7dd99d7d0de8cc422d926f1a909b37957" not found in pod's containers Sep 27 02:58:18 minikube kubelet[2693]: W0927 02:58:18.578488 2693 pod_container_deletor.go:77] Container "718f5e8b04acf1de82fb51ed6cf62e8662f15a2ed44d386e83f4475b5d57ead1" not found in pod's containers Sep 27 02:58:18 minikube kubelet[2693]: W0927 02:58:18.578495 2693 pod_container_deletor.go:77] Container "a66811a90dcf09ef764d7aa64e801ff33b2c9133dd46e078c899ee03fc4c01b6" not found in pod's containers Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.578549 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.592862 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.593129 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.603070 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.612210 2693 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "addons" (UniqueName: "kubernetes.io/host-path/3afaf06535cc3b85be93c31632b765da-addons") pod "kube-addon-manager-minikube" (UID: "3afaf06535cc3b85be93c31632b765da") Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.612251 2693 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "kubeconfig" (UniqueName: "kubernetes.io/host-path/3afaf06535cc3b85be93c31632b765da-kubeconfig") pod "kube-addon-manager-minikube" (UID: "3afaf06535cc3b85be93c31632b765da") Sep 27 02:58:18 minikube kubelet[2693]: W0927 02:58:18.612869 2693 status_manager.go:461] Failed to get status for pod "kube-addon-manager-minikube_kube-system(3afaf06535cc3b85be93c31632b765da)": Get https://localhost:8443/api/v1/namespaces/kube-system/pods/kube-addon-manager-minikube: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.617739 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 02:58:18 minikube kubelet[2693]: E0927 02:58:18.618274 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.618596 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.619325 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:18 minikube kubelet[2693]: W0927 02:58:18.622675 2693 status_manager.go:461] Failed to get status for pod "etcd-minikube_kube-system(2a2bc19679cc9ec706a86a529c6f37b0)": Get https://localhost:8443/api/v1/namespaces/kube-system/pods/etcd-minikube: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.638614 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.638864 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.643893 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:18 minikube kubelet[2693]: W0927 02:58:18.644683 2693 status_manager.go:461] Failed to get status for pod "kube-apiserver-minikube_kube-system(09eddb205ac749afd96943c8b76939b8)": Get https://localhost:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-minikube: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.645063 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:18 minikube kubelet[2693]: W0927 02:58:18.652348 2693 pod_container_deletor.go:77] Container "2b5143db4bcffde8d980346fe6442c7425e8d7aa199255ae00e8d26a934c63e9" not found in pod's containers Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.652877 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:18 minikube kubelet[2693]: W0927 02:58:18.658757 2693 status_manager.go:461] Failed to get status for pod "kube-scheduler-minikube_kube-system(099f1c2b79126109140a1f77e211df00)": Get https://localhost:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-minikube: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:18 minikube kubelet[2693]: W0927 02:58:18.666799 2693 status_manager.go:461] Failed to get status for pod "kube-controller-manager-minikube_kube-system(11b68d8fd5d4ab0c48750a8839d7c1a6)": Get https://localhost:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-minikube: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.713548 2693 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "etcd-data" (UniqueName: "kubernetes.io/host-path/2a2bc19679cc9ec706a86a529c6f37b0-etcd-data") pod "etcd-minikube" (UID: "2a2bc19679cc9ec706a86a529c6f37b0") Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.713636 2693 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "etcd-certs" (UniqueName: "kubernetes.io/host-path/2a2bc19679cc9ec706a86a529c6f37b0-etcd-certs") pod "etcd-minikube" (UID: "2a2bc19679cc9ec706a86a529c6f37b0") Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.713654 2693 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "ca-certs" (UniqueName: "kubernetes.io/host-path/09eddb205ac749afd96943c8b76939b8-ca-certs") pod "kube-apiserver-minikube" (UID: "09eddb205ac749afd96943c8b76939b8") Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.713673 2693 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "k8s-certs" (UniqueName: "kubernetes.io/host-path/11b68d8fd5d4ab0c48750a8839d7c1a6-k8s-certs") pod "kube-controller-manager-minikube" (UID: "11b68d8fd5d4ab0c48750a8839d7c1a6") Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.713693 2693 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "ca-certs" (UniqueName: "kubernetes.io/host-path/11b68d8fd5d4ab0c48750a8839d7c1a6-ca-certs") pod "kube-controller-manager-minikube" (UID: "11b68d8fd5d4ab0c48750a8839d7c1a6") Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.713722 2693 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "k8s-certs" (UniqueName: "kubernetes.io/host-path/09eddb205ac749afd96943c8b76939b8-k8s-certs") pod "kube-apiserver-minikube" (UID: "09eddb205ac749afd96943c8b76939b8") Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.713738 2693 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "kubeconfig" (UniqueName: "kubernetes.io/host-path/11b68d8fd5d4ab0c48750a8839d7c1a6-kubeconfig") pod "kube-controller-manager-minikube" (UID: "11b68d8fd5d4ab0c48750a8839d7c1a6") Sep 27 02:58:18 minikube kubelet[2693]: I0927 02:58:18.713754 2693 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "kubeconfig" (UniqueName: "kubernetes.io/host-path/099f1c2b79126109140a1f77e211df00-kubeconfig") pod "kube-scheduler-minikube" (UID: "099f1c2b79126109140a1f77e211df00") Sep 27 02:58:19 minikube kubelet[2693]: E0927 02:58:19.326388 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:19 minikube kubelet[2693]: E0927 02:58:19.328859 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:19 minikube kubelet[2693]: E0927 02:58:19.332198 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:20 minikube kubelet[2693]: I0927 02:58:20.281145 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:20 minikube kubelet[2693]: E0927 02:58:20.327574 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:20 minikube kubelet[2693]: E0927 02:58:20.364950 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:20 minikube kubelet[2693]: E0927 02:58:20.368293 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:20 minikube kubelet[2693]: W0927 02:58:20.406983 2693 status_manager.go:461] Failed to get status for pod "kube-apiserver-minikube_kube-system(09eddb205ac749afd96943c8b76939b8)": Get https://localhost:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-minikube: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:21 minikube kubelet[2693]: E0927 02:58:21.341830 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:21 minikube kubelet[2693]: E0927 02:58:21.377713 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:21 minikube kubelet[2693]: E0927 02:58:21.377426 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:21 minikube kubelet[2693]: I0927 02:58:21.574452 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:21 minikube kubelet[2693]: W0927 02:58:21.691198 2693 status_manager.go:461] Failed to get status for pod "kube-scheduler-minikube_kube-system(099f1c2b79126109140a1f77e211df00)": Get https://localhost:8443/api/v1/namespaces/kube-system/pods/kube-scheduler-minikube: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:21 minikube kubelet[2693]: I0927 02:58:21.818854 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:21 minikube kubelet[2693]: I0927 02:58:21.822528 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 02:58:21 minikube kubelet[2693]: E0927 02:58:21.825701 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:22 minikube kubelet[2693]: E0927 02:58:22.346446 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:22 minikube kubelet[2693]: E0927 02:58:22.381704 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:22 minikube kubelet[2693]: E0927 02:58:22.404817 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:22 minikube kubelet[2693]: I0927 02:58:22.849770 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:22 minikube kubelet[2693]: I0927 02:58:22.946332 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:22 minikube kubelet[2693]: W0927 02:58:22.961245 2693 status_manager.go:461] Failed to get status for pod "kube-controller-manager-minikube_kube-system(11b68d8fd5d4ab0c48750a8839d7c1a6)": Get https://localhost:8443/api/v1/namespaces/kube-system/pods/kube-controller-manager-minikube: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:23 minikube kubelet[2693]: I0927 02:58:23.104248 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:23 minikube kubelet[2693]: W0927 02:58:23.116195 2693 status_manager.go:461] Failed to get status for pod "etcd-minikube_kube-system(2a2bc19679cc9ec706a86a529c6f37b0)": Get https://localhost:8443/api/v1/namespaces/kube-system/pods/etcd-minikube: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:23 minikube kubelet[2693]: I0927 02:58:23.167765 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:23 minikube kubelet[2693]: W0927 02:58:23.178887 2693 status_manager.go:461] Failed to get status for pod "kube-addon-manager-minikube_kube-system(3afaf06535cc3b85be93c31632b765da)": Get https://localhost:8443/api/v1/namespaces/kube-system/pods/kube-addon-manager-minikube: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:23 minikube kubelet[2693]: I0927 02:58:23.251743 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:23 minikube kubelet[2693]: E0927 02:58:23.349534 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:23 minikube kubelet[2693]: E0927 02:58:23.382822 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:23 minikube kubelet[2693]: E0927 02:58:23.406734 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:24 minikube kubelet[2693]: E0927 02:58:24.351325 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:24 minikube kubelet[2693]: I0927 02:58:24.357183 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:24 minikube kubelet[2693]: I0927 02:58:24.360614 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:24 minikube kubelet[2693]: I0927 02:58:24.360994 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:24 minikube kubelet[2693]: E0927 02:58:24.395267 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:24 minikube kubelet[2693]: E0927 02:58:24.407710 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 02:58:27 minikube kubelet[2693]: E0927 02:58:27.667494 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 02:58:28 minikube kubelet[2693]: I0927 02:58:28.226158 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:28 minikube kubelet[2693]: I0927 02:58:28.232855 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 02:58:35 minikube kubelet[2693]: E0927 02:58:35.366936 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 02:58:35 minikube kubelet[2693]: E0927 02:58:35.489289 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 02:58:35 minikube kubelet[2693]: E0927 02:58:35.514319 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 02:58:37 minikube kubelet[2693]: E0927 02:58:37.087208 2693 event.go:209] Unable to write event: 'Post https://localhost:8443/api/v1/namespaces/default/events: net/http: TLS handshake timeout' (may retry after sleeping) Sep 27 02:58:37 minikube kubelet[2693]: E0927 02:58:37.689589 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 02:58:38 minikube kubelet[2693]: E0927 02:58:38.263712 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: net/http: TLS handshake timeout Sep 27 02:58:45 minikube kubelet[2693]: I0927 02:58:45.290315 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:58:45 minikube kubelet[2693]: I0927 02:58:45.472342 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 02:58:46 minikube kubelet[2693]: E0927 02:58:46.422339 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 02:58:46 minikube kubelet[2693]: E0927 02:58:46.635747 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 02:58:46 minikube kubelet[2693]: E0927 02:58:46.678934 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 02:58:47 minikube kubelet[2693]: E0927 02:58:47.768747 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 02:58:55 minikube kubelet[2693]: E0927 02:58:55.484770 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: net/http: TLS handshake timeout Sep 27 02:58:57 minikube kubelet[2693]: E0927 02:58:57.147769 2693 event.go:209] Unable to write event: 'Post https://localhost:8443/api/v1/namespaces/default/events: net/http: TLS handshake timeout' (may retry after sleeping) Sep 27 02:58:57 minikube kubelet[2693]: E0927 02:58:57.449761 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 02:58:57 minikube kubelet[2693]: E0927 02:58:57.666002 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 02:58:57 minikube kubelet[2693]: E0927 02:58:57.866570 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 02:58:57 minikube kubelet[2693]: E0927 02:58:57.878784 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 02:59:02 minikube kubelet[2693]: I0927 02:59:02.529718 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:59:02 minikube kubelet[2693]: I0927 02:59:02.787723 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 02:59:07 minikube kubelet[2693]: E0927 02:59:07.889310 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 02:59:08 minikube kubelet[2693]: E0927 02:59:08.495451 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 02:59:08 minikube kubelet[2693]: E0927 02:59:08.733222 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 02:59:08 minikube kubelet[2693]: E0927 02:59:08.906284 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 02:59:12 minikube kubelet[2693]: E0927 02:59:12.927266 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: net/http: TLS handshake timeout Sep 27 02:59:18 minikube kubelet[2693]: E0927 02:59:18.705153 2693 event.go:209] Unable to write event: 'Post https://localhost:8443/api/v1/namespaces/default/events: net/http: TLS handshake timeout' (may retry after sleeping) Sep 27 02:59:19 minikube kubelet[2693]: E0927 02:59:19.229071 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 02:59:20 minikube kubelet[2693]: I0927 02:59:20.030708 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:59:21 minikube kubelet[2693]: E0927 02:59:21.887403 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 02:59:21 minikube kubelet[2693]: E0927 02:59:21.892739 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 02:59:22 minikube kubelet[2693]: E0927 02:59:21.806114 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 02:59:30 minikube kubelet[2693]: E0927 02:59:30.182524 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 02:59:33 minikube kubelet[2693]: I0927 02:59:33.524293 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:59:37 minikube kubelet[2693]: I0927 02:59:37.908534 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 02:59:40 minikube kubelet[2693]: E0927 02:59:40.851924 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 02:59:43 minikube kubelet[2693]: I0927 02:59:43.523995 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:59:43 minikube kubelet[2693]: I0927 02:59:43.554929 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:59:44 minikube kubelet[2693]: I0927 02:59:44.520128 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 02:59:51 minikube kubelet[2693]: E0927 02:59:51.066971 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 02:59:52 minikube kubelet[2693]: I0927 02:59:52.108780 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:00:01 minikube kubelet[2693]: E0927 03:00:01.410344 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:00:11 minikube kubelet[2693]: E0927 03:00:11.608409 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:00:14 minikube kubelet[2693]: I0927 03:00:14.494372 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:00:21 minikube kubelet[2693]: E0927 03:00:21.616162 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:00:31 minikube kubelet[2693]: E0927 03:00:31.644203 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:00:32 minikube kubelet[2693]: E0927 03:00:32.412226 2693 reflector.go:322] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to watch v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&resourceVersion=1&timeoutSeconds=504&watch=true: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:32 minikube kubelet[2693]: I0927 03:00:32.435945 2693 reconciler.go:154] Reconciler: start to sync state Sep 27 03:00:33 minikube kubelet[2693]: E0927 03:00:33.732119 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:33 minikube kubelet[2693]: E0927 03:00:33.840383 2693 reconciler.go:544] updateStates in reconciler: could not get node status with error Get https://localhost:8443/api/v1/nodes/minikube: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:34 minikube kubelet[2693]: I0927 03:00:34.014901 2693 reconciler.go:181] operationExecutor.UnmountVolume started for volume "kube-dns-config" (UniqueName: "kubernetes.io/configmap/dd06fd2b-c200-11e8-aeb2-08002786b439-kube-dns-config") pod "dd06fd2b-c200-11e8-aeb2-08002786b439" (UID: "dd06fd2b-c200-11e8-aeb2-08002786b439") Sep 27 03:00:34 minikube kubelet[2693]: I0927 03:00:34.015932 2693 operation_generator.go:686] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd06fd2b-c200-11e8-aeb2-08002786b439-kube-dns-config" (OuterVolumeSpecName: "kube-dns-config") pod "dd06fd2b-c200-11e8-aeb2-08002786b439" (UID: "dd06fd2b-c200-11e8-aeb2-08002786b439"). InnerVolumeSpecName "kube-dns-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Sep 27 03:00:34 minikube kubelet[2693]: I0927 03:00:34.016670 2693 reconciler.go:181] operationExecutor.UnmountVolume started for volume "default-token-szpjq" (UniqueName: "kubernetes.io/secret/dd06fd2b-c200-11e8-aeb2-08002786b439-default-token-szpjq") pod "dd06fd2b-c200-11e8-aeb2-08002786b439" (UID: "dd06fd2b-c200-11e8-aeb2-08002786b439") Sep 27 03:00:34 minikube kubelet[2693]: I0927 03:00:34.018483 2693 reconciler.go:181] operationExecutor.UnmountVolume started for volume "kube-dns-config" (UniqueName: "kubernetes.io/configmap/e06a81a7-c200-11e8-aeb2-08002786b439-kube-dns-config") pod "e06a81a7-c200-11e8-aeb2-08002786b439" (UID: "e06a81a7-c200-11e8-aeb2-08002786b439") Sep 27 03:00:34 minikube kubelet[2693]: I0927 03:00:34.193033 2693 reconciler.go:181] operationExecutor.UnmountVolume started for volume "default-token-szpjq" (UniqueName: "kubernetes.io/secret/e06a81a7-c200-11e8-aeb2-08002786b439-default-token-szpjq") pod "e06a81a7-c200-11e8-aeb2-08002786b439" (UID: "e06a81a7-c200-11e8-aeb2-08002786b439") Sep 27 03:00:34 minikube kubelet[2693]: I0927 03:00:34.229280 2693 operation_generator.go:686] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e06a81a7-c200-11e8-aeb2-08002786b439-default-token-szpjq" (OuterVolumeSpecName: "default-token-szpjq") pod "e06a81a7-c200-11e8-aeb2-08002786b439" (UID: "e06a81a7-c200-11e8-aeb2-08002786b439"). InnerVolumeSpecName "default-token-szpjq". PluginName "kubernetes.io/secret", VolumeGidValue "" Sep 27 03:00:34 minikube kubelet[2693]: I0927 03:00:34.229434 2693 operation_generator.go:686] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd06fd2b-c200-11e8-aeb2-08002786b439-default-token-szpjq" (OuterVolumeSpecName: "default-token-szpjq") pod "dd06fd2b-c200-11e8-aeb2-08002786b439" (UID: "dd06fd2b-c200-11e8-aeb2-08002786b439"). InnerVolumeSpecName "default-token-szpjq". PluginName "kubernetes.io/secret", VolumeGidValue "" Sep 27 03:00:34 minikube kubelet[2693]: I0927 03:00:34.266881 2693 reconciler.go:285] Volume detached for volume "kube-dns-config" (UniqueName: "kubernetes.io/configmap/dd06fd2b-c200-11e8-aeb2-08002786b439-kube-dns-config") on node "minikube" DevicePath "" Sep 27 03:00:34 minikube kubelet[2693]: I0927 03:00:34.272328 2693 operation_generator.go:686] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e06a81a7-c200-11e8-aeb2-08002786b439-kube-dns-config" (OuterVolumeSpecName: "kube-dns-config") pod "e06a81a7-c200-11e8-aeb2-08002786b439" (UID: "e06a81a7-c200-11e8-aeb2-08002786b439"). InnerVolumeSpecName "kube-dns-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Sep 27 03:00:34 minikube kubelet[2693]: I0927 03:00:34.384200 2693 reconciler.go:285] Volume detached for volume "kube-dns-config" (UniqueName: "kubernetes.io/configmap/e06a81a7-c200-11e8-aeb2-08002786b439-kube-dns-config") on node "minikube" DevicePath "" Sep 27 03:00:34 minikube kubelet[2693]: I0927 03:00:34.384243 2693 reconciler.go:285] Volume detached for volume "default-token-szpjq" (UniqueName: "kubernetes.io/secret/e06a81a7-c200-11e8-aeb2-08002786b439-default-token-szpjq") on node "minikube" DevicePath "" Sep 27 03:00:34 minikube kubelet[2693]: I0927 03:00:34.384253 2693 reconciler.go:285] Volume detached for volume "default-token-szpjq" (UniqueName: "kubernetes.io/secret/dd06fd2b-c200-11e8-aeb2-08002786b439-default-token-szpjq") on node "minikube" DevicePath "" Sep 27 03:00:34 minikube kubelet[2693]: E0927 03:00:34.824526 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:35 minikube kubelet[2693]: E0927 03:00:35.844171 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:36 minikube kubelet[2693]: E0927 03:00:36.923373 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:38 minikube kubelet[2693]: E0927 03:00:38.559460 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:40 minikube kubelet[2693]: E0927 03:00:40.894510 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:41 minikube kubelet[2693]: E0927 03:00:41.486256 2693 reflector.go:322] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to watch v1.Service: Get https://localhost:8443/api/v1/services?resourceVersion=1&timeoutSeconds=592&watch=true: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:41 minikube kubelet[2693]: E0927 03:00:41.728851 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:00:42 minikube kubelet[2693]: E0927 03:00:42.033849 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:42 minikube kubelet[2693]: E0927 03:00:42.530761 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:43 minikube kubelet[2693]: E0927 03:00:43.554232 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:43 minikube kubelet[2693]: E0927 03:00:43.899552 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:44 minikube kubelet[2693]: E0927 03:00:44.757053 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:45 minikube kubelet[2693]: E0927 03:00:45.248817 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:45 minikube kubelet[2693]: E0927 03:00:45.824359 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:46 minikube kubelet[2693]: E0927 03:00:46.348604 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:47 minikube kubelet[2693]: E0927 03:00:47.003927 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:47 minikube kubelet[2693]: E0927 03:00:47.468295 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:48 minikube kubelet[2693]: E0927 03:00:48.023687 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:48 minikube kubelet[2693]: E0927 03:00:48.513463 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:49 minikube kubelet[2693]: E0927 03:00:49.162455 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:49 minikube kubelet[2693]: E0927 03:00:49.867997 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:50 minikube kubelet[2693]: E0927 03:00:50.330732 2693 event.go:209] Unable to write event: 'Post https://localhost:8443/api/v1/namespaces/default/events: http2: server sent GOAWAY and closed the connection; LastStreamID=9, ErrCode=NO_ERROR, debug=""; some request body already written' (may retry after sleeping) Sep 27 03:00:50 minikube kubelet[2693]: E0927 03:00:50.381199 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:50 minikube kubelet[2693]: E0927 03:00:50.404084 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: http2: server sent GOAWAY and closed the connection; LastStreamID=9, ErrCode=NO_ERROR, debug=""; some request body already written Sep 27 03:00:50 minikube kubelet[2693]: E0927 03:00:50.412634 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: http2: server sent GOAWAY and closed the connection; LastStreamID=9, ErrCode=NO_ERROR, debug="" Sep 27 03:00:50 minikube kubelet[2693]: E0927 03:00:50.880203 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:51 minikube kubelet[2693]: I0927 03:00:51.208715 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:00:51 minikube kubelet[2693]: E0927 03:00:51.383603 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:51 minikube kubelet[2693]: E0927 03:00:51.422489 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:51 minikube kubelet[2693]: W0927 03:00:51.694732 2693 docker_sandbox.go:353] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kube-dns-6dcb57bcc8-2d2xv through plugin: invalid network status for Sep 27 03:00:51 minikube kubelet[2693]: E0927 03:00:51.758988 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:00:51 minikube kubelet[2693]: E0927 03:00:51.891451 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:51 minikube kubelet[2693]: W0927 03:00:51.933396 2693 docker_sandbox.go:353] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kube-dns-6dcb57bcc8-hjhss through plugin: invalid network status for Sep 27 03:00:52 minikube kubelet[2693]: E0927 03:00:52.391634 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:52 minikube kubelet[2693]: E0927 03:00:52.424323 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:52 minikube kubelet[2693]: E0927 03:00:52.892760 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:53 minikube kubelet[2693]: W0927 03:00:53.018155 2693 pod_container_deletor.go:77] Container "3a594cc0397b0b0ee75508c3dfab3ee4707a2f5b7f866742c9e7c707284e1bd7" not found in pod's containers Sep 27 03:00:53 minikube kubelet[2693]: I0927 03:00:53.158385 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:00:53 minikube kubelet[2693]: I0927 03:00:53.158816 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:00:53 minikube kubelet[2693]: W0927 03:00:53.166707 2693 pod_container_deletor.go:77] Container "d0e4c858884df19a124cb4dbf1239e43c65127176d4b2a61ed3dd36f5b048e00" not found in pod's containers Sep 27 03:00:53 minikube kubelet[2693]: W0927 03:00:53.166860 2693 pod_container_deletor.go:77] Container "f70e6a1ba48d36a624230f468a9b8588daed2b6b480d2ca62aae83664d15fbac" not found in pod's containers Sep 27 03:00:53 minikube kubelet[2693]: W0927 03:00:53.182213 2693 status_manager.go:461] Failed to get status for pod "kube-apiserver-minikube_kube-system(09eddb205ac749afd96943c8b76939b8)": Get https://localhost:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-minikube: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:00:54 minikube kubelet[2693]: I0927 03:00:54.178737 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:00:56 minikube kubelet[2693]: I0927 03:00:56.479181 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:00:57 minikube kubelet[2693]: I0927 03:00:57.405656 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:00:57 minikube kubelet[2693]: I0927 03:00:57.504334 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 03:01:00 minikube kubelet[2693]: I0927 03:01:00.483318 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:01:01 minikube kubelet[2693]: E0927 03:01:01.780556 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:01:03 minikube kubelet[2693]: E0927 03:01:03.417321 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:01:03 minikube kubelet[2693]: E0927 03:01:03.543245 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:01:03 minikube kubelet[2693]: I0927 03:01:03.588638 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:01:04 minikube kubelet[2693]: E0927 03:01:04.266686 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:01:07 minikube kubelet[2693]: E0927 03:01:07.526033 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: net/http: TLS handshake timeout Sep 27 03:01:10 minikube kubelet[2693]: E0927 03:01:10.653542 2693 event.go:209] Unable to write event: 'Post https://localhost:8443/api/v1/namespaces/default/events: net/http: TLS handshake timeout' (may retry after sleeping) Sep 27 03:01:11 minikube kubelet[2693]: E0927 03:01:11.897030 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:01:14 minikube kubelet[2693]: I0927 03:01:14.529800 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:01:14 minikube kubelet[2693]: E0927 03:01:14.566279 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:01:14 minikube kubelet[2693]: E0927 03:01:14.777893 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:01:15 minikube kubelet[2693]: I0927 03:01:15.059856 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 03:01:15 minikube kubelet[2693]: E0927 03:01:15.318202 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:01:21 minikube kubelet[2693]: E0927 03:01:21.902417 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:01:25 minikube kubelet[2693]: E0927 03:01:25.585836 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: net/http: TLS handshake timeout Sep 27 03:01:25 minikube kubelet[2693]: E0927 03:01:25.869913 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:01:25 minikube kubelet[2693]: E0927 03:01:25.998440 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:01:26 minikube kubelet[2693]: E0927 03:01:26.754800 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:01:31 minikube kubelet[2693]: E0927 03:01:31.078843 2693 event.go:209] Unable to write event: 'Post https://localhost:8443/api/v1/namespaces/default/events: net/http: TLS handshake timeout' (may retry after sleeping) Sep 27 03:01:32 minikube kubelet[2693]: E0927 03:01:32.003116 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:01:32 minikube kubelet[2693]: I0927 03:01:32.592530 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:01:34 minikube kubelet[2693]: I0927 03:01:34.030361 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 03:01:37 minikube kubelet[2693]: E0927 03:01:37.011589 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:01:37 minikube kubelet[2693]: E0927 03:01:37.086981 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:01:37 minikube kubelet[2693]: E0927 03:01:37.805502 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:01:42 minikube kubelet[2693]: E0927 03:01:42.055357 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:01:44 minikube kubelet[2693]: E0927 03:01:44.251791 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: net/http: TLS handshake timeout Sep 27 03:01:48 minikube kubelet[2693]: E0927 03:01:48.321293 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:01:48 minikube kubelet[2693]: E0927 03:01:48.372480 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:01:50 minikube kubelet[2693]: E0927 03:01:50.769321 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:01:51 minikube kubelet[2693]: I0927 03:01:51.348521 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:01:51 minikube kubelet[2693]: E0927 03:01:51.356392 2693 event.go:209] Unable to write event: 'Post https://localhost:8443/api/v1/namespaces/default/events: net/http: TLS handshake timeout' (may retry after sleeping) Sep 27 03:01:52 minikube kubelet[2693]: E0927 03:01:52.136204 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:01:55 minikube kubelet[2693]: I0927 03:01:55.039616 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 03:01:59 minikube kubelet[2693]: I0927 03:01:59.258666 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:01:59 minikube kubelet[2693]: E0927 03:01:59.642532 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:01:59 minikube kubelet[2693]: E0927 03:01:59.599590 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:02:02 minikube kubelet[2693]: E0927 03:02:02.160384 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:02:03 minikube kubelet[2693]: E0927 03:02:03.505322 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:02:05 minikube kubelet[2693]: E0927 03:02:05.326384 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: net/http: TLS handshake timeout Sep 27 03:02:10 minikube kubelet[2693]: I0927 03:02:10.112377 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:02:10 minikube kubelet[2693]: E0927 03:02:10.836681 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:02:10 minikube kubelet[2693]: E0927 03:02:10.989063 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:02:11 minikube kubelet[2693]: E0927 03:02:11.416639 2693 event.go:209] Unable to write event: 'Post https://localhost:8443/api/v1/namespaces/default/events: net/http: TLS handshake timeout' (may retry after sleeping) Sep 27 03:02:12 minikube kubelet[2693]: E0927 03:02:12.179218 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:02:12 minikube kubelet[2693]: I0927 03:02:12.347919 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:02:12 minikube kubelet[2693]: I0927 03:02:12.410064 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 03:02:14 minikube kubelet[2693]: E0927 03:02:14.583625 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:02:19 minikube kubelet[2693]: I0927 03:02:19.518034 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:02:22 minikube kubelet[2693]: E0927 03:02:22.185090 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:02:24 minikube kubelet[2693]: I0927 03:02:24.506736 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:02:27 minikube kubelet[2693]: I0927 03:02:27.162968 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:02:32 minikube kubelet[2693]: E0927 03:02:32.266603 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:02:33 minikube kubelet[2693]: E0927 03:02:33.185095 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: http2: no cached connection was available Sep 27 03:02:33 minikube kubelet[2693]: E0927 03:02:33.250670 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: http2: Transport received Server's graceful shutdown GOAWAY; some request body already written Sep 27 03:02:33 minikube kubelet[2693]: I0927 03:02:33.542298 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:02:35 minikube kubelet[2693]: E0927 03:02:35.133691 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:02:36 minikube kubelet[2693]: E0927 03:02:36.612520 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: an error on the server ("apiserver is shutting down.") has prevented the request from succeeding (get services) Sep 27 03:02:36 minikube kubelet[2693]: E0927 03:02:36.628086 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: an error on the server ("apiserver is shutting down.") has prevented the request from succeeding (get pods) Sep 27 03:02:37 minikube kubelet[2693]: E0927 03:02:37.692971 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:02:37 minikube kubelet[2693]: E0927 03:02:37.746167 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:02:38 minikube kubelet[2693]: E0927 03:02:38.816645 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:02:39 minikube kubelet[2693]: E0927 03:02:38.900548 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:02:39 minikube kubelet[2693]: E0927 03:02:39.869666 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:02:40 minikube kubelet[2693]: E0927 03:02:39.926276 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:02:40 minikube kubelet[2693]: I0927 03:02:40.258608 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:02:41 minikube kubelet[2693]: E0927 03:02:41.168030 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:02:41 minikube kubelet[2693]: I0927 03:02:41.254474 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 03:02:41 minikube kubelet[2693]: E0927 03:02:41.311259 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:02:41 minikube kubelet[2693]: E0927 03:02:41.366647 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:02:42 minikube kubelet[2693]: E0927 03:02:42.180226 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: http2: server sent GOAWAY and closed the connection; LastStreamID=3, ErrCode=NO_ERROR, debug="" Sep 27 03:02:42 minikube kubelet[2693]: E0927 03:02:42.194683 2693 event.go:209] Unable to write event: 'Post https://localhost:8443/api/v1/namespaces/default/events: http2: server sent GOAWAY and closed the connection; LastStreamID=3, ErrCode=NO_ERROR, debug=""; some request body already written' (may retry after sleeping) Sep 27 03:02:42 minikube kubelet[2693]: E0927 03:02:42.239486 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:02:42 minikube kubelet[2693]: E0927 03:02:42.277586 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:02:42 minikube kubelet[2693]: E0927 03:02:42.332414 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:02:43 minikube kubelet[2693]: I0927 03:02:43.044464 2693 kuberuntime_manager.go:757] checking backoff for container "kube-apiserver" in pod "kube-apiserver-minikube_kube-system(09eddb205ac749afd96943c8b76939b8)" Sep 27 03:02:43 minikube kubelet[2693]: E0927 03:02:43.220571 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:02:43 minikube kubelet[2693]: E0927 03:02:43.246249 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:02:43 minikube kubelet[2693]: E0927 03:02:43.341520 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8443: getsockopt: connection refused Sep 27 03:02:44 minikube kubelet[2693]: I0927 03:02:44.072448 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:02:44 minikube kubelet[2693]: I0927 03:02:44.072544 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:02:45 minikube kubelet[2693]: I0927 03:02:45.135874 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:02:48 minikube kubelet[2693]: I0927 03:02:48.370559 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:02:48 minikube kubelet[2693]: I0927 03:02:48.428009 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 03:02:52 minikube kubelet[2693]: E0927 03:02:52.395456 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:02:54 minikube kubelet[2693]: W0927 03:02:54.227712 2693 status_manager.go:461] Failed to get status for pod "kube-apiserver-minikube_kube-system(09eddb205ac749afd96943c8b76939b8)": Get https://localhost:8443/api/v1/namespaces/kube-system/pods/kube-apiserver-minikube: net/http: TLS handshake timeout Sep 27 03:02:54 minikube kubelet[2693]: E0927 03:02:54.386814 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:02:54 minikube kubelet[2693]: E0927 03:02:54.470930 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:02:54 minikube kubelet[2693]: E0927 03:02:54.529133 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:02:58 minikube kubelet[2693]: E0927 03:02:58.450069 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: net/http: TLS handshake timeout Sep 27 03:03:02 minikube kubelet[2693]: E0927 03:03:02.663699 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:03:02 minikube kubelet[2693]: E0927 03:03:02.732567 2693 event.go:209] Unable to write event: 'Post https://localhost:8443/api/v1/namespaces/default/events: net/http: TLS handshake timeout' (may retry after sleeping) Sep 27 03:03:03 minikube kubelet[2693]: E0927 03:03:02.732619 2693 event.go:144] Unable to write event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"minikube.155822664a293669", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(time.Location)(nil)}}, DeletionTimestamp:(v1.Time)(nil), DeletionGracePeriodSeconds:(int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"minikube", UID:"minikube", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"Starting", Message:"Starting kubelet.", Source:v1.EventSource{Component:"kubelet", Host:"minikube"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbee32fb1dc353069, ext:465029178, loc:(time.Location)(0x5b98020)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbee32fb1dc353069, ext:465029178, loc:(time.Location)(0x5b98020)}}, Count:1, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(time.Location)(nil)}}, Series:(v1.EventSeries)(nil), Action:"", Related:(v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}' (retry limit exceeded!) Sep 27 03:03:05 minikube kubelet[2693]: I0927 03:03:05.464421 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:03:05 minikube kubelet[2693]: E0927 03:03:05.780822 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:03:05 minikube kubelet[2693]: E0927 03:03:05.929957 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:03:05 minikube kubelet[2693]: E0927 03:03:05.931825 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:03:06 minikube kubelet[2693]: I0927 03:03:06.190672 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 03:03:10 minikube kubelet[2693]: I0927 03:03:10.657455 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:03:12 minikube kubelet[2693]: E0927 03:03:12.749900 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:03:14 minikube kubelet[2693]: E0927 03:03:14.221474 2693 event.go:209] Unable to write event: 'Post https://localhost:8443/api/v1/namespaces/default/events: net/http: TLS handshake timeout' (may retry after sleeping) Sep 27 03:03:16 minikube kubelet[2693]: E0927 03:03:16.556456 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: net/http: TLS handshake timeout Sep 27 03:03:17 minikube kubelet[2693]: E0927 03:03:17.228423 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:03:17 minikube kubelet[2693]: E0927 03:03:17.295483 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:03:17 minikube kubelet[2693]: E0927 03:03:17.307397 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:03:24 minikube kubelet[2693]: I0927 03:03:24.621775 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:03:24 minikube kubelet[2693]: E0927 03:03:24.926261 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:03:29 minikube kubelet[2693]: I0927 03:03:29.435157 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 03:03:32 minikube kubelet[2693]: E0927 03:03:32.815828 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:03:33 minikube kubelet[2693]: E0927 03:03:33.346923 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:03:33 minikube kubelet[2693]: E0927 03:03:33.802592 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:03:34 minikube kubelet[2693]: E0927 03:03:34.168490 2693 event.go:209] Unable to write event: 'Post https://localhost:8443/api/v1/namespaces/default/events: net/http: TLS handshake timeout' (may retry after sleeping) Sep 27 03:03:34 minikube kubelet[2693]: E0927 03:03:34.978363 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:03:37 minikube kubelet[2693]: I0927 03:03:37.023180 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:03:39 minikube kubelet[2693]: I0927 03:03:39.618138 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:03:40 minikube kubelet[2693]: E0927 03:03:40.239856 2693 kubelet_node_status.go:106] Unable to register node "minikube" with API server: Post https://localhost:8443/api/v1/nodes: net/http: TLS handshake timeout Sep 27 03:03:40 minikube kubelet[2693]: I0927 03:03:40.529966 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:03:44 minikube kubelet[2693]: E0927 03:03:44.016006 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list v1.Pod: Get https://localhost:8443/api/v1/pods?fieldSelector=spec.nodeName%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:03:45 minikube kubelet[2693]: E0927 03:03:45.135295 2693 eviction_manager.go:246] eviction manager: failed to get get summary stats: failed to get node info: node "minikube" not found Sep 27 03:03:45 minikube kubelet[2693]: E0927 03:03:45.449170 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:460: Failed to list v1.Node: Get https://localhost:8443/api/v1/nodes?fieldSelector=metadata.name%3Dminikube&limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:03:45 minikube kubelet[2693]: E0927 03:03:45.454465 2693 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/kubelet.go:451: Failed to list *v1.Service: Get https://localhost:8443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout Sep 27 03:03:50 minikube kubelet[2693]: I0927 03:03:49.371076 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:03:51 minikube kubelet[2693]: I0927 03:03:51.173048 2693 kubelet_node_status.go:82] Attempting to register node minikube Sep 27 03:03:51 minikube kubelet[2693]: I0927 03:03:51.203089 2693 kubelet_node_status.go:271] Setting node annotation to enable volume controller attach/detach Sep 27 03:03:54 minikube kubelet[2693]: E0927 03:03:54.575688 2693 event.go:209] Unable to write event: 'Post https://localhost:8443/api/v1/namespaces/default/events: net/http: TLS handshake timeout' (may retry after sleeping)

SYip commented 6 years ago

Update. It works after:

minikube delete && minikube start

abdennour commented 6 years ago

Bad issue! The longest issue in my software life 😓. If you consider minikube delete && minikube start is a fix, tell me how to specify kubernetes version without facing the same error? Never ever work... I mean minikube delete && minikube start --kubernetes-version v1.11.3

vmudimupersonal commented 6 years ago

I am on a macOS 10.12.6 and Kubernetes v1.10.0.

I found if I use control C to interrupt the minikube start process on next startup the process hangs and I have to delete the local cluster but if I use minikube stop the next start works fine. In my case being diligent about using: minikube stop to stop the cluster keeps me having to experience the issue

Yaxian commented 6 years ago

hello, guys: I record a resolution to run minikube for macOS, you can try it for free.

crazypig13 commented 6 years ago

I tried @Yaxian 's solution, and it works perfectly, thanks a lot!

ferry77 commented 6 years ago

I also tried @Yaxian solution and works. thanks!

However if you had error doing sudo /usr/bin/kubeadm init --config /var/lib/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests --ignore-preflight-errors=DirAvailable--data-minikube --ignore-preflight-errors=Port-10250 --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml --ignore-preflight-errors=FileAvailable--etc-kubernetes-manifests-etcd.yaml --ignore-preflight-errors=Swap --ignore-preflight-errors=CRI

Try to do below beforehand sudo kubeadm reset

And if you got below issue kubectl version Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.0", GitCommit:"0ed33881dc4355495f623c6f22e7dd0b7632b7c0", GitTreeState:"clean", BuildDate:"2018-09-28T15:18:13Z", GoVersion:"go1.11", Compiler:"gc", Platform:"darwin/amd64"} Unable to connect to the server: Forbidden

It because proxy and this helps https://stackoverflow.com/questions/41482485/kubectl-behind-a-proxy

PaulWaldo commented 5 years ago

minikube delete fixed the problem for me

351zyf commented 5 years ago

use v0.25.x

balintant commented 5 years ago

Just had the same issue with the hyperkit driver and it seems to be conflicting with my dns setup. After running brew services stop dnsmasq, it worked smoothly.

muratzorer commented 5 years ago

MacOS v10.14.2 Virtualbox v5.2.24 r128163 (Qt5.6.3) Yes minikube delete then minikube start works. But when I want to continue from stopped VM I also have this issue. Then I realized that if I remove certificate configuration lines (cert-file & cert-key) from the command:

minikube start --memory=4096 --cpus=4 --kubernetes-version=v1.10.0 \
    --extra-config=controller-manager.cluster-signing-cert-file="/var/lib/localkube/certs/ca.crt" \
    --extra-config=controller-manager.cluster-signing-key-file="/var/lib/localkube/certs/ca.key" \
    --extra-config=apiserver.admission-control="NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota" \
    --vm-driver=virtualbox

It works, but as expected, when I check ~/.minikube/profiles/minikube/config.json I do not see these lines anymore:

{
        "Component": "controller-manager",
        "Key": "cluster-signing-cert-file",
        "Value": "/var/lib/localkube/certs/ca.crt"
},
{
        "Component": "controller-manager",
        "Key": "cluster-signing-key-file",
        "Value": "/var/lib/localkube/certs/ca.key"
}

I think @thebetterjort is correct, it is related with certificates. I added the recurring part of the error log. minikube.log

tdensmore commented 5 years ago

Upgraded to the latest minikube and failing to start: minikube start --v=5

gives me the following error:

Starting local Kubernetes v1.13.2 cluster...
.
.
.
[init] Using Kubernetes version: v1.13.2
[preflight] Running pre-flight checks
    [WARNING Swap]: running with swap on is not supported. Please disable swap
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/var/lib/minikube/certs/"
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [minikube localhost] and IPs [192.168.99.102 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [minikube localhost] and IPs [192.168.99.102 127.0.0.1 ::1]
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 28.007920 seconds
[uploadconfig] storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.13" in namespace kube-system with the configuration for the kubelets in the cluster
[patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "minikube" as an annotation
[kubelet-check] Initial timeout of 40s passed.
error execution phase upload-config/kubelet: Error writing Crisocket information for the control-plane node: timed out waiting for the condition

: Process exited with status 1

minikube failed :( exiting with error code 1

NOTE: --bootstrapper=localkube does not work:

minikube start --bootstrapper=localkube --v=5                                                                                                                      
Starting local Kubernetes v1.13.2 cluster...
Starting VM...
Waiting for SSH to be available...
Detecting the provisioner...
Setting Docker configuration on the remote daemon...
Getting VM IP address...
F0208 11:04:19.406016   99727 start.go:279] Error getting cluster bootstrapper: Unknown bootstrapper: localkube
tdensmore commented 5 years ago

I needed to remove minikube and reinstall using brew...

mightyhorst commented 5 years ago

Mine came down to proxy problems on my host.

test this by:

kubectl get pods

if this echos Unable to connect to the server: proxyconnect there's your issue

try echo 

echo $HTTP_PROXY
echo $HTTPS_PROXY

make sure these are either (1) not set or (2) correctly configured for your network.

stupidoes-day commented 5 years ago

I ran into this and had to use localkube as the bootstrapper instead ( #2765 (comment) )

minikube start --bootstrapper=localkube

I am running High Sierra (10.13.2) but shouldn't matter since this seemed to fix the issue for a lot of people

it works for me thanks

vikrantgoel commented 5 years ago

Issues be related to Cisco AnyConnect as well, like was for me. (tldr; reboot your computer) https://github.com/kubernetes/minikube/issues/3503

krustowski commented 1 year ago

Issues be related to Cisco AnyConnect as well, like was for me. (tldr; reboot your computer) #3503

lmao thank you, such a simple solution