search

kubernetes / minikube

Run Kubernetes locally
https://minikube.sigs.k8s.io/
Apache License 2.0
29.31k stars 4.88k forks source link

Don't work resolve dns name. #2350

Closed sah4ez closed 6 years ago

sah4ez commented 6 years ago

BUG REPORT

Environment:

Minikube version (use minikube version): minikube version: v0.24.1

sidecar

kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c sidecar
ERROR: logging before flag.Parse: I1221 13:49:25.085555       1 main.go:48] Version v1.14.4-2-g5584e04
ERROR: logging before flag.Parse: I1221 13:49:25.085647       1 server.go:45] Starting server (options {DnsMasqPort:53 DnsMasqAddr:127.0.0.1 DnsMasqPollIntervalMs:5000 Probes:[{Label:kubedns Server:127.0.0.1:10053 Name:kubernetes.default.svc.cluster.local. Interval:5s Type:1} {Label:dnsmasq Server:127.0.0.1:53 Name:kubernetes.default.svc.cluster.local. Interval:5s Type:1}] PrometheusAddr:0.0.0.0 PrometheusPort:10054 PrometheusPath:/metrics PrometheusNamespace:kubedns})
ERROR: logging before flag.Parse: I1221 13:49:25.085854       1 dnsprobe.go:75] Starting dnsProbe {Label:kubedns Server:127.0.0.1:10053 Name:kubernetes.default.svc.cluster.local. Interval:5s Type:1}
ERROR: logging before flag.Parse: I1221 13:49:25.086013       1 dnsprobe.go:75] Starting dnsProbe {Label:dnsmasq Server:127.0.0.1:53 Name:kubernetes.default.svc.cluster.local. Interval:5s Type:1}

dnsmasq

kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c dnsmasq
I1221 13:49:24.134834       1 main.go:76] opts: {{/usr/sbin/dnsmasq [-k --cache-size=1000 --log-facility=- --server=/cluster.local/127.0.0.1#10053 --server=/in-addr.arpa/127.0.0.1#10053 --server=/ip6.arpa/127.0.0.1#10053] true} /etc/k8s/dns/dnsmasq-nanny 10000000000}
I1221 13:49:24.135086       1 nanny.go:86] Starting dnsmasq [-k --cache-size=1000 --log-facility=- --server=/cluster.local/127.0.0.1#10053 --server=/in-addr.arpa/127.0.0.1#10053 --server=/ip6.arpa/127.0.0.1#10053]
I1221 13:49:24.353157       1 nanny.go:111] 
W1221 13:49:24.353184       1 nanny.go:112] Got EOF from stdout
I1221 13:49:24.353308       1 nanny.go:108] dnsmasq[10]: started, version 2.78-security-prerelease cachesize 1000
I1221 13:49:24.353340       1 nanny.go:108] dnsmasq[10]: compile time options: IPv6 GNU-getopt no-DBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth no-DNSSEC loop-detect inotify
I1221 13:49:24.353364       1 nanny.go:108] dnsmasq[10]: using nameserver 127.0.0.1#10053 for domain ip6.arpa 
I1221 13:49:24.353385       1 nanny.go:108] dnsmasq[10]: using nameserver 127.0.0.1#10053 for domain in-addr.arpa 
I1221 13:49:24.353419       1 nanny.go:108] dnsmasq[10]: using nameserver 127.0.0.1#10053 for domain cluster.local 
I1221 13:49:24.353457       1 nanny.go:108] dnsmasq[10]: reading /etc/resolv.conf
I1221 13:49:24.353487       1 nanny.go:108] dnsmasq[10]: using nameserver 127.0.0.1#10053 for domain ip6.arpa 
I1221 13:49:24.353514       1 nanny.go:108] dnsmasq[10]: using nameserver 127.0.0.1#10053 for domain in-addr.arpa 
I1221 13:49:24.353534       1 nanny.go:108] dnsmasq[10]: using nameserver 127.0.0.1#10053 for domain cluster.local 
I1221 13:49:24.353554       1 nanny.go:108] dnsmasq[10]: using nameserver 10.110.7.1#53
I1221 13:49:24.353617       1 nanny.go:108] dnsmasq[10]: read /etc/hosts - 7 addresses

kubedns

kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c kubedns
I1221 13:49:23.122626       1 dns.go:48] version: 1.14.4-2-g5584e04
I1221 13:49:23.202663       1 server.go:66] Using configuration read from ConfigMap: kube-system:kube-dns
I1221 13:49:23.202797       1 server.go:113] FLAG: --alsologtostderr="false"
I1221 13:49:23.202924       1 server.go:113] FLAG: --config-dir=""
I1221 13:49:23.202932       1 server.go:113] FLAG: --config-map="kube-dns"
I1221 13:49:23.202936       1 server.go:113] FLAG: --config-map-namespace="kube-system"
I1221 13:49:23.202959       1 server.go:113] FLAG: --config-period="10s"
I1221 13:49:23.203028       1 server.go:113] FLAG: --dns-bind-address="0.0.0.0"
I1221 13:49:23.203042       1 server.go:113] FLAG: --dns-port="10053"
I1221 13:49:23.203082       1 server.go:113] FLAG: --domain="cluster.local."
I1221 13:49:23.203101       1 server.go:113] FLAG: --federations=""
I1221 13:49:23.203107       1 server.go:113] FLAG: --healthz-port="8081"
I1221 13:49:23.203111       1 server.go:113] FLAG: --initial-sync-timeout="1m0s"
I1221 13:49:23.203115       1 server.go:113] FLAG: --kube-master-url=""
I1221 13:49:23.203194       1 server.go:113] FLAG: --kubecfg-file=""
I1221 13:49:23.203198       1 server.go:113] FLAG: --log-backtrace-at=":0"
I1221 13:49:23.203249       1 server.go:113] FLAG: --log-dir=""
I1221 13:49:23.203254       1 server.go:113] FLAG: --log-flush-frequency="5s"
I1221 13:49:23.203277       1 server.go:113] FLAG: --logtostderr="true"
I1221 13:49:23.203281       1 server.go:113] FLAG: --nameservers=""
I1221 13:49:23.203348       1 server.go:113] FLAG: --stderrthreshold="2"
I1221 13:49:23.203369       1 server.go:113] FLAG: --v="2"
I1221 13:49:23.203416       1 server.go:113] FLAG: --version="false"
I1221 13:49:23.203447       1 server.go:113] FLAG: --vmodule=""
I1221 13:49:23.203554       1 server.go:176] Starting SkyDNS server (0.0.0.0:10053)
I1221 13:49:23.203842       1 server.go:198] Skydns metrics enabled (/metrics:10055)
I1221 13:49:23.203858       1 dns.go:147] Starting endpointsController
I1221 13:49:23.203863       1 dns.go:150] Starting serviceController
I1221 13:49:23.204165       1 logs.go:41] skydns: ready for queries on cluster.local. for tcp://0.0.0.0:10053 [rcache 0]
I1221 13:49:23.204175       1 logs.go:41] skydns: ready for queries on cluster.local. for udp://0.0.0.0:10053 [rcache 0]
I1221 13:49:23.555942       1 sync_configmap.go:107] ConfigMap kube-system:kube-dns was created
I1221 13:49:24.054105       1 dns.go:171] Initialized services and endpoints from apiserver
I1221 13:49:24.054128       1 server.go:129] Setting up Healthz Handler (/readiness)
I1221 13:49:24.054206       1 server.go:134] Setting up cache handler (/cache)
I1221 13:49:24.054257       1 server.go:120] Status HTTP port 8081

What happened: Can not ping or traceroute service via service name or IP.

What you expected to happen:

ping to the service via service name.

How to reproduce it (as minimally and precisely as possible):

Default namesapce

$ kc get all
NAME               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/memcached   1         1         1            1           1h
deploy/mongo       1         1         1            1           1h

NAME                      DESIRED   CURRENT   READY     AGE
rs/memcached-64dcdbc9f6   1         1         1         1h
rs/mongo-67d67fddf9       1         1         1         39m
rs/mongo-6fc9bd6d6c       0         0         0         1h

NAME               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/memcached   1         1         1            1           1h
deploy/mongo       1         1         1            1           1h

NAME                      DESIRED   CURRENT   READY     AGE
rs/memcached-64dcdbc9f6   1         1         1         1h
rs/mongo-67d67fddf9       1         1         1         39m
rs/mongo-6fc9bd6d6c       0         0         0         1h

NAME                            READY     STATUS    RESTARTS   AGE
po/busybox                      1/1       Running   0          29m
po/memcached-64dcdbc9f6-j2v97   1/1       Running   0          1h
po/mongo-67d67fddf9-55zgd       1/1       Running   0          39m

NAME             TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)     AGE
svc/kubernetes   ClusterIP   10.96.0.1      <none>        443/TCP     1h
svc/memcached    ClusterIP   10.100.42.68   <none>        55555/TCP   1h
svc/mongo        ClusterIP   10.99.92.189   <none>        27017/TCP   1h

kube-system

$ kc get --namespace=kube-system all
NAME              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/kube-dns   1         1         1            1           1h

NAME                     DESIRED   CURRENT   READY     AGE
rs/kube-dns-86f6f55dd5   1         1         1         1h

NAME              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/kube-dns   1         1         1            1           1h

NAME                     DESIRED   CURRENT   READY     AGE
rs/kube-dns-86f6f55dd5   1         1         1         1h

NAME                             READY     STATUS    RESTARTS   AGE
po/kube-addon-manager-minikube   1/1       Running   1          1h
po/kube-dns-86f6f55dd5-mrtrm     3/3       Running   3          1h
po/kubernetes-dashboard-5sgcl    1/1       Running   1          1h
po/storage-provisioner           1/1       Running   1          1h

NAME                      DESIRED   CURRENT   READY     AGE
rc/kubernetes-dashboard   1         1         1         1h

NAME                       TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)         AGE
svc/kube-dns               ClusterIP   10.96.0.10     <none>        53/UDP,53/TCP   1h
svc/kubernetes-dashboard   NodePort    10.110.68.80   <none>        80:30000/TCP    1h

resolv.conf

$ kc exec -it mongo-67d67fddf9-55zgd -- cat /etc/resolv.conf
nameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:5

nslookup test

$ kc exec -it mongo-67d67fddf9-55zgd nslookup kubernetes
Server:     10.96.0.10
Address:    10.96.0.10#53

Non-authoritative answer:
Name:   kubernetes.default.svc.cluster.local
Address: 10.96.0.1

ping test

$ kc exec -it mongo-67d67fddf9-55zgd -- ping kubernetes
PING kubernetes.default.svc.cluster.local (10.96.0.1): 56 data bytes
64 bytes from 10.96.0.1: icmp_seq=0 ttl=250 time=2.873 ms
64 bytes from 10.96.0.1: icmp_seq=1 ttl=250 time=1.845 ms
64 bytes from 10.96.0.1: icmp_seq=2 ttl=250 time=1.809 ms
64 bytes from 10.96.0.1: icmp_seq=3 ttl=250 time=2.035 ms
64 bytes from 10.96.0.1: icmp_seq=4 ttl=250 time=1.805 ms
--- kubernetes.default.svc.cluster.local ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 1.805/2.073/2.873/0.409 ms

traceroute test (ok)

$ kc exec -it mongo-67d67fddf9-55zgd -- traceroute -n kubernetes
traceroute to kubernetes (10.96.0.1), 30 hops max, 60 byte packets
 1  10.110.7.1  0.207 ms  0.195 ms  0.186 ms
 2  192.168.1.1  0.317 ms  0.392 ms  0.456 ms
 3  10.77.0.1  2.261 ms  2.977 ms  3.755 ms
 4  10.128.132.1  1.568 ms  1.721 ms  1.934 ms
 5  192.168.39.136  2.055 ms  2.329 ms  2.456 ms
 6  10.128.145.2  8.603 ms  8.971 ms  9.391 ms

test nslookup

$ kc exec -it mongo-67d67fddf9-55zgd -- nslookup mongo
Server:     10.96.0.10
Address:    10.96.0.10#53

Name:   mongo.default.svc.cluster.local
Address: 10.99.92.189

test ping

$ kc exec -it mongo-67d67fddf9-55zgd -- ping mongo
PING mongo.default.svc.cluster.local (10.99.92.189): 56 data bytes
--- mongo.default.svc.cluster.local ping statistics ---
210 packets transmitted, 0 packets received, 100% packet loss
command terminated with exit code 1

test traceroute (bad)

$ kc exec -it mongo-67d67fddf9-55zgd -- traceroute -n mongo
traceroute to mongo (10.99.92.189), 30 hops max, 60 byte packets
 1  10.110.7.1  0.228 ms  0.203 ms  0.194 ms
 2  192.168.1.1  0.438 ms  0.519 ms  0.582 ms
 3  10.77.0.1  2.290 ms  3.599 ms  4.396 ms
 4  10.128.132.1  1.851 ms  1.949 ms  2.166 ms
 5  192.168.39.136  2.258 ms  2.421 ms  2.618 ms
 6  10.128.145.5  5.193 ms  6.084 ms  8.301 ms
 7  * * *
 8  * * *
 9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *

traceroute IP (bad)

$ kc exec -it mongo-67d67fddf9-55zgd -- traceroute -n 10.99.92.189
traceroute to 10.99.92.189 (10.99.92.189), 30 hops max, 60 byte packets
 1  10.110.7.1  0.190 ms  0.136 ms  0.124 ms
 2  192.168.1.1  0.431 ms  0.485 ms  0.547 ms
 3  10.77.0.1  2.402 ms  3.256 ms  4.040 ms
 4  10.128.132.1  1.780 ms  1.790 ms  1.930 ms
 5  192.168.39.136  2.214 ms  2.209 ms  2.562 ms
 6  10.128.145.5  7.645 ms  8.028 ms  8.284 ms
 7  * * *
 8  * * *
 9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *

iptables from node

$ sudo iptables-save | grep mongo
-A KUBE-SEP-HYCP7OGZ3WQCZP76 -s 172.17.0.6/32 -m comment --comment "default/mongo:27017" -j KUBE-MARK-MASQ
-A KUBE-SEP-HYCP7OGZ3WQCZP76 -p tcp -m comment --comment "default/mongo:27017" -m tcp -j DNAT --to-destination 172.17.0.6:27017
-A KUBE-SERVICES -d 10.99.92.189/32 -p tcp -m comment --comment "default/mongo:27017 cluster IP" -m tcp --dport 27017 -j KUBE-SVC-VMEO5WN4YXST2YCP
-A KUBE-SVC-VMEO5WN4YXST2YCP -m comment --comment "default/mongo:27017" -j KUBE-SEP-HYCP7OGZ3WQCZP76

mongo-deployment.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    kompose.cmd: kompose convert -f docker-compose.yml
    kompose.version: 1.6.0 (e4adfef)
  creationTimestamp: null
  labels:
    io.kompose.service: mongo
  name: mongo
spec:
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      creationTimestamp: null
      labels:
        io.kompose.service: mongo
    spec:
      containers:
      - image: docker.scnetservices.ru/mongo:dev
        name: mongo
        resources: {}
        volumeMounts:
        - mountPath: /data/db
          name: mongo-claim0
      restartPolicy: Always
      volumes:
      - name: mongo-claim0
        persistentVolumeClaim:
          claimName: mongo-claim0
status: {}

mongo-service.yaml

apiVersion: v1
kind: Service
metadata:
  annotations:
    kompose.cmd: kompose convert -f docker-compose.yml
    kompose.version: 1.6.0 (e4adfef)
  creationTimestamp: null
  labels:
    io.kompose.service: mongo
  name: mongo
spec:
  ports:
  - name: "27017"
    port: 27017
    targetPort: 27017
  selector:
    io.kompose.service: mongo
status:
  loadBalancer: {}

mongo-volume0-persistentvolume.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  creationTimestamp: null
  labels:
    type: local
  name: mongo-volume0
spec:
  storageClassName: manual
  capacity: 
    storage: 5Gi
  accessModes:
  - ReadWriteOnce
  hostPath:
    path: "/home/docker/mongo"
    type: "DirectoryOrCreate"
  persistentVolumeReclaimPolicy: Recycle
  claimRef:
    namespace: default
    name: mongo-claim0

mongo-claim0-persistentvolumeclaim.yaml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: mongo-claim0
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

But if I connect to container mongo via ssh tunneling it work:

$ ssh -fN -l docker -i "~/.minikube/machines/minikube/id_rsa" -L 27017:localhost:27017 $(minikube ip)
sah4ez@PC001:~$ mongo localhost:27017
MongoDB shell version v3.4.9
connecting to: localhost:27017
MongoDB server version: 3.4.2
Server has startup warnings: 
2017-12-21T14:48:20.434+0000 I CONTROL  [initandlisten] 
2017-12-21T14:48:20.434+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-12-21T14:48:20.434+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2017-12-21T14:48:20.434+0000 I CONTROL  [initandlisten] 
> db.hostInfo()
{
    "system" : {
        "currentTime" : ISODate("2017-12-21T16:23:35.940Z"),
        "hostname" : "minikube",
        "cpuAddrSize" : 64,
        "memSizeMB" : 1906,
        "numCores" : 2,
        "cpuArch" : "x86_64",
        "numaEnabled" : false
    },
    "os" : {
        "type" : "Linux",
        "name" : "PRETTY_NAME=\"Debian GNU/Linux 8 (jessie)\"",
        "version" : "Kernel 4.9.13"
    },
    "extra" : {
        "versionString" : "Linux version 4.9.13 (jenkins@jenkins) (gcc version 5.4.0 (Buildroot 2017.02) ) #1 SMP Thu Oct 19 17:14:00 UTC 2017",
        "libcVersion" : "2.19",
        "kernelVersion" : "4.9.13",
        "cpuFrequencyMHz" : "2993.200",
        "cpuFeatures" : "fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm rep_good nopl eagerfpu pni vmx cx16 x2apic hypervisor lahf_lm tpr_shadow vnmi flexpriority ept vpid",
        "pageSize" : NumberLong(4096),
        "numPages" : 487940,
        "maxOpenFiles" : 65536
    },
    "ok" : 1
}

My OS info:

sah4ez@PC001:~$ uname -a
Linux PC001 4.8.0-58-generic #63~16.04.1-Ubuntu SMP Mon Jun 26 18:08:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

In example up mongo connected to minikube and Kernel 4.9.13...

mo-hit commented 6 years ago

Can also confirm that we are also seeing this. Setup is as follows: K8s 1.8 minikube 0.24.1 ISO version tried both 0.23.6 and 0.23.5 Running on OS X with the hyperkit driver

sah4ez commented 6 years ago

minikube logs

-- Logs begin at Fri 2017-12-22 07:34:16 UTC, end at Fri 2017-12-22 07:42:45 UTC. --
Dec 22 07:34:45 minikube systemd[1]: Starting Localkube...
Dec 22 07:34:45 minikube localkube[2818]: listening for peers on http://localhost:2380
Dec 22 07:34:45 minikube localkube[2818]: listening for client requests on localhost:2379
Dec 22 07:34:45 minikube localkube[2818]: recovered store from snapshot at index 20002
Dec 22 07:34:45 minikube localkube[2818]: name = default
Dec 22 07:34:45 minikube localkube[2818]: data dir = /var/lib/localkube/etcd
Dec 22 07:34:45 minikube localkube[2818]: member dir = /var/lib/localkube/etcd/member
Dec 22 07:34:45 minikube localkube[2818]: heartbeat = 100ms
Dec 22 07:34:45 minikube localkube[2818]: election = 1000ms
Dec 22 07:34:45 minikube localkube[2818]: snapshot count = 10000
Dec 22 07:34:45 minikube localkube[2818]: advertise client URLs = http://localhost:2379
Dec 22 07:34:46 minikube localkube[2818]: restarting member 8e9e05c52164694d in cluster cdf818194e3a8c32 at commit index 20080
Dec 22 07:34:46 minikube localkube[2818]: 8e9e05c52164694d became follower at term 5
Dec 22 07:34:46 minikube localkube[2818]: newRaft 8e9e05c52164694d [peers: [8e9e05c52164694d], term: 5, commit: 20080, applied: 20002, lastindex: 20080, lastterm: 5]
Dec 22 07:34:46 minikube localkube[2818]: enabled capabilities for version 3.1
Dec 22 07:34:46 minikube localkube[2818]: added member 8e9e05c52164694d [http://localhost:2380] to cluster cdf818194e3a8c32 from store
Dec 22 07:34:46 minikube localkube[2818]: set the cluster version to 3.1 from store
Dec 22 07:34:46 minikube localkube[2818]: restore compact to 18513
Dec 22 07:34:46 minikube localkube[2818]: starting server... [version: 3.1.10, cluster version: 3.1]
Dec 22 07:34:46 minikube localkube[2818]: 8e9e05c52164694d is starting a new election at term 5
Dec 22 07:34:46 minikube localkube[2818]: 8e9e05c52164694d became candidate at term 6
Dec 22 07:34:46 minikube localkube[2818]: 8e9e05c52164694d received MsgVoteResp from 8e9e05c52164694d at term 6
Dec 22 07:34:46 minikube localkube[2818]: 8e9e05c52164694d became leader at term 6
Dec 22 07:34:46 minikube localkube[2818]: raft.node: 8e9e05c52164694d elected leader 8e9e05c52164694d at term 6
Dec 22 07:34:46 minikube localkube[2818]: published {Name:default ClientURLs:[http://localhost:2379]} to cluster cdf818194e3a8c32
Dec 22 07:34:46 minikube localkube[2818]: I1222 07:34:46.441011    2818 etcd.go:58] Etcd server is ready
Dec 22 07:34:46 minikube localkube[2818]: ready to serve client requests
Dec 22 07:34:46 minikube localkube[2818]: localkube host ip address: 10.110.7.2
Dec 22 07:34:46 minikube localkube[2818]: serving insecure client requests on 127.0.0.1:2379, this is strongly discouraged!
Dec 22 07:34:46 minikube localkube[2818]: Starting apiserver...
Dec 22 07:34:46 minikube localkube[2818]: Waiting for apiserver to be healthy...
Dec 22 07:34:46 minikube localkube[2818]: I1222 07:34:46.442460    2818 server.go:114] Version: v1.8.0
Dec 22 07:34:46 minikube localkube[2818]: W1222 07:34:46.442847    2818 authentication.go:380] AnonymousAuth is not allowed with the AllowAll authorizer.  Resetting AnonymousAuth to false. You should use a different authorizer
Dec 22 07:34:46 minikube localkube[2818]: I1222 07:34:46.443381    2818 plugins.go:101] No cloud provider specified.
Dec 22 07:34:47 minikube localkube[2818]: [restful] 2017/12/22 07:34:47 log.go:33: [restful/swagger] listing is available at https://10.110.7.2:8443/swaggerapi
Dec 22 07:34:47 minikube localkube[2818]: [restful] 2017/12/22 07:34:47 log.go:33: [restful/swagger] https://10.110.7.2:8443/swaggerui/ is mapped to folder /swagger-ui/
Dec 22 07:34:47 minikube localkube[2818]: I1222 07:34:47.443173    2818 ready.go:30] Performing healthcheck on https://localhost:8443/healthz
Dec 22 07:34:47 minikube localkube[2818]: E1222 07:34:47.444329    2818 ready.go:40] Error performing healthcheck: Get https://localhost:8443/healthz: dial tcp 127.0.0.1:8443: getsockopt: connection refused
Dec 22 07:34:48 minikube localkube[2818]: [restful] 2017/12/22 07:34:48 log.go:33: [restful/swagger] listing is available at https://10.110.7.2:8443/swaggerapi
Dec 22 07:34:48 minikube localkube[2818]: [restful] 2017/12/22 07:34:48 log.go:33: [restful/swagger] https://10.110.7.2:8443/swaggerui/ is mapped to folder /swagger-ui/
Dec 22 07:34:48 minikube localkube[2818]: I1222 07:34:48.442992    2818 ready.go:30] Performing healthcheck on https://localhost:8443/healthz
Dec 22 07:34:48 minikube localkube[2818]: E1222 07:34:48.443960    2818 ready.go:40] Error performing healthcheck: Get https://localhost:8443/healthz: dial tcp 127.0.0.1:8443: getsockopt: connection refused
Dec 22 07:34:49 minikube localkube[2818]: I1222 07:34:49.442982    2818 ready.go:30] Performing healthcheck on https://localhost:8443/healthz
Dec 22 07:34:49 minikube localkube[2818]: E1222 07:34:49.444168    2818 ready.go:40] Error performing healthcheck: Get https://localhost:8443/healthz: dial tcp 127.0.0.1:8443: getsockopt: connection refused
Dec 22 07:34:50 minikube localkube[2818]: I1222 07:34:50.442975    2818 ready.go:30] Performing healthcheck on https://localhost:8443/healthz
Dec 22 07:34:50 minikube localkube[2818]: E1222 07:34:50.443903    2818 ready.go:40] Error performing healthcheck: Get https://localhost:8443/healthz: dial tcp 127.0.0.1:8443: getsockopt: connection refused
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.117704    2818 aggregator.go:138] Skipping APIService creation for scheduling.k8s.io/v1alpha1
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.118451    2818 serve.go:85] Serving securely on 0.0.0.0:8443
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.118623    2818 crd_finalizer.go:242] Starting CRDFinalizer
Dec 22 07:34:51 minikube systemd[1]: Started Localkube.
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.119326    2818 available_controller.go:192] Starting AvailableConditionController
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.119468    2818 cache.go:32] Waiting for caches to sync for AvailableConditionController controller
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.119617    2818 controller.go:84] Starting OpenAPI AggregationController
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.119770    2818 customresource_discovery_controller.go:152] Starting DiscoveryController
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.119980    2818 naming_controller.go:277] Starting NamingConditionController
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.120145    2818 crdregistration_controller.go:112] Starting crd-autoregister controller
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.120274    2818 controller_utils.go:1041] Waiting for caches to sync for crd-autoregister controller
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.121024    2818 apiservice_controller.go:112] Starting APIServiceRegistrationController
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.121244    2818 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller
Dec 22 07:34:51 minikube localkube[2818]: W1222 07:34:51.196055    2818 controller.go:386] Resetting endpoints for master service "kubernetes" to &{{ } {kubernetes  default /api/v1/namespaces/default/endpoints/kubernetes 2464c1a1-e653-11e7-80fb-4417e309dda0 18084 0 2017-12-21 13:30:42 +0000 UTC <nil> <nil> map[] map[] [] nil [] } [{[{10.110.7.2  <nil> <nil>}] [] [{https 8443 TCP}]}]}
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.219715    2818 cache.go:39] Caches are synced for AvailableConditionController controller
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.220925    2818 controller_utils.go:1048] Caches are synced for crd-autoregister controller
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.221149    2818 autoregister_controller.go:136] Starting autoregister controller
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.221165    2818 cache.go:32] Waiting for caches to sync for autoregister controller
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.221481    2818 cache.go:39] Caches are synced for APIServiceRegistrationController controller
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.321313    2818 cache.go:39] Caches are synced for autoregister controller
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.443045    2818 ready.go:30] Performing healthcheck on https://localhost:8443/healthz
Dec 22 07:34:51 minikube localkube[2818]: I1222 07:34:51.455038    2818 ready.go:49] Got healthcheck response: [+]ping ok
Dec 22 07:34:51 minikube localkube[2818]: [+]etcd ok
Dec 22 07:34:51 minikube localkube[2818]: [+]poststarthook/generic-apiserver-start-informers ok
Dec 22 07:34:51 minikube localkube[2818]: [+]poststarthook/start-apiextensions-informers ok
Dec 22 07:34:51 minikube localkube[2818]: [+]poststarthook/start-apiextensions-controllers ok
Dec 22 07:34:51 minikube localkube[2818]: [+]poststarthook/bootstrap-controller ok
Dec 22 07:34:51 minikube localkube[2818]: [-]poststarthook/ca-registration failed: reason withheld
Dec 22 07:34:51 minikube localkube[2818]: [+]poststarthook/start-kube-apiserver-informers ok
Dec 22 07:34:51 minikube localkube[2818]: [+]poststarthook/start-kube-aggregator-informers ok
Dec 22 07:34:51 minikube localkube[2818]: [+]poststarthook/apiservice-registration-controller ok
Dec 22 07:34:51 minikube localkube[2818]: [+]poststarthook/apiservice-status-available-controller ok
Dec 22 07:34:51 minikube localkube[2818]: [+]poststarthook/apiservice-openapi-controller ok
Dec 22 07:34:51 minikube localkube[2818]: [+]poststarthook/kube-apiserver-autoregistration ok
Dec 22 07:34:51 minikube localkube[2818]: [+]autoregister-completion ok
Dec 22 07:34:51 minikube localkube[2818]: healthz check failed
Dec 22 07:34:52 minikube localkube[2818]: I1222 07:34:52.443050    2818 ready.go:30] Performing healthcheck on https://localhost:8443/healthz
Dec 22 07:34:52 minikube localkube[2818]: I1222 07:34:52.453594    2818 ready.go:49] Got healthcheck response: ok
Dec 22 07:34:52 minikube localkube[2818]: apiserver is ready!
Dec 22 07:34:52 minikube localkube[2818]: Starting controller-manager...
Dec 22 07:34:52 minikube localkube[2818]: Waiting for controller-manager to be healthy...
Dec 22 07:34:52 minikube localkube[2818]: I1222 07:34:52.453675    2818 controllermanager.go:109] Version: v1.8.0
Dec 22 07:34:52 minikube localkube[2818]: I1222 07:34:52.458686    2818 leaderelection.go:174] attempting to acquire leader lease...
Dec 22 07:34:52 minikube localkube[2818]: I1222 07:34:52.471611    2818 leaderelection.go:184] successfully acquired lease kube-system/kube-controller-manager
Dec 22 07:34:52 minikube localkube[2818]: I1222 07:34:52.474235    2818 event.go:218] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"kube-controller-manager", UID:"24f27df0-e653-11e7-80fb-4417e309dda0", APIVersion:"v1", ResourceVersion:"18941", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' minikube became leader
Dec 22 07:34:52 minikube localkube[2818]: I1222 07:34:52.488505    2818 plugins.go:101] No cloud provider specified.
Dec 22 07:34:52 minikube localkube[2818]: I1222 07:34:52.491498    2818 controllermanager.go:487] Started "serviceaccount"
Dec 22 07:34:52 minikube localkube[2818]: E1222 07:34:52.492804    2818 certificates.go:48] Failed to start certificate controller: error reading CA cert file "/etc/kubernetes/ca/ca.pem": open /etc/kubernetes/ca/ca.pem: no such file or directory
Dec 22 07:34:52 minikube localkube[2818]: W1222 07:34:52.492830    2818 controllermanager.go:484] Skipping "csrsigning"
Dec 22 07:34:52 minikube localkube[2818]: I1222 07:34:52.494185    2818 controllermanager.go:487] Started "csrapproving"
Dec 22 07:34:52 minikube localkube[2818]: W1222 07:34:52.494211    2818 controllermanager.go:471] "bootstrapsigner" is disabled
Dec 22 07:34:52 minikube localkube[2818]: E1222 07:34:52.495446    2818 core.go:70] Failed to start service controller: WARNING: no cloud provider provided, services of type LoadBalancer will fail.
Dec 22 07:34:52 minikube localkube[2818]: W1222 07:34:52.495468    2818 controllermanager.go:484] Skipping "service"
Dec 22 07:34:52 minikube localkube[2818]: W1222 07:34:52.495477    2818 controllermanager.go:484] Skipping "persistentvolume-expander"
Dec 22 07:34:52 minikube localkube[2818]: I1222 07:34:52.496672    2818 controllermanager.go:487] Started "endpoint"
Dec 22 07:34:52 minikube localkube[2818]: I1222 07:34:52.497902    2818 endpoints_controller.go:153] Starting endpoint controller
Dec 22 07:34:52 minikube localkube[2818]: I1222 07:34:52.497934    2818 controller_utils.go:1041] Waiting for caches to sync for endpoint controller
Dec 22 07:34:52 minikube localkube[2818]: I1222 07:34:52.497986    2818 controller_utils.go:1041] Waiting for caches to sync for tokens controller
Dec 22 07:34:52 minikube localkube[2818]: I1222 07:34:52.498681    2818 serviceaccounts_controller.go:113] Starting service account controller
Dec 22 07:34:52 minikube localkube[2818]: I1222 07:34:52.498729    2818 controller_utils.go:1041] Waiting for caches to sync for service account controller
Dec 22 07:34:52 minikube localkube[2818]: I1222 07:34:52.498757    2818 certificate_controller.go:109] Starting certificate controller
Dec 22 07:34:52 minikube localkube[2818]: I1222 07:34:52.498764    2818 controller_utils.go:1041] Waiting for caches to sync for certificate controller
Dec 22 07:34:52 minikube localkube[2818]: I1222 07:34:52.598151    2818 controller_utils.go:1048] Caches are synced for tokens controller
Dec 22 07:34:53 minikube localkube[2818]: controller-manager is ready!
Dec 22 07:34:53 minikube localkube[2818]: Starting scheduler...
Dec 22 07:34:53 minikube localkube[2818]: Waiting for scheduler to be healthy...
Dec 22 07:34:53 minikube localkube[2818]: E1222 07:34:53.456763    2818 server.go:173] unable to register configz: register config "componentconfig" twice
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.700313    2818 controllermanager.go:487] Started "garbagecollector"
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.701624    2818 controllermanager.go:487] Started "daemonset"
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.703170    2818 garbagecollector.go:136] Starting garbage collector controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.703400    2818 controller_utils.go:1041] Waiting for caches to sync for garbage collector controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.703645    2818 graph_builder.go:321] GraphBuilder running
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.701831    2818 daemon_controller.go:230] Starting daemon sets controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.703942    2818 controller_utils.go:1041] Waiting for caches to sync for daemon sets controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.704355    2818 controllermanager.go:487] Started "job"
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.704476    2818 job_controller.go:138] Starting job controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.704765    2818 controller_utils.go:1041] Waiting for caches to sync for job controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.708943    2818 controllermanager.go:487] Started "horizontalpodautoscaling"
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.709089    2818 horizontal.go:145] Starting HPA controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.709399    2818 controller_utils.go:1041] Waiting for caches to sync for HPA controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.710545    2818 controllermanager.go:487] Started "disruption"
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.710674    2818 disruption.go:288] Starting disruption controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.710911    2818 controller_utils.go:1041] Waiting for caches to sync for disruption controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.711845    2818 node_controller.go:249] Sending events to api server.
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.712109    2818 taint_controller.go:158] Sending events to api server.
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.712300    2818 controllermanager.go:487] Started "node"
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.712417    2818 node_controller.go:516] Starting node controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.712640    2818 controller_utils.go:1041] Waiting for caches to sync for node controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.713664    2818 controllermanager.go:487] Started "replicationcontroller"
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.713758    2818 replication_controller.go:151] Starting RC controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.714046    2818 controller_utils.go:1041] Waiting for caches to sync for RC controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.716660    2818 controllermanager.go:487] Started "deployment"
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.716898    2818 deployment_controller.go:151] Starting deployment controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.717043    2818 controller_utils.go:1041] Waiting for caches to sync for deployment controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.717901    2818 controllermanager.go:487] Started "statefulset"
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.718171    2818 stateful_set.go:146] Starting stateful set controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.718300    2818 controller_utils.go:1041] Waiting for caches to sync for stateful set controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.719053    2818 controllermanager.go:487] Started "cronjob"
Dec 22 07:34:53 minikube localkube[2818]: W1222 07:34:53.719086    2818 controllermanager.go:471] "tokencleaner" is disabled
Dec 22 07:34:53 minikube localkube[2818]: W1222 07:34:53.719095    2818 core.go:128] Unsuccessful parsing of cluster CIDR : invalid CIDR address:
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.719105    2818 core.go:131] Will not configure cloud provider routes for allocate-node-cidrs: false, configure-cloud-routes: true.
Dec 22 07:34:53 minikube localkube[2818]: W1222 07:34:53.719110    2818 controllermanager.go:484] Skipping "route"
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.719819    2818 cronjob_controller.go:98] Starting CronJob Manager
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.720880    2818 controllermanager.go:487] Started "persistentvolume-binder"
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.722433    2818 pv_controller_base.go:259] Starting persistent volume controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.722461    2818 controller_utils.go:1041] Waiting for caches to sync for persistent volume controller
Dec 22 07:34:53 minikube localkube[2818]: W1222 07:34:53.723147    2818 probe.go:215] Flexvolume plugin directory at /usr/libexec/kubernetes/kubelet-plugins/volume/exec/ does not exist. Recreating.
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.723674    2818 controllermanager.go:487] Started "attachdetach"
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.724893    2818 controllermanager.go:487] Started "podgc"
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.726277    2818 attach_detach_controller.go:255] Starting attach detach controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.726306    2818 controller_utils.go:1041] Waiting for caches to sync for attach detach controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.726337    2818 gc_controller.go:76] Starting GC controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.726342    2818 controller_utils.go:1041] Waiting for caches to sync for GC controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.740233    2818 controllermanager.go:487] Started "namespace"
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.743121    2818 namespace_controller.go:186] Starting namespace controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.743168    2818 controller_utils.go:1041] Waiting for caches to sync for namespace controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.744856    2818 controllermanager.go:487] Started "replicaset"
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.744995    2818 replica_set.go:156] Starting replica set controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.745296    2818 controller_utils.go:1041] Waiting for caches to sync for replica set controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.746531    2818 controllermanager.go:487] Started "ttl"
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.746711    2818 ttl_controller.go:116] Starting TTL controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.746774    2818 controller_utils.go:1041] Waiting for caches to sync for TTL controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.748306    2818 controllermanager.go:487] Started "resourcequota"
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.748552    2818 resource_quota_controller.go:238] Starting resource quota controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.748562    2818 controller_utils.go:1041] Waiting for caches to sync for resource quota controller
Dec 22 07:34:53 minikube localkube[2818]: E1222 07:34:53.779048    2818 actual_state_of_world.go:483] Failed to set statusUpdateNeeded to needed true because nodeName="minikube"  does not exist
Dec 22 07:34:53 minikube localkube[2818]: E1222 07:34:53.779319    2818 actual_state_of_world.go:497] Failed to update statusUpdateNeeded field in actual state of world: Failed to set statusUpdateNeeded to needed true because nodeName="minikube"  does not exist
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.798909    2818 controller_utils.go:1048] Caches are synced for certificate controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.799275    2818 controller_utils.go:1048] Caches are synced for service account controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.804362    2818 controller_utils.go:1048] Caches are synced for daemon sets controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.812933    2818 controller_utils.go:1048] Caches are synced for node controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.813224    2818 taint_controller.go:181] Starting NoExecuteTaintManager
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.813272    2818 node_controller.go:563] Initializing eviction metric for zone:
Dec 22 07:34:53 minikube localkube[2818]: W1222 07:34:53.813603    2818 node_controller.go:916] Missing timestamp for Node minikube. Assuming now as a timestamp.
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.813890    2818 node_controller.go:832] Controller detected that zone  is now in state Normal.
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.813321    2818 event.go:218] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"minikube", UID:"268fedf7-e653-11e7-80fb-4417e309dda0", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RegisteredNode' Node minikube event: Registered Node minikube in Controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.814880    2818 controller_utils.go:1048] Caches are synced for RC controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.818458    2818 controller_utils.go:1048] Caches are synced for stateful set controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.818724    2818 controller_utils.go:1048] Caches are synced for deployment controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.826460    2818 controller_utils.go:1048] Caches are synced for GC controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.826860    2818 controller_utils.go:1048] Caches are synced for attach detach controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.843312    2818 controller_utils.go:1048] Caches are synced for namespace controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.845575    2818 controller_utils.go:1048] Caches are synced for replica set controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.846860    2818 controller_utils.go:1048] Caches are synced for TTL controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.848741    2818 controller_utils.go:1048] Caches are synced for resource quota controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.898028    2818 controller_utils.go:1048] Caches are synced for endpoint controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.911176    2818 controller_utils.go:1048] Caches are synced for disruption controller
Dec 22 07:34:53 minikube localkube[2818]: I1222 07:34:53.911193    2818 disruption.go:296] Sending events to api server.
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.005016    2818 controller_utils.go:1048] Caches are synced for job controller
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.009667    2818 controller_utils.go:1048] Caches are synced for HPA controller
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.024812    2818 controller_utils.go:1048] Caches are synced for persistent volume controller
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.103714    2818 controller_utils.go:1048] Caches are synced for garbage collector controller
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.103759    2818 garbagecollector.go:145] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.157966    2818 controller_utils.go:1041] Waiting for caches to sync for scheduler controller
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.258422    2818 controller_utils.go:1048] Caches are synced for scheduler controller
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.258727    2818 leaderelection.go:174] attempting to acquire leader lease...
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.261553    2818 leaderelection.go:184] successfully acquired lease kube-system/kube-scheduler
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.261806    2818 event.go:218] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"kube-scheduler", UID:"262329e1-e653-11e7-80fb-4417e309dda0", APIVersion:"v1", ResourceVersion:"18944", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' minikube became leader
Dec 22 07:34:54 minikube localkube[2818]: scheduler is ready!
Dec 22 07:34:54 minikube localkube[2818]: Starting kubelet...
Dec 22 07:34:54 minikube localkube[2818]: Waiting for kubelet to be healthy...
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.455276    2818 feature_gate.go:156] feature gates: map[]
Dec 22 07:34:54 minikube localkube[2818]: W1222 07:34:54.455525    2818 server.go:276] --require-kubeconfig is deprecated. Set --kubeconfig without using --require-kubeconfig.
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.789674    2818 client.go:75] Connecting to docker on unix:///var/run/docker.sock
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.789876    2818 client.go:95] Start docker client with request timeout=2m0s
Dec 22 07:34:54 minikube localkube[2818]: W1222 07:34:54.794913    2818 server.go:289] --cloud-provider=auto-detect is deprecated. The desired cloud provider should be set explicitly
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.839433    2818 manager.go:149] cAdvisor running in container: "/sys/fs/cgroup/cpu,cpuacct/system.slice/localkube.service"
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.875182    2818 fs.go:139] Filesystem UUIDs: map[9e466cd6-789d-4a81-86d1-ae2d6f825419:/dev/vda2 bc6de4d3-3487-4715-a595-711d67e9f04e:/dev/vda1]
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.875205    2818 fs.go:140] Filesystem partitions: map[/dev/vda1:{mountpoint:/mnt/vda1 major:253 minor:1 fsType:ext4 blockSize:0} tmpfs:{mountpoint:/dev/shm major:0 minor:17 fsType:tmpfs blockSize:0}]
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.877089    2818 manager.go:216] Machine: {NumCores:2 CpuFrequency:2993200 MemoryCapacity:1998602240 HugePages:[{PageSize:2048 NumPages:0}] MachineID:095a0ca0973049b4a86f1005c6c85a0e SystemUUID:13B41841-A3F1-430A-8296-E1A169C48D93 BootID:ef5318f6-adc8-45ae-86dd-7fd2e845ce4f Filesystems:[{Device:tmpfs DeviceMajor:0 DeviceMinor:17 Capacity:999301120 Type:vfs Inodes:243970 HasInodes:true} {Device:/dev/vda1 DeviceMajor:253 DeviceMinor:1 Capacity:16444203008 Type:vfs Inodes:9255232 HasInodes:true} {Device:rootfs DeviceMajor:0 DeviceMinor:1 Capacity:0 Type:vfs Inodes:0 HasInodes:true}] DiskMap:map[253:0:{Name:vda Major:253 Minor:0 Size:20000000000 Scheduler:none}] NetworkDevices:[{Name:eth0 MacAddress:44:17:e3:09:dd:a0 Speed:-1 Mtu:1500} {Name:eth1 MacAddress:44:17:e3:09:dd:a0 Speed:-1 Mtu:1500} {Name:sit0 MacAddress:00:00:00:00 Speed:0 Mtu:1480}] Topology:[{Id:0 Memory:1998602240 Cores:[{Id:0 Threads:[0] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:1 Memory:0 Cores:[{Id:0 Threads:[1] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.880027    2818 manager.go:222] Version: {KernelVersion:4.9.13 ContainerOsVersion:Buildroot 2017.02 DockerVersion:17.06.0-ce DockerAPIVersion:1.30 CadvisorVersion: CadvisorRevision:}
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.880476    2818 server.go:422] --cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.881726    2818 container_manager_linux.go:252] container manager verified user specified cgroup-root exists: /
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.881936    2818 container_manager_linux.go:257] Creating Container Manager object based on Node Config: {RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:docker CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:memory.available Operator:LessThan Value:{Quantity:100Mi Percentage:0} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.1} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.inodesFree Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>}]} ExperimentalQOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalCPUManagerReconcilePeriod:10s}
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.882791    2818 container_manager_linux.go:288] Creating device plugin handler: false
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.882942    2818 kubelet.go:273] Adding manifest file: /etc/kubernetes/manifests
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.882965    2818 kubelet.go:283] Watching apiserver
Dec 22 07:34:54 minikube localkube[2818]: W1222 07:34:54.892912    2818 kubelet_network.go:69] Hairpin mode set to "promiscuous-bridge" but kubenet is not enabled, falling back to "hairpin-veth"
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.893192    2818 kubelet.go:517] Hairpin mode set to "hairpin-veth"
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.916146    2818 docker_service.go:207] Docker cri networking managed by kubernetes.io/no-op
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.941330    2818 docker_service.go:224] Setting cgroupDriver to cgroupfs
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.965764    2818 remote_runtime.go:43] Connecting to runtime service unix:///var/run/dockershim.sock
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.967340    2818 kuberuntime_manager.go:174] Container runtime docker initialized, version: 17.06.0-ce, apiVersion: 1.30.0
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.967405    2818 kuberuntime_manager.go:898] updating runtime config through cri with podcidr 10.180.1.0/24
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.967538    2818 docker_service.go:306] docker cri received runtime config &RuntimeConfig{NetworkConfig:&NetworkConfig{PodCidr:10.180.1.0/24,},}
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.967751    2818 kubelet_network.go:276] Setting Pod CIDR:  -> 10.180.1.0/24
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.969305    2818 server.go:718] Started kubelet v1.8.0
Dec 22 07:34:54 minikube localkube[2818]: E1222 07:34:54.969503    2818 kubelet.go:1234] Image garbage collection failed once. Stats initialization may not have completed yet: unable to find data for container /
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.970747    2818 server.go:128] Starting to listen on 0.0.0.0:10250
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.971465    2818 server.go:296] Adding debug handlers to kubelet server.
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.992207    2818 fs_resource_analyzer.go:66] Starting FS ResourceAnalyzer
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.992242    2818 status_manager.go:140] Starting to sync pod status with apiserver
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.992258    2818 kubelet.go:1768] Starting kubelet main sync loop.
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.992276    2818 kubelet.go:1779] skipping pod synchronization - [container runtime is down PLEG is not healthy: pleg was last seen active 2562047h47m16.854775807s ago; threshold is 3m0s]
Dec 22 07:34:54 minikube localkube[2818]: E1222 07:34:54.992644    2818 container_manager_linux.go:603] [ContainerManager]: Fail to get rootfs information unable to find data for container /
Dec 22 07:34:54 minikube localkube[2818]: I1222 07:34:54.992673    2818 volume_manager.go:246] Starting Kubelet Volume Manager
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.086740    2818 factory.go:355] Registering Docker factory
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.094264    2818 kubelet_node_status.go:276] Setting node annotation to enable volume controller attach/detach
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.094890    2818 factory.go:89] Registering Rkt factory
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.113163    2818 factory.go:157] Registering CRI-O factory
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.113205    2818 factory.go:54] Registering systemd factory
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.113414    2818 factory.go:86] Registering Raw factory
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.113637    2818 manager.go:1140] Started watching for new ooms in manager
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.114303    2818 manager.go:311] Starting recovery of all containers
Dec 22 07:34:55 minikube localkube[2818]: W1222 07:34:55.147668    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kubernetes-dashboard-5sgcl through plugin: invalid network status for
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.150526    2818 manager.go:316] Recovery completed
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.162620    2818 kubelet_node_status.go:83] Attempting to register node minikube
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.165497    2818 rkt.go:56] starting detectRktContainers thread
Dec 22 07:34:55 minikube localkube[2818]: E1222 07:34:55.214223    2818 helpers.go:832] Could not find capacity information for resource ephemeral-storage
Dec 22 07:34:55 minikube localkube[2818]: W1222 07:34:55.214269    2818 helpers.go:843] eviction manager: no observation found for eviction signal allocatableNodeFs.available
Dec 22 07:34:55 minikube localkube[2818]: W1222 07:34:55.214730    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kubernetes-dashboard-5sgcl through plugin: invalid network status for
Dec 22 07:34:55 minikube localkube[2818]: apply entries took too long [112.466773ms for 1 entries]
Dec 22 07:34:55 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.301466    2818 kubelet_node_status.go:134] Node minikube was previously registered
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.301495    2818 kubelet_node_status.go:86] Successfully registered node minikube
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.302415    2818 kuberuntime_manager.go:898] updating runtime config through cri with podcidr
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.302522    2818 docker_service.go:306] docker cri received runtime config &RuntimeConfig{NetworkConfig:&NetworkConfig{PodCidr:,},}
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.302614    2818 kubelet_network.go:276] Setting Pod CIDR: 10.180.1.0/24 ->
Dec 22 07:34:55 minikube localkube[2818]: kubelet is ready!
Dec 22 07:34:55 minikube localkube[2818]: Starting proxy...
Dec 22 07:34:55 minikube localkube[2818]: Waiting for proxy to be healthy...
Dec 22 07:34:55 minikube localkube[2818]: W1222 07:34:55.455510    2818 server_others.go:63] unable to register configz: register config "componentconfig" twice
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.477386    2818 server_others.go:117] Using iptables Proxier.
Dec 22 07:34:55 minikube localkube[2818]: W1222 07:34:55.584251    2818 proxier.go:473] clusterCIDR not specified, unable to distinguish between internal and external traffic
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.584588    2818 server_others.go:152] Tearing down inactive rules.
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.599765    2818 config.go:202] Starting service config controller
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.599788    2818 controller_utils.go:1041] Waiting for caches to sync for service config controller
Dec 22 07:34:55 minikube localkube[2818]: E1222 07:34:55.599825    2818 healthcheck.go:317] Failed to start node healthz on 0: listen tcp: address 0: missing port in address
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.599842    2818 config.go:102] Starting endpoints config controller
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.599847    2818 controller_utils.go:1041] Waiting for caches to sync for endpoints config controller
Dec 22 07:34:55 minikube localkube[2818]: W1222 07:34:55.643581    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for default/busybox through plugin: invalid network status for
Dec 22 07:34:55 minikube localkube[2818]: W1222 07:34:55.695922    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for default/busybox through plugin: invalid network status for
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.699872    2818 controller_utils.go:1048] Caches are synced for service config controller
Dec 22 07:34:55 minikube localkube[2818]: I1222 07:34:55.700136    2818 controller_utils.go:1048] Caches are synced for endpoints config controller
Dec 22 07:34:55 minikube localkube[2818]: E1222 07:34:55.721493    2818 proxier.go:1621] Failed to delete stale service IP 10.96.0.10 connections, error: error deleting connection tracking state for UDP service IP: 10.96.0.10, error: error looking for path of conntrack: exec: "conntrack": executable file not found in $PATH
Dec 22 07:34:55 minikube localkube[2818]: W1222 07:34:55.798830    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kubernetes-dashboard-ks82m through plugin: invalid network status for
Dec 22 07:34:55 minikube localkube[2818]: W1222 07:34:55.845952    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kubernetes-dashboard-ks82m through plugin: invalid network status for
Dec 22 07:34:56 minikube localkube[2818]: W1222 07:34:56.211417    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kube-dns-86f6f55dd5-mrtrm through plugin: invalid network status for
Dec 22 07:34:56 minikube localkube[2818]: E1222 07:34:56.429044    2818 remote_runtime.go:278] ContainerStatus "6cf6ec071403c078c940aa3640be3e67bea2719f7ee5817e08aff83fc886fa18" from runtime service failed: rpc error: code = Unknown desc = Error: No such container: 6cf6ec071403c078c940aa3640be3e67bea2719f7ee5817e08aff83fc886fa18
Dec 22 07:34:56 minikube localkube[2818]: E1222 07:34:56.429098    2818 kuberuntime_container.go:416] ContainerStatus for 6cf6ec071403c078c940aa3640be3e67bea2719f7ee5817e08aff83fc886fa18 error: rpc error: code = Unknown desc = Error: No such container: 6cf6ec071403c078c940aa3640be3e67bea2719f7ee5817e08aff83fc886fa18
Dec 22 07:34:56 minikube localkube[2818]: E1222 07:34:56.429109    2818 kuberuntime_manager.go:854] getPodContainerStatuses for pod "kube-dns-86f6f55dd5-mrtrm_kube-system(2d13fb34-e653-11e7-80fb-4417e309dda0)" failed: rpc error: code = Unknown desc = Error: No such container: 6cf6ec071403c078c940aa3640be3e67bea2719f7ee5817e08aff83fc886fa18
Dec 22 07:34:56 minikube localkube[2818]: E1222 07:34:56.429125    2818 generic.go:241] PLEG: Ignoring events for pod kube-dns-86f6f55dd5-mrtrm/kube-system: rpc error: code = Unknown desc = Error: No such container: 6cf6ec071403c078c940aa3640be3e67bea2719f7ee5817e08aff83fc886fa18
Dec 22 07:34:56 minikube localkube[2818]: W1222 07:34:56.446224    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for default/mongo-6fc9bd6d6c-96htc through plugin: invalid network status for
Dec 22 07:34:56 minikube localkube[2818]: W1222 07:34:56.447458    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for default/mongo-6fc9bd6d6c-96htc through plugin: invalid network status for
Dec 22 07:34:56 minikube localkube[2818]: proxy is ready!
Dec 22 07:34:56 minikube localkube[2818]: W1222 07:34:56.783483    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for default/memcached-64dcdbc9f6-j2v97 through plugin: invalid network status for
Dec 22 07:34:56 minikube localkube[2818]: W1222 07:34:56.785491    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for default/memcached-64dcdbc9f6-j2v97 through plugin: invalid network status for
Dec 22 07:34:57 minikube localkube[2818]: W1222 07:34:57.505535    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kubernetes-dashboard-ks82m through plugin: invalid network status for
Dec 22 07:34:57 minikube localkube[2818]: W1222 07:34:57.686217    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kube-dns-86f6f55dd5-mrtrm through plugin: invalid network status for
Dec 22 07:34:57 minikube localkube[2818]: W1222 07:34:57.769627    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for default/mongo-6fc9bd6d6c-96htc through plugin: invalid network status for
Dec 22 07:34:57 minikube localkube[2818]: W1222 07:34:57.960420    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for default/memcached-64dcdbc9f6-j2v97 through plugin: invalid network status for
Dec 22 07:34:58 minikube localkube[2818]: W1222 07:34:58.084773    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for default/mongo-6fc9bd6d6c-96htc through plugin: invalid network status for
Dec 22 07:34:58 minikube localkube[2818]: W1222 07:34:58.086409    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kubernetes-dashboard-5sgcl through plugin: invalid network status for
Dec 22 07:34:58 minikube localkube[2818]: W1222 07:34:58.232948    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for default/memcached-64dcdbc9f6-j2v97 through plugin: invalid network status for
Dec 22 07:34:58 minikube localkube[2818]: W1222 07:34:58.233470    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for default/busybox through plugin: invalid network status for
Dec 22 07:34:58 minikube localkube[2818]: W1222 07:34:58.297979    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kubernetes-dashboard-5sgcl through plugin: invalid network status for
Dec 22 07:34:58 minikube localkube[2818]: W1222 07:34:58.303846    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for default/busybox through plugin: invalid network status for
Dec 22 07:34:58 minikube localkube[2818]: W1222 07:34:58.308189    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kubernetes-dashboard-ks82m through plugin: invalid network status for
Dec 22 07:34:59 minikube localkube[2818]: W1222 07:34:59.330602    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for default/memcached-64dcdbc9f6-j2v97 through plugin: invalid network status for
Dec 22 07:34:59 minikube localkube[2818]: W1222 07:34:59.336201    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kubernetes-dashboard-5sgcl through plugin: invalid network status for
Dec 22 07:34:59 minikube localkube[2818]: W1222 07:34:59.340985    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for default/busybox through plugin: invalid network status for
Dec 22 07:34:59 minikube localkube[2818]: W1222 07:34:59.992656    2818 pod_container_deletor.go:77] Container "ac78cd42cca75f3f933fa33e6eaa4811e7974d59732604207070283687244b64" not found in pod's containers
Dec 22 07:34:59 minikube localkube[2818]: W1222 07:34:59.992711    2818 pod_container_deletor.go:77] Container "080dce37b0fa2a41f7e7a6e885f2b862ede4715efd91f225915dcec05b5c4469" not found in pod's containers
Dec 22 07:34:59 minikube localkube[2818]: W1222 07:34:59.992723    2818 pod_container_deletor.go:77] Container "b05cfdbc2c19744d8f02ffe2125e352a493b9fbf9bfc5c4247470bd00caaeb35" not found in pod's containers
Dec 22 07:34:59 minikube localkube[2818]: W1222 07:34:59.992739    2818 pod_container_deletor.go:77] Container "a2b208590bd60c342bb0fc6097cefb0af102b18eaad6382bb704a4e65e5abc4a" not found in pod's containers
Dec 22 07:34:59 minikube localkube[2818]: W1222 07:34:59.992777    2818 pod_container_deletor.go:77] Container "0ddf4f3d5e1d4f65c5122a3780531f1a6d953320b14d7067011f3e4eb9f1f078" not found in pod's containers
Dec 22 07:35:00 minikube localkube[2818]: E1222 07:35:00.000542    2818 helpers.go:468] PercpuUsage had 0 cpus, but the actual number is 2; ignoring extra CPUs
Dec 22 07:35:00 minikube localkube[2818]: E1222 07:35:00.019099    2818 helpers.go:468] PercpuUsage had 0 cpus, but the actual number is 2; ignoring extra CPUs
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.019350    2818 pod_container_deletor.go:77] Container "f371c5c3f319555a9a5fd4292c4b823e11ca7a2d8422ff73c5ee0bf741117a15" not found in pod's containers
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.019382    2818 pod_container_deletor.go:77] Container "d3c06d587451b31f03d787d56097a33759f3fee35f8ea55b2286e3aad270db60" not found in pod's containers
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.019394    2818 pod_container_deletor.go:77] Container "c49f9fbf3abb4d3786cd897b808ea86eb5dad83618cc7904dd70b86164b5946e" not found in pod's containers
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.019405    2818 pod_container_deletor.go:77] Container "000a7230198a8e490823ee910421c92f0cfd65158718b95792401b39f3d73b5e" not found in pod's containers
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.019422    2818 pod_container_deletor.go:77] Container "0042a2a59236ca6a3c8863faeae35f7a2bce21306242ee08bab26ffe65df125f" not found in pod's containers
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.019433    2818 pod_container_deletor.go:77] Container "87a2c4c8d013473e202575007ce85e464b5ac08d50786b17d245d3aa448d0760" not found in pod's containers
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.019444    2818 pod_container_deletor.go:77] Container "0526563a4691ad5e6216ef325aef4748ebc79fd764de847b432f69778834fa71" not found in pod's containers
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.019459    2818 pod_container_deletor.go:77] Container "b1cba84ddfcda709695fd0aff24a735895c1e5e3affb166f121e4511d977df3e" not found in pod's containers
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.019470    2818 pod_container_deletor.go:77] Container "20ea7b2f3e90197bb0e8748ece49013c0b9970fc42f2e0247cce3a497ea59627" not found in pod's containers
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.019481    2818 pod_container_deletor.go:77] Container "999733a60044e5413cd189438706686fe74966ad4a26c02afbbed17acb1ae044" not found in pod's containers
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.019496    2818 pod_container_deletor.go:77] Container "dd8b0adafd090b6451868270018ed3293d46477d82a9261d2d22c9e483b0e275" not found in pod's containers
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.019507    2818 pod_container_deletor.go:77] Container "73ab0d3d819da53d90a1d0c4bb698e5fd3fc939fdd9b455ca52cb0e36c4cf470" not found in pod's containers
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.019517    2818 pod_container_deletor.go:77] Container "a3cbe8c2d7e89948e4884b074366823ef51fa509206855697a0699d897de3437" not found in pod's containers
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.019533    2818 pod_container_deletor.go:77] Container "26a4da423b266514f85620c88cf27e6c9f4b3efee92416112006a61e7bc93864" not found in pod's containers
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.019544    2818 pod_container_deletor.go:77] Container "83aeb6e2d3ec8f5167646d03bb442056f052578e29a0395a9a2533828b380607" not found in pod's containers
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.019554    2818 pod_container_deletor.go:77] Container "347b3dc87608af367b50580ed57255382d2bfb6e2fe089173ef186db30c7c534" not found in pod's containers
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.019589    2818 pod_container_deletor.go:77] Container "7d19fc2a879f4f7517ee6980a221ab2da983fb6c03e9005302d40987a3084774" not found in pod's containers
Dec 22 07:35:00 minikube localkube[2818]: I1222 07:35:00.093040    2818 reconciler.go:212] operationExecutor.VerifyControllerAttachedVolume started for volume "default-token-qtknm" (UniqueName: "kubernetes.io/secret/2c05f938-e653-11e7-80fb-4417e309dda0-default-token-qtknm") pod "storage-provisioner" (UID: "2c05f938-e653-11e7-80fb-4417e309dda0")
Dec 22 07:35:00 minikube localkube[2818]: I1222 07:35:00.093336    2818 reconciler.go:212] operationExecutor.VerifyControllerAttachedVolume started for volume "default-token-qtknm" (UniqueName: "kubernetes.io/secret/2c715de1-e653-11e7-80fb-4417e309dda0-default-token-qtknm") pod "kubernetes-dashboard-5sgcl" (UID: "2c715de1-e653-11e7-80fb-4417e309dda0")
Dec 22 07:35:00 minikube localkube[2818]: I1222 07:35:00.093515    2818 reconciler.go:212] operationExecutor.VerifyControllerAttachedVolume started for volume "kube-dns-config" (UniqueName: "kubernetes.io/configmap/2d13fb34-e653-11e7-80fb-4417e309dda0-kube-dns-config") pod "kube-dns-86f6f55dd5-mrtrm" (UID: "2d13fb34-e653-11e7-80fb-4417e309dda0")
Dec 22 07:35:00 minikube localkube[2818]: I1222 07:35:00.093694    2818 reconciler.go:212] operationExecutor.VerifyControllerAttachedVolume started for volume "default-token-qtknm" (UniqueName: "kubernetes.io/secret/2d13fb34-e653-11e7-80fb-4417e309dda0-default-token-qtknm") pod "kube-dns-86f6f55dd5-mrtrm" (UID: "2d13fb34-e653-11e7-80fb-4417e309dda0")
Dec 22 07:35:00 minikube localkube[2818]: I1222 07:35:00.093861    2818 reconciler.go:212] operationExecutor.VerifyControllerAttachedVolume started for volume "default-token-lhk9c" (UniqueName: "kubernetes.io/secret/23077aeb-e658-11e7-97ae-4417e309dda0-default-token-lhk9c") pod "memcached-64dcdbc9f6-j2v97" (UID: "23077aeb-e658-11e7-97ae-4417e309dda0")
Dec 22 07:35:00 minikube localkube[2818]: I1222 07:35:00.094019    2818 reconciler.go:212] operationExecutor.VerifyControllerAttachedVolume started for volume "default-token-lhk9c" (UniqueName: "kubernetes.io/secret/771c91a4-e65f-11e7-97ae-4417e309dda0-default-token-lhk9c") pod "busybox" (UID: "771c91a4-e65f-11e7-97ae-4417e309dda0")
Dec 22 07:35:00 minikube localkube[2818]: I1222 07:35:00.094203    2818 reconciler.go:212] operationExecutor.VerifyControllerAttachedVolume started for volume "addons" (UniqueName: "kubernetes.io/host-path/7b19c3ba446df5355649563d32723e4f-addons") pod "kube-addon-manager-minikube" (UID: "7b19c3ba446df5355649563d32723e4f")
Dec 22 07:35:00 minikube localkube[2818]: I1222 07:35:00.094364    2818 reconciler.go:212] operationExecutor.VerifyControllerAttachedVolume started for volume "kubeconfig" (UniqueName: "kubernetes.io/host-path/7b19c3ba446df5355649563d32723e4f-kubeconfig") pod "kube-addon-manager-minikube" (UID: "7b19c3ba446df5355649563d32723e4f")
Dec 22 07:35:00 minikube localkube[2818]: I1222 07:35:00.094559    2818 reconciler.go:212] operationExecutor.VerifyControllerAttachedVolume started for volume "default-token-qtknm" (UniqueName: "kubernetes.io/secret/ece4febf-e677-11e7-9d13-4417e309dda0-default-token-qtknm") pod "kubernetes-dashboard-ks82m" (UID: "ece4febf-e677-11e7-9d13-4417e309dda0")
Dec 22 07:35:00 minikube localkube[2818]: I1222 07:35:00.094742    2818 reconciler.go:212] operationExecutor.VerifyControllerAttachedVolume started for volume "mongo-volume0" (UniqueName: "kubernetes.io/host-path/a6831561-e66c-11e7-97ae-4417e309dda0-mongo-volume0") pod "mongo-6fc9bd6d6c-96htc" (UID: "a6831561-e66c-11e7-97ae-4417e309dda0")
Dec 22 07:35:00 minikube localkube[2818]: I1222 07:35:00.094902    2818 reconciler.go:212] operationExecutor.VerifyControllerAttachedVolume started for volume "default-token-lhk9c" (UniqueName: "kubernetes.io/secret/a6831561-e66c-11e7-97ae-4417e309dda0-default-token-lhk9c") pod "mongo-6fc9bd6d6c-96htc" (UID: "a6831561-e66c-11e7-97ae-4417e309dda0")
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.321769    2818 raw.go:87] Error while processing event ("/sys/fs/cgroup/memory/system.slice/run-r237b977c778349e3b0ff7cffd2965381.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/memory/system.slice/run-r237b977c778349e3b0ff7cffd2965381.scope: no such file or directory
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.321913    2818 container.go:354] Failed to create summary reader for "/system.slice/run-r237b977c778349e3b0ff7cffd2965381.scope": none of the resources are being tracked.
Dec 22 07:35:00 minikube localkube[2818]: E1222 07:35:00.322028    2818 helpers.go:468] PercpuUsage had 0 cpus, but the actual number is 2; ignoring extra CPUs
Dec 22 07:35:00 minikube localkube[2818]: E1222 07:35:00.446746    2818 helpers.go:468] PercpuUsage had 0 cpus, but the actual number is 2; ignoring extra CPUs
Dec 22 07:35:00 minikube localkube[2818]: E1222 07:35:00.446904    2818 helpers.go:468] PercpuUsage had 0 cpus, but the actual number is 2; ignoring extra CPUs
Dec 22 07:35:00 minikube localkube[2818]: E1222 07:35:00.491405    2818 proxier.go:942] Failed to delete kube-system/kube-dns:dns endpoint connections, error: error deleting conntrack entries for UDP peer {10.96.0.10, 172.17.0.3}, error: error looking for path of conntrack: exec: "conntrack": executable file not found in $PATH
Dec 22 07:35:00 minikube localkube[2818]: W1222 07:35:00.926240    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kubernetes-dashboard-ks82m through plugin: invalid network status for
Dec 22 07:35:01 minikube localkube[2818]: W1222 07:35:01.219970    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for default/mongo-6fc9bd6d6c-96htc through plugin: invalid network status for
Dec 22 07:35:01 minikube localkube[2818]: W1222 07:35:01.511063    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for kube-system/kubernetes-dashboard-5sgcl through plugin: invalid network status for
Dec 22 07:35:01 minikube localkube[2818]: W1222 07:35:01.801079    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for default/memcached-64dcdbc9f6-j2v97 through plugin: invalid network status for
Dec 22 07:35:02 minikube localkube[2818]: I1222 07:35:02.246081    2818 kuberuntime_manager.go:738] checking backoff for container "kube-addon-manager" in pod "kube-addon-manager-minikube_kube-system(7b19c3ba446df5355649563d32723e4f)"
Dec 22 07:35:02 minikube localkube[2818]: W1222 07:35:02.405910    2818 docker_sandbox.go:343] failed to read pod IP from plugin/docker: Couldn't find network status for default/busybox through plugin: invalid network status for
Dec 22 07:35:02 minikube localkube[2818]: apply entries took too long [103.852841ms for 1 entries]
Dec 22 07:35:02 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:35:03 minikube localkube[2818]: I1222 07:35:03.293216    2818 kuberuntime_manager.go:738] checking backoff for container "kubernetes-dashboard" in pod "kubernetes-dashboard-ks82m_kube-system(ece4febf-e677-11e7-9d13-4417e309dda0)"
Dec 22 07:35:03 minikube localkube[2818]: apply entries took too long [113.398982ms for 1 entries]
Dec 22 07:35:03 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:35:04 minikube localkube[2818]: I1222 07:35:04.143111    2818 kuberuntime_manager.go:738] checking backoff for container "storage-provisioner" in pod "storage-provisioner_kube-system(2c05f938-e653-11e7-80fb-4417e309dda0)"
Dec 22 07:35:04 minikube localkube[2818]: apply entries took too long [257.418645ms for 1 entries]
Dec 22 07:35:04 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:35:04 minikube localkube[2818]: apply entries took too long [221.96714ms for 1 entries]
Dec 22 07:35:04 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:35:06 minikube localkube[2818]: apply entries took too long [302.56661ms for 1 entries]
Dec 22 07:35:06 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:35:06 minikube localkube[2818]: I1222 07:35:06.222943    2818 trace.go:76] Trace[205556639]: "Create /api/v1/namespaces/kube-system/events" (started: 2017-12-22 07:35:05.662063445 +0000 UTC m=+20.074122810) (total time: 560.846931ms):
Dec 22 07:35:06 minikube localkube[2818]: Trace[205556639]: [560.788503ms] [560.670682ms] Object stored in database
Dec 22 07:35:06 minikube localkube[2818]: apply entries took too long [180.166321ms for 1 entries]
Dec 22 07:35:06 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:35:06 minikube localkube[2818]: I1222 07:35:06.993797    2818 kuberuntime_manager.go:738] checking backoff for container "mongo" in pod "mongo-6fc9bd6d6c-96htc_default(a6831561-e66c-11e7-97ae-4417e309dda0)"
Dec 22 07:35:07 minikube localkube[2818]: apply entries took too long [498.955488ms for 2 entries]
Dec 22 07:35:07 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:35:07 minikube localkube[2818]: I1222 07:35:07.534842    2818 trace.go:76] Trace[883052665]: "GuaranteedUpdate etcd3: *api.Endpoints" (started: 2017-12-22 07:35:06.910346194 +0000 UTC m=+21.322405576) (total time: 624.46395ms):
Dec 22 07:35:07 minikube localkube[2818]: Trace[883052665]: [624.432176ms] [624.312051ms] Transaction committed
Dec 22 07:35:07 minikube localkube[2818]: I1222 07:35:07.534946    2818 trace.go:76] Trace[1193855968]: "Update /api/v1/namespaces/kube-system/endpoints/kube-controller-manager" (started: 2017-12-22 07:35:06.910288611 +0000 UTC m=+21.322347968) (total time: 624.641213ms):
Dec 22 07:35:07 minikube localkube[2818]: Trace[1193855968]: [624.579756ms] [624.546282ms] Object stored in database
Dec 22 07:35:07 minikube localkube[2818]: I1222 07:35:07.536459    2818 trace.go:76] Trace[1420098417]: "Create /api/v1/namespaces/default/events" (started: 2017-12-22 07:35:06.996548569 +0000 UTC m=+21.408607926) (total time: 539.887666ms):
Dec 22 07:35:07 minikube localkube[2818]: Trace[1420098417]: [539.844099ms] [539.788795ms] Object stored in database
Dec 22 07:35:08 minikube localkube[2818]: I1222 07:35:08.644304    2818 kuberuntime_manager.go:738] checking backoff for container "kubernetes-dashboard" in pod "kubernetes-dashboard-5sgcl_kube-system(2c715de1-e653-11e7-80fb-4417e309dda0)"
Dec 22 07:35:08 minikube localkube[2818]: E1222 07:35:08.651130    2818 cadvisor_stats_provider.go:87] Partial failure issuing cadvisor.ContainerInfoV2: partial failures: ["/kubepods/besteffort/pod2c715de1-e653-11e7-80fb-4417e309dda0/0534ee566d9ad4780ee3667c4b02ca42de5d367bfa7f433ccc3ba79312d9d334": RecentStats: unable to find data for container /kubepods/besteffort/pod2c715de1-e653-11e7-80fb-4417e309dda0/0534ee566d9ad4780ee3667c4b02ca42de5d367bfa7f433ccc3ba79312d9d334]
Dec 22 07:35:08 minikube localkube[2818]: apply entries took too long [100.653131ms for 1 entries]
Dec 22 07:35:08 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:35:10 minikube localkube[2818]: apply entries took too long [171.059416ms for 1 entries]
Dec 22 07:35:10 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:35:10 minikube localkube[2818]: apply entries took too long [107.830961ms for 1 entries]
Dec 22 07:35:10 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:35:11 minikube localkube[2818]: I1222 07:35:11.607953    2818 kuberuntime_manager.go:738] checking backoff for container "memcached" in pod "memcached-64dcdbc9f6-j2v97_default(23077aeb-e658-11e7-97ae-4417e309dda0)"
Dec 22 07:35:11 minikube localkube[2818]: W1222 07:35:11.614900    2818 kuberuntime_container.go:191] Non-root verification doesn't support non-numeric user (memcache)
Dec 22 07:35:11 minikube localkube[2818]: apply entries took too long [307.080321ms for 1 entries]
Dec 22 07:35:11 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:35:11 minikube localkube[2818]: I1222 07:35:11.922086    2818 kuberuntime_manager.go:738] checking backoff for container "busybox" in pod "busybox_default(771c91a4-e65f-11e7-97ae-4417e309dda0)"
Dec 22 07:35:12 minikube localkube[2818]: I1222 07:35:12.244120    2818 kuberuntime_manager.go:738] checking backoff for container "kubedns" in pod "kube-dns-86f6f55dd5-mrtrm_kube-system(2d13fb34-e653-11e7-80fb-4417e309dda0)"
Dec 22 07:35:12 minikube localkube[2818]: apply entries took too long [369.038639ms for 2 entries]
Dec 22 07:35:12 minikube localkube[2818]: I1222 07:35:12.776108    2818 trace.go:76] Trace[361783492]: "GuaranteedUpdate etcd3: *api.Pod" (started: 2017-12-22 07:35:12.266405122 +0000 UTC m=+26.678464491) (total time: 509.679774ms):
Dec 22 07:35:12 minikube localkube[2818]: Trace[361783492]: [509.60462ms] [509.40002ms] Transaction committed
Dec 22 07:35:12 minikube localkube[2818]: I1222 07:35:12.776199    2818 trace.go:76] Trace[631905750]: "Update /api/v1/namespaces/kube-system/pods/kube-addon-manager-minikube/status" (started: 2017-12-22 07:35:12.266297219 +0000 UTC m=+26.678356577) (total time: 509.88736ms):
Dec 22 07:35:12 minikube localkube[2818]: Trace[631905750]: [509.826025ms] [509.755483ms] Object stored in database
Dec 22 07:35:12 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:35:13 minikube localkube[2818]: apply entries took too long [166.510078ms for 1 entries]
Dec 22 07:35:13 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:35:13 minikube localkube[2818]: I1222 07:35:13.069670    2818 trace.go:76] Trace[1180161878]: "Create /api/v1/namespaces/default/events" (started: 2017-12-22 07:35:12.227557677 +0000 UTC m=+26.639617033) (total time: 842.071599ms):
Dec 22 07:35:13 minikube localkube[2818]: Trace[1180161878]: [841.973813ms] [841.776035ms] Object stored in database
Dec 22 07:35:14 minikube localkube[2818]: I1222 07:35:14.767800    2818 trace.go:76] Trace[357138399]: "GuaranteedUpdate etcd3: *api.Endpoints" (started: 2017-12-22 07:35:14.246125182 +0000 UTC m=+28.658184598) (total time: 521.643668ms):
Dec 22 07:35:14 minikube localkube[2818]: Trace[357138399]: [521.61422ms] [521.275807ms] Transaction committed
Dec 22 07:35:14 minikube localkube[2818]: I1222 07:35:14.767899    2818 trace.go:76] Trace[1159833397]: "Update /api/v1/namespaces/kube-system/endpoints/kubernetes-dashboard" (started: 2017-12-22 07:35:14.244112897 +0000 UTC m=+28.656172252) (total time: 523.771874ms):
Dec 22 07:35:14 minikube localkube[2818]: Trace[1159833397]: [523.716004ms] [523.589579ms] Object stored in database
Dec 22 07:35:14 minikube localkube[2818]: I1222 07:35:14.768514    2818 trace.go:76] Trace[87011467]: "List /api/v1/nodes" (started: 2017-12-22 07:35:13.854868101 +0000 UTC m=+28.266927449) (total time: 913.617884ms):
Dec 22 07:35:14 minikube localkube[2818]: Trace[87011467]: [912.915606ms] [912.905161ms] Listing from storage done
Dec 22 07:35:14 minikube localkube[2818]: I1222 07:35:14.768617    2818 trace.go:76] Trace[263862587]: "GuaranteedUpdate etcd3: *api.Pod" (started: 2017-12-22 07:35:14.241421294 +0000 UTC m=+28.653480656) (total time: 527.18151ms):
Dec 22 07:35:14 minikube localkube[2818]: Trace[263862587]: [527.128958ms] [526.893128ms] Transaction committed
Dec 22 07:35:14 minikube localkube[2818]: I1222 07:35:14.769579    2818 trace.go:76] Trace[1180589326]: "Update /api/v1/namespaces/default/pods/mongo-6fc9bd6d6c-96htc/status" (started: 2017-12-22 07:35:14.241290279 +0000 UTC m=+28.653349633) (total time: 528.259125ms):
Dec 22 07:35:14 minikube localkube[2818]: Trace[1180589326]: [528.203816ms] [528.10994ms] Object stored in database
Dec 22 07:35:14 minikube localkube[2818]: I1222 07:35:14.770826    2818 trace.go:76] Trace[1293522007]: "GuaranteedUpdate etcd3: *api.ReplicationController" (started: 2017-12-22 07:35:14.243378699 +0000 UTC m=+28.655438071) (total time: 527.431829ms):
Dec 22 07:35:14 minikube localkube[2818]: Trace[1293522007]: [527.40662ms] [527.147443ms] Transaction committed
Dec 22 07:35:14 minikube localkube[2818]: I1222 07:35:14.771194    2818 trace.go:76] Trace[2127665142]: "Update /api/v1/namespaces/kube-system/replicationcontrollers/kubernetes-dashboard/status" (started: 2017-12-22 07:35:14.243177164 +0000 UTC m=+28.655236516) (total time: 527.980963ms):
Dec 22 07:35:14 minikube localkube[2818]: Trace[2127665142]: [527.931607ms] [527.786525ms] Object stored in database
Dec 22 07:35:14 minikube localkube[2818]: I1222 07:35:14.772010    2818 trace.go:76] Trace[259762650]: "Create /api/v1/namespaces/default/events" (started: 2017-12-22 07:35:14.074613443 +0000 UTC m=+28.486672800) (total time: 697.379944ms):
Dec 22 07:35:14 minikube localkube[2818]: Trace[259762650]: [697.341491ms] [697.281352ms] Object stored in database
Dec 22 07:35:14 minikube localkube[2818]: I1222 07:35:14.773048    2818 trace.go:76] Trace[797879057]: "syncReplicationController: kube-system/kubernetes-dashboard" (started: 2017-12-22 07:35:14.240483257 +0000 UTC m=+28.652542610) (total time: 532.533437ms):
Dec 22 07:35:14 minikube localkube[2818]: Trace[797879057]: [532.533437ms] [532.49745ms] END
Dec 22 07:35:15 minikube localkube[2818]: I1222 07:35:15.321684    2818 trace.go:76] Trace[954081160]: "GuaranteedUpdate etcd3: *api.ReplicationController" (started: 2017-12-22 07:35:14.244677367 +0000 UTC m=+28.656736733) (total time: 1.076971851s):
Dec 22 07:35:15 minikube localkube[2818]: Trace[954081160]: [525.348543ms] [504.973605ms] Transaction committed
Dec 22 07:35:15 minikube localkube[2818]: Trace[954081160]: [1.076910371s] [550.889406ms] Transaction committed
Dec 22 07:35:15 minikube localkube[2818]: I1222 07:35:15.323372    2818 trace.go:76] Trace[1435115477]: "GuaranteedUpdate etcd3: *extensions.ReplicaSet" (started: 2017-12-22 07:35:14.775050167 +0000 UTC m=+29.187109532) (total time: 548.299234ms):
Dec 22 07:35:15 minikube localkube[2818]: Trace[1435115477]: [548.240599ms] [547.978264ms] Transaction committed
Dec 22 07:35:15 minikube localkube[2818]: I1222 07:35:15.323477    2818 trace.go:76] Trace[560279054]: "Update /apis/extensions/v1beta1/namespaces/default/replicasets/mongo-6fc9bd6d6c/status" (started: 2017-12-22 07:35:14.774862558 +0000 UTC m=+29.186921915) (total time: 548.599514ms):
Dec 22 07:35:15 minikube localkube[2818]: Trace[560279054]: [548.527136ms] [548.365704ms] Object stored in database
Dec 22 07:35:15 minikube localkube[2818]: I1222 07:35:15.325249    2818 trace.go:76] Trace[829951375]: "GuaranteedUpdate etcd3: *api.Endpoints" (started: 2017-12-22 07:35:14.774539455 +0000 UTC m=+29.186598819) (total time: 550.678113ms):
Dec 22 07:35:15 minikube localkube[2818]: Trace[829951375]: [550.63875ms] [550.456718ms] Transaction committed
Dec 22 07:35:15 minikube localkube[2818]: I1222 07:35:15.325344    2818 trace.go:76] Trace[580309756]: "Update /api/v1/namespaces/default/endpoints/mongo" (started: 2017-12-22 07:35:14.774386162 +0000 UTC m=+29.186445510) (total time: 550.945192ms):
Dec 22 07:35:15 minikube localkube[2818]: Trace[580309756]: [550.880918ms] [550.749956ms] Object stored in database
Dec 22 07:35:15 minikube localkube[2818]: I1222 07:35:15.564092    2818 kuberuntime_manager.go:738] checking backoff for container "dnsmasq" in pod "kube-dns-86f6f55dd5-mrtrm_kube-system(2d13fb34-e653-11e7-80fb-4417e309dda0)"
Dec 22 07:35:15 minikube localkube[2818]: I1222 07:35:15.902446    2818 trace.go:76] Trace[154696060]: "GuaranteedUpdate etcd3: *extensions.Deployment" (started: 2017-12-22 07:35:15.337020944 +0000 UTC m=+29.749080309) (total time: 565.40176ms):
Dec 22 07:35:15 minikube localkube[2818]: Trace[154696060]: [565.369297ms] [565.192794ms] Transaction committed
Dec 22 07:35:15 minikube localkube[2818]: I1222 07:35:15.902552    2818 trace.go:76] Trace[1934279133]: "Update /apis/extensions/v1beta1/namespaces/default/deployments/mongo/status" (started: 2017-12-22 07:35:15.336896992 +0000 UTC m=+29.748956353) (total time: 565.639743ms):
Dec 22 07:35:15 minikube localkube[2818]: Trace[1934279133]: [565.573904ms] [565.480181ms] Object stored in database
Dec 22 07:35:15 minikube localkube[2818]: I1222 07:35:15.902781    2818 trace.go:76] Trace[1637991688]: "Create /api/v1/namespaces/default/events" (started: 2017-12-22 07:35:14.773366036 +0000 UTC m=+29.185425393) (total time: 1.129400133s):
Dec 22 07:35:15 minikube localkube[2818]: Trace[1637991688]: [1.129360381s] [1.129316327s] Object stored in database
Dec 22 07:35:15 minikube localkube[2818]: I1222 07:35:15.902969    2818 trace.go:76] Trace[1716737377]: "GuaranteedUpdate etcd3: *api.Pod" (started: 2017-12-22 07:35:15.328626617 +0000 UTC m=+29.740685986) (total time: 574.329319ms):
Dec 22 07:35:15 minikube localkube[2818]: Trace[1716737377]: [574.282669ms] [567.978434ms] Transaction committed
Dec 22 07:35:15 minikube localkube[2818]: I1222 07:35:15.903240    2818 trace.go:76] Trace[1681140479]: "GuaranteedUpdate etcd3: *api.Endpoints" (started: 2017-12-22 07:35:15.324170212 +0000 UTC m=+29.736229577) (total time: 579.053336ms):
Dec 22 07:35:15 minikube localkube[2818]: Trace[1681140479]: [579.038882ms] [578.917149ms] Transaction committed
Dec 22 07:35:15 minikube localkube[2818]: I1222 07:35:15.903294    2818 trace.go:76] Trace[2069829614]: "Update /api/v1/namespaces/kube-system/endpoints/kube-scheduler" (started: 2017-12-22 07:35:15.324101714 +0000 UTC m=+29.736161071) (total time: 579.178318ms):
Dec 22 07:35:15 minikube localkube[2818]: Trace[2069829614]: [579.149202ms] [579.106087ms] Object stored in database
Dec 22 07:35:15 minikube localkube[2818]: I1222 07:35:15.905108    2818 trace.go:76] Trace[169972720]: "GuaranteedUpdate etcd3: *api.Node" (started: 2017-12-22 07:35:15.372524602 +0000 UTC m=+29.784583968) (total time: 532.567573ms):
Dec 22 07:35:15 minikube localkube[2818]: Trace[169972720]: [532.49916ms] [531.427145ms] Transaction committed
Dec 22 07:35:16 minikube localkube[2818]: I1222 07:35:16.348083    2818 trace.go:76] Trace[1954299806]: "syncReplicationController: kube-system/kubernetes-dashboard" (started: 2017-12-22 07:35:15.326983719 +0000 UTC m=+29.739043076) (total time: 1.021062468s):
Dec 22 07:35:16 minikube localkube[2818]: Trace[1954299806]: [656.69124ms] [656.685746ms] manageReplicas done
Dec 22 07:35:16 minikube localkube[2818]: Trace[1954299806]: [1.021062468s] [364.371228ms] END
Dec 22 07:35:17 minikube localkube[2818]: I1222 07:35:17.492467    2818 trace.go:76] Trace[1039994551]: "Create /api/v1/namespaces/kube-system/events" (started: 2017-12-22 07:35:16.614169102 +0000 UTC m=+31.026228459) (total time: 878.272986ms):
Dec 22 07:35:17 minikube localkube[2818]: Trace[1039994551]: [878.240354ms] [878.027186ms] Object stored in database
Dec 22 07:35:17 minikube localkube[2818]: I1222 07:35:17.616278    2818 kuberuntime_manager.go:738] checking backoff for container "sidecar" in pod "kube-dns-86f6f55dd5-mrtrm_kube-system(2d13fb34-e653-11e7-80fb-4417e309dda0)"
Dec 22 07:35:17 minikube localkube[2818]: W1222 07:35:17.619601    2818 kuberuntime_container.go:191] Non-root verification doesn't support non-numeric user (nobody)
Dec 22 07:35:17 minikube localkube[2818]: apply entries took too long [240.185371ms for 1 entries]
Dec 22 07:35:17 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:35:18 minikube localkube[2818]: apply entries took too long [203.611097ms for 1 entries]
Dec 22 07:35:18 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:35:19 minikube localkube[2818]: apply entries took too long [307.088624ms for 1 entries]
Dec 22 07:35:19 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:35:19 minikube localkube[2818]: I1222 07:35:19.025940    2818 trace.go:76] Trace[1327800518]: "Create /api/v1/namespaces/kube-system/events" (started: 2017-12-22 07:35:18.454891297 +0000 UTC m=+32.866950654) (total time: 571.025973ms):
Dec 22 07:35:19 minikube localkube[2818]: Trace[1327800518]: [570.97488ms] [570.924251ms] Object stored in database
Dec 22 07:35:19 minikube localkube[2818]: apply entries took too long [157.706985ms for 1 entries]
Dec 22 07:35:19 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:35:20 minikube localkube[2818]: apply entries took too long [513.020062ms for 2 entries]
Dec 22 07:35:20 minikube localkube[2818]: avoid queries with large range/delete range!
Dec 22 07:35:20 minikube localkube[2818]: I1222 07:35:20.020257    2818 trace.go:76] Trace[695637770]: "GuaranteedUpdate etcd3: *api.Endpoints" (started: 2017-12-22 07:35:19.495413578 +0000 UTC m=+33.907472944) (total time: 524.819651ms):
Dec 22 07:35:20 minikube localkube[2818]: Trace[695637770]: [524.80701ms] [524.6907ms] Transaction committed
Dec 22 07:35:20 minikube localkube[2818]: I1222 07:35:20.020553    2818 trace.go:76] Trace[1170577788]: "Update /api/v1/namespaces/kube-system/endpoints/kube-controller-manager" (started: 2017-12-22 07:35:19.495363054 +0000 UTC m=+33.907422403) (total time: 525.174235ms):
Dec 22 07:35:20 minikube localkube[2818]: Trace[1170577788]: [525.137153ms] [525.108833ms] Object stored in database
Dec 22 07:35:20 minikube localkube[2818]: I1222 07:35:20.020291    2818 trace.go:76] Trace[745901439]: "Create /api/v1/namespaces/kube-system/events" (started: 2017-12-22 07:35:19.413874997 +0000 UTC m=+33.825934361) (total time: 606.399174ms):
Dec 22 07:35:20 minikube localkube[2818]: Trace[745901439]: [606.355036ms] [606.276516ms] Object stored in database
Dec 22 07:35:21 minikube localkube[2818]: W1222 07:35:21.618313    2818 conversion.go:110] Could not get instant cpu stats: different number of cpus
Dec 22 07:35:21 minikube localkube[2818]: W1222 07:35:21.621387    2818 conversion.go:110] Could not get instant cpu stats: different number of cpus
Dec 22 07:35:25 minikube localkube[2818]: E1222 07:35:25.692850    2818 proxier.go:1621] Failed to delete stale service IP 10.96.0.10 connections, error: error deleting connection tracking state for UDP service IP: 10.96.0.10, error: error looking for path of conntrack: exec: "conntrack": executable file not found in $PATH
Dec 22 07:35:55 minikube localkube[2818]: E1222 07:35:55.599931    2818 healthcheck.go:317] Failed to start node healthz on 0: listen tcp: address 0: missing port in address
Dec 22 07:36:55 minikube localkube[2818]: E1222 07:36:55.600480    2818 healthcheck.go:317] Failed to start node healthz on 0: listen tcp: address 0: missing port in address
Dec 22 07:37:55 minikube localkube[2818]: E1222 07:37:55.600986    2818 healthcheck.go:317] Failed to start node healthz on 0: listen tcp: address 0: missing port in address
Dec 22 07:38:55 minikube localkube[2818]: E1222 07:38:55.601533    2818 healthcheck.go:317] Failed to start node healthz on 0: listen tcp: address 0: missing port in address
Dec 22 07:39:55 minikube localkube[2818]: E1222 07:39:55.601796    2818 healthcheck.go:317] Failed to start node healthz on 0: listen tcp: address 0: missing port in address
Dec 22 07:40:55 minikube localkube[2818]: E1222 07:40:55.602307    2818 healthcheck.go:317] Failed to start node healthz on 0: listen tcp: address 0: missing port in address
Dec 22 07:41:55 minikube localkube[2818]: E1222 07:41:55.602774    2818 healthcheck.go:317] Failed to start node healthz on 0: listen tcp: address 0: missing port in address
deependratyagi commented 6 years ago

@sah4ez Any update on this issue, I am facing the similar problem

deependratyagi commented 6 years ago

@r2d4 Any update on the issue?

fejta-bot commented 6 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

fejta-bot commented 6 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten

fejta-bot commented 6 years ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /close