search

rancher / rke

Rancher Kubernetes Engine (RKE), an extremely simple, lightning fast Kubernetes distribution that runs entirely within containers.
Apache License 2.0
3.22k stars 582 forks source link

Local cluster support for OSx #840

Closed s1113950 closed 3 years ago

s1113950 commented 6 years ago

RKE version: I tried v0.1.8 and v0.1.9-rc6

Docker version: (docker version,docker info preferred)

Client:
 Version:      17.03.1-ce
 API version:  1.27
 Go version:   go1.7.5
 Git commit:   c6d412e
 Built:        Tue Mar 28 00:40:02 2017
 OS/Arch:      darwin/amd64

Server:
 Version:      17.03.1-ce
 API version:  1.27 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   c6d412e
 Built:        Fri Mar 24 00:00:50 2017
 OS/Arch:      linux/amd64
 Experimental: true

Operating system and kernel: (cat /etc/os-release, uname -r preferred) OSX 10.12.6 uname -r: 16.7.0

Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO) Bare-metal on my Mac

cluster.yml file:

nodes:
- address: localhost
  port: "22"
  internal_address: 127.0.0.1
  role:
  - controlplane
  - worker
  - etcd
  user: robertso
  docker_socket: /var/run/docker.sock
  ssh_key: ""
  ssh_key_path: ~/.ssh/id_rsa
  labels: {}
services:
  etcd:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
    external_urls: []
    ca_cert: ""
    cert: ""
    key: ""
    path: ""
    snapshot: false
    retention: ""
    creation: ""
  kube-api:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
    service_cluster_ip_range: 10.43.0.0/16
    service_node_port_range: ""
    pod_security_policy: false
  kube-controller:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
    cluster_cidr: 10.42.0.0/16
    service_cluster_ip_range: 10.43.0.0/16
  scheduler:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
  kubelet:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
    cluster_domain: cluster.local
    infra_container_image: ""
    cluster_dns_server: 10.43.0.10
    fail_swap_on: false
  kubeproxy:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
network:
  plugin: canal
  options: {}
authentication:
  strategy: x509
  options: {}
  sans: []
addons: ""
addons_include: []
system_images:
  etcd: rancher/coreos-etcd:v3.1.12
  alpine: rancher/rke-tools:v0.1.10
  nginx_proxy: rancher/rke-tools:v0.1.10
  cert_downloader: rancher/rke-tools:v0.1.10
  kubernetes_services_sidecar: rancher/rke-tools:v0.1.10
  kubedns: rancher/k8s-dns-kube-dns-amd64:1.14.8
  dnsmasq: rancher/k8s-dns-dnsmasq-nanny-amd64:1.14.8
  kubedns_sidecar: rancher/k8s-dns-sidecar-amd64:1.14.8
  kubedns_autoscaler: rancher/cluster-proportional-autoscaler-amd64:1.0.0
  kubernetes: rancher/hyperkube:v1.10.5-rancher1
  flannel: rancher/coreos-flannel:v0.9.1
  flannel_cni: rancher/coreos-flannel-cni:v0.2.0
  calico_node: rancher/calico-node:v3.1.1
  calico_cni: rancher/calico-cni:v3.1.1
  calico_controllers: ""
  calico_ctl: rancher/calico-ctl:v2.0.0
  canal_node: rancher/calico-node:v3.1.1
  canal_cni: rancher/calico-cni:v3.1.1
  canal_flannel: rancher/coreos-flannel:v0.9.1
  wave_node: weaveworks/weave-kube:2.1.2
  weave_cni: weaveworks/weave-npc:2.1.2
  pod_infra_container: rancher/pause-amd64:3.1
  ingress: rancher/nginx-ingress-controller:0.10.2-rancher3
  ingress_backend: rancher/nginx-ingress-controller-defaultbackend:1.4
ssh_key_path: ~/.ssh/id_rsa
ssh_agent_auth: false
authorization:
  mode: rbac
  options: {}
ignore_docker_version: false
kubernetes_version: ""
private_registries: []
ingress:
  provider: ""
  options: {}
  node_selector: {}
  extra_args: {}
cluster_name: ""
cloud_provider:
  name: ""
prefix_path: ""
addon_job_timeout: 0
bastion_host:
  address: ""
  port: ""
  user: ""
  ssh_key: ""
  ssh_key_path: ""

Steps to Reproduce: rke up --config local_cluster.yml

Results: If I run that command once, I get this output:

$ rke up --config local_cluster.yml
WARN[0000] This is not an officially supported version (v0.1.9-rc6) of RKE. Please download the latest official release at https://github.com/rancher/rke/releases/latest
INFO[0000] Building Kubernetes cluster
INFO[0000] [dialer] Setup tunnel for host [localhost]
INFO[0000] [network] Deploying port listener containers
INFO[0001] [network] Successfully updated [rke-etcd-port-listener] container on host [localhost]
INFO[0002] [network] Successfully started [rke-cp-port-listener] container on host [localhost]
INFO[0003] [network] Successfully updated [rke-worker-port-listener] container on host [localhost]
INFO[0003] [network] Port listener containers deployed successfully
INFO[0003] [network] Running control plane -> etcd port checks
INFO[0004] [network] Successfully started [rke-port-checker] container on host [localhost]
INFO[0005] [network] Running control plane -> worker port checks
INFO[0005] [network] Successfully started [rke-port-checker] container on host [localhost]
INFO[0005] [network] Running workers -> control plane port checks
INFO[0006] [network] Successfully started [rke-port-checker] container on host [localhost]
INFO[0006] [network] Checking KubeAPI port Control Plane hosts
INFO[0006] [network] Removing port listener containers
INFO[0007] [remove/rke-etcd-port-listener] Successfully removed container on host [localhost]
INFO[0008] [remove/rke-cp-port-listener] Successfully removed container on host [localhost]
INFO[0008] [remove/rke-worker-port-listener] Successfully removed container on host [localhost]
INFO[0008] [network] Port listener containers removed successfully
INFO[0008] [certificates] Attempting to recover certificates from backup on [etcd,controlPlane] hosts
INFO[0008] [certificates] No Certificate backup found on [etcd,controlPlane] hosts
INFO[0008] [certificates] Generating CA kubernetes certificates
INFO[0009] [certificates] Generating Kubernetes API server certificates
INFO[0009] [certificates] Generating Kube Controller certificates
INFO[0009] [certificates] Generating Kube Scheduler certificates
INFO[0009] [certificates] Generating Kube Proxy certificates
INFO[0010] [certificates] Generating Node certificate
INFO[0010] [certificates] Generating admin certificates and kubeconfig
INFO[0010] [certificates] Generating etcd-127.0.0.1 certificate and key
INFO[0010] [certificates] Generating Kubernetes API server aggregation layer requestheader client CA certificates
INFO[0011] [certificates] Generating Kubernetes API server proxy client certificates
INFO[0011] [certificates] Temporarily saving certs to [etcd,controlPlane] hosts
INFO[0016] [certificates] Saved certs to [etcd,controlPlane] hosts
INFO[0016] [reconcile] Reconciling cluster state
INFO[0016] [reconcile] This is newly generated cluster
INFO[0016] [certificates] Deploying kubernetes certificates to Cluster nodes
INFO[0022] Successfully Deployed local admin kubeconfig at [./kube_config_local_cluster.yml]
INFO[0022] [certificates] Successfully deployed kubernetes certificates to Cluster nodes
INFO[0022] Pre-pulling kubernetes images
INFO[0022] Kubernetes images pulled successfully
INFO[0022] [etcd] Building up etcd plane..
INFO[0023] [etcd] Successfully started [etcd] container on host [localhost]
INFO[0023] [etcd] Successfully started [rke-log-linker] container on host [localhost]
INFO[0024] [remove/rke-log-linker] Successfully removed container on host [localhost]
INFO[0024] [etcd] Successfully started etcd plane..
INFO[0024] [controlplane] Building up Controller Plane..
INFO[0025] [controlplane] Successfully started [kube-apiserver] container on host [localhost]
INFO[0025] [healthcheck] Start Healthcheck on service [kube-apiserver] on host [localhost]
FATA[0076] [controlPlane] Failed to bring up Control Plane: Failed to verify healthcheck: Failed to check https://localhost:6443/healthz for service [kube-apiserver] on host [localhost]: Get https://localhost:6443/healthz: Unable to access the service on localhost:6443. The service might be still starting up. Error: ssh: rejected: connect failed (Connection refused), log: I0803 22:13:15.311700       1 plugins.go:149] Loaded 7 admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultTolerationSeconds,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota.

I then see 2 docker containers created:

$ docker ps
CONTAINER ID        IMAGE                                COMMAND                  CREATED              STATUS                          PORTS               NAMES
b4e912fac617        rancher/hyperkube:v1.10.5-rancher1   "/opt/rke/entrypoi..."   About a minute ago   Up 8 seconds                                        kube-apiserver
10067fb5587b        rancher/coreos-etcd:v3.1.12          "/usr/local/bin/et..."   About a minute ago   Restarting (1) 34 seconds ago                       etcd

Logs from apiserver that keep looping:

I0803 22:14:00.007527       1 server.go:135] Version: v1.10.5
I0803 22:14:00.008062       1 server.go:724] external host was not specified, using 192.168.65.2
W0803 22:14:00.458600       1 admission.go:68] PersistentVolumeLabel admission controller is deprecated. Please remove this controller from your configuration files and scripts.
I0803 22:14:00.458814       1 plugins.go:149] Loaded 7 admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultTolerationSeconds,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota.
W0803 22:14:00.462101       1 admission.go:68] PersistentVolumeLabel admission controller is deprecated. Please remove this controller from your configuration files and scripts.
I0803 22:14:00.462376       1 plugins.go:149] Loaded 7 admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultTolerationSeconds,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota.
F0803 22:14:10.470612       1 storage_decorator.go:57] Unable to create storage backend: config (&{etcd3 /registry [https://127.0.0.1:2379] /etc/kubernetes/ssl/kube-node-key.pem /etc/kubernetes/ssl/kube-node.pem /etc/kubernetes/ssl/kube-ca.pem true false 1000 0xc4200c8500 <nil> 5m0s 1m0s}), err (dial tcp 127.0.0.1:2379: getsockopt: connection refused)
+ echo kube-apiserver --secure-port=6443 --service-account-key-file=/etc/kubernetes/ssl/kube-apiserver-key.pem --endpoint-reconciler-type=lease --bind-address=0.0.0.0 --kubelet-client-key=/etc/kubernetes/ssl/kube-apiserver-key.pem --admission-control=ServiceAccount,NamespaceLifecycle,LimitRanger,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota,DefaultTolerationSeconds --requestheader-group-headers=X-Remote-Group --tls-cert-file=/etc/kubernetes/ssl/kube-apiserver.pem --tls-private-key-file=/etc/kubernetes/ssl/kube-apiserver-key.pem --etcd-keyfile=/etc/kubernetes/ssl/kube-node-key.pem --storage-backend=etcd3 --requestheader-extra-headers-prefix=X-Remote-Extra- --kubelet-client-certificate=/etc/kubernetes/ssl/kube-apiserver.pem --proxy-client-key-file=/etc/kubernetes/ssl/kube-apiserver-proxy-client-key.pem --authorization-mode=Node,RBAC --insecure-bind-address=127.0.0.1 --requestheader-allowed-names=kube-apiserver-proxy-client --requestheader-username-headers=X-Remote-User --etcd-prefix=/r+ grep -q cloud-provider=azure
egistry --requestheader-client-ca-file=/etc/kubernetes/ssl/kube-apiserver-requestheader-ca.pem --insecure-port=0 --allow-privileged=true --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --client-ca-file=/etc/kubernetes/ssl/kube-ca.pem --etcd-cafile=/etc/kubernetes/ssl/kube-ca.pem --etcd-servers=https://127.0.0.1:2379 --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --cloud-provider= --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --etcd-certfile=/etc/kubernetes/ssl/kube-node.pem --proxy-client-cert-file=/etc/kubernetes/ssl/kube-apiserver-proxy-client.pem
+ '[' kube-apiserver = kubelet ']'
+ exec kube-apiserver --secure-port=6443 --service-account-key-file=/etc/kubernetes/ssl/kube-apiserver-key.pem --endpoint-reconciler-type=lease --bind-address=0.0.0.0 --kubelet-client-key=/etc/kubernetes/ssl/kube-apiserver-key.pem --admission-control=ServiceAccount,NamespaceLifecycle,LimitRanger,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota,DefaultTolerationSeconds --requestheader-group-headers=X-Remote-Group --tls-cert-file=/etc/kubernetes/ssl/kube-apiserver.pem --tls-private-key-file=/etc/kubernetes/ssl/kube-apiserver-key.pem --etcd-keyfile=/etc/kubernetes/ssl/kube-node-key.pem --storage-backend=etcd3 --requestheader-extra-headers-prefix=X-Remote-Extra- --kubelet-client-certificate=/etc/kubernetes/ssl/kube-apiserver.pem --proxy-client-key-file=/etc/kubernetes/ssl/kube-apiserver-proxy-client-key.pem --authorization-mode=Node,RBAC --insecure-bind-address=127.0.0.1 --requestheader-allowed-names=kube-apiserver-proxy-client --requestheader-username-headers=X-Remote-User --etcd-prefix=/registry --requestheader-client-ca-file=/etc/kubernetes/ssl/kube-apiserver-requestheader-ca.pem --insecure-port=0 --allow-privileged=true --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --client-ca-file=/etc/kubernetes/ssl/kube-ca.pem --etcd-cafile=/etc/kubernetes/ssl/kube-ca.pem --etcd-servers=https://127.0.0.1:2379 --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --cloud-provider= --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --etcd-certfile=/etc/kubernetes/ssl/kube-node.pem --proxy-client-cert-file=/etc/kubernetes/ssl/kube-apiserver-proxy-client.pem

Specifically the part about Unable to create storage backend, so I checked the etcd logs:

robertso@robertso-mac02:~/Nervana/k8-cluster-creation/rancher$ docker logs 10067fb5587b
2018-08-03 22:12:27.111720 I | etcdmain: etcd Version: 3.1.12
2018-08-03 22:12:27.111829 I | etcdmain: Git SHA: 918698add
2018-08-03 22:12:27.111836 I | etcdmain: Go Version: go1.8.7
2018-08-03 22:12:27.111842 I | etcdmain: Go OS/Arch: linux/amd64
2018-08-03 22:12:27.111849 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
2018-08-03 22:12:27.113048 I | embed: peerTLS: cert = /etc/kubernetes/ssl/kube-etcd-127-0-0-1.pem, key = /etc/kubernetes/ssl/kube-etcd-127-0-0-1-key.pem, ca = , trusted-ca = /etc/kubernetes/ssl/kube-ca.pem, client-cert-auth = true
2018-08-03 22:12:27.128374 I | embed: listening for peers on https://127.0.0.1:2380
2018-08-03 22:12:27.128460 I | embed: listening for client requests on 127.0.0.1:2379
2018-08-03 22:12:27.130207 C | etcdmain: cannot access data directory: open /var/lib/rancher/etcd/.touch: permission denied
2018-08-03 22:12:27.678095 I | etcdmain: etcd Version: 3.1.12
2018-08-03 22:12:27.678165 I | etcdmain: Git SHA: 918698add
2018-08-03 22:12:27.678171 I | etcdmain: Go Version: go1.8.7
2018-08-03 22:12:27.678179 I | etcdmain: Go OS/Arch: linux/amd64
2018-08-03 22:12:27.678184 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
2018-08-03 22:12:27.679273 I | embed: peerTLS: cert = /etc/kubernetes/ssl/kube-etcd-127-0-0-1.pem, key = /etc/kubernetes/ssl/kube-etcd-127-0-0-1-key.pem, ca = , trusted-ca = /etc/kubernetes/ssl/kube-ca.pem, client-cert-auth = true
2018-08-03 22:12:27.684777 I | embed: listening for peers on https://127.0.0.1:2380
2018-08-03 22:12:27.684820 I | embed: listening for client requests on 127.0.0.1:2379
2018-08-03 22:12:27.685814 C | etcdmain: cannot access data directory: open /var/lib/rancher/etcd/.touch: permission denied
2018-08-03 22:12:28.347438 I | etcdmain: etcd Version: 3.1.12
2018-08-03 22:12:28.347537 I | etcdmain: Git SHA: 918698add
2018-08-03 22:12:28.347545 I | etcdmain: Go Version: go1.8.7
2018-08-03 22:12:28.347552 I | etcdmain: Go OS/Arch: linux/amd64
2018-08-03 22:12:28.347561 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
2018-08-03 22:12:28.348738 I | embed: peerTLS: cert = /etc/kubernetes/ssl/kube-etcd-127-0-0-1.pem, key = /etc/kubernetes/ssl/kube-etcd-127-0-0-1-key.pem, ca = , trusted-ca = /etc/kubernetes/ssl/kube-ca.pem, client-cert-auth = true
2018-08-03 22:12:28.356643 I | embed: listening for peers on https://127.0.0.1:2380
2018-08-03 22:12:28.356723 I | embed: listening for client requests on 127.0.0.1:2379
2018-08-03 22:12:28.358124 C | etcdmain: cannot access data directory: open /var/lib/rancher/etcd/.touch: permission denied
2018-08-03 22:12:29.163920 I | etcdmain: etcd Version: 3.1.12
2018-08-03 22:12:29.163977 I | etcdmain: Git SHA: 918698add
2018-08-03 22:12:29.163984 I | etcdmain: Go Version: go1.8.7
2018-08-03 22:12:29.163990 I | etcdmain: Go OS/Arch: linux/amd64
2018-08-03 22:12:29.164020 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
2018-08-03 22:12:29.165287 I | embed: peerTLS: cert = /etc/kubernetes/ssl/kube-etcd-127-0-0-1.pem, key = /etc/kubernetes/ssl/kube-etcd-127-0-0-1-key.pem, ca = , trusted-ca = /etc/kubernetes/ssl/kube-ca.pem, client-cert-auth = true
2018-08-03 22:12:29.172153 I | embed: listening for peers on https://127.0.0.1:2380
2018-08-03 22:12:29.172281 I | embed: listening for client requests on 127.0.0.1:2379
2018-08-03 22:12:29.173569 C | etcdmain: cannot access data directory: open /var/lib/rancher/etcd/.touch: permission denied
2018-08-03 22:12:30.452382 I | etcdmain: etcd Version: 3.1.12
2018-08-03 22:12:30.452470 I | etcdmain: Git SHA: 918698add
2018-08-03 22:12:30.452476 I | etcdmain: Go Version: go1.8.7
2018-08-03 22:12:30.452482 I | etcdmain: Go OS/Arch: linux/amd64
2018-08-03 22:12:30.452486 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
2018-08-03 22:12:30.453598 I | embed: peerTLS: cert = /etc/kubernetes/ssl/kube-etcd-127-0-0-1.pem, key = /etc/kubernetes/ssl/kube-etcd-127-0-0-1-key.pem, ca = , trusted-ca = /etc/kubernetes/ssl/kube-ca.pem, client-cert-auth = true
2018-08-03 22:12:30.459962 I | embed: listening for peers on https://127.0.0.1:2380
2018-08-03 22:12:30.460019 I | embed: listening for client requests on 127.0.0.1:2379
2018-08-03 22:12:30.461177 C | etcdmain: cannot access data directory: open /var/lib/rancher/etcd/.touch: permission denied
2018-08-03 22:12:32.433139 I | etcdmain: etcd Version: 3.1.12
2018-08-03 22:12:32.433260 I | etcdmain: Git SHA: 918698add
2018-08-03 22:12:32.433267 I | etcdmain: Go Version: go1.8.7
2018-08-03 22:12:32.433273 I | etcdmain: Go OS/Arch: linux/amd64
2018-08-03 22:12:32.433279 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
2018-08-03 22:12:32.434354 I | embed: peerTLS: cert = /etc/kubernetes/ssl/kube-etcd-127-0-0-1.pem, key = /etc/kubernetes/ssl/kube-etcd-127-0-0-1-key.pem, ca = , trusted-ca = /etc/kubernetes/ssl/kube-ca.pem, client-cert-auth = true
2018-08-03 22:12:32.442398 I | embed: listening for peers on https://127.0.0.1:2380
2018-08-03 22:12:32.442448 I | embed: listening for client requests on 127.0.0.1:2379
2018-08-03 22:12:32.443441 C | etcdmain: cannot access data directory: open /var/lib/rancher/etcd/.touch: permission denied
2018-08-03 22:12:36.019167 I | etcdmain: etcd Version: 3.1.12
2018-08-03 22:12:36.019220 I | etcdmain: Git SHA: 918698add
2018-08-03 22:12:36.019227 I | etcdmain: Go Version: go1.8.7
2018-08-03 22:12:36.019236 I | etcdmain: Go OS/Arch: linux/amd64
2018-08-03 22:12:36.019243 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
2018-08-03 22:12:36.020250 I | embed: peerTLS: cert = /etc/kubernetes/ssl/kube-etcd-127-0-0-1.pem, key = /etc/kubernetes/ssl/kube-etcd-127-0-0-1-key.pem, ca = , trusted-ca = /etc/kubernetes/ssl/kube-ca.pem, client-cert-auth = true
2018-08-03 22:12:36.029823 I | embed: listening for peers on https://127.0.0.1:2380
2018-08-03 22:12:36.029878 I | embed: listening for client requests on 127.0.0.1:2379
2018-08-03 22:12:36.030907 C | etcdmain: cannot access data directory: open /var/lib/rancher/etcd/.touch: permission denied
2018-08-03 22:12:42.786376 I | etcdmain: etcd Version: 3.1.12
2018-08-03 22:12:42.786433 I | etcdmain: Git SHA: 918698add
2018-08-03 22:12:42.786440 I | etcdmain: Go Version: go1.8.7
2018-08-03 22:12:42.786450 I | etcdmain: Go OS/Arch: linux/amd64
2018-08-03 22:12:42.786457 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
2018-08-03 22:12:42.787538 I | embed: peerTLS: cert = /etc/kubernetes/ssl/kube-etcd-127-0-0-1.pem, key = /etc/kubernetes/ssl/kube-etcd-127-0-0-1-key.pem, ca = , trusted-ca = /etc/kubernetes/ssl/kube-ca.pem, client-cert-auth = true
2018-08-03 22:12:42.793604 I | embed: listening for peers on https://127.0.0.1:2380
2018-08-03 22:12:42.793650 I | embed: listening for client requests on 127.0.0.1:2379
2018-08-03 22:12:42.794676 C | etcdmain: cannot access data directory: open /var/lib/rancher/etcd/.touch: permission denied
2018-08-03 22:12:55.954731 I | etcdmain: etcd Version: 3.1.12
2018-08-03 22:12:55.954787 I | etcdmain: Git SHA: 918698add
2018-08-03 22:12:55.954794 I | etcdmain: Go Version: go1.8.7
2018-08-03 22:12:55.954801 I | etcdmain: Go OS/Arch: linux/amd64
2018-08-03 22:12:55.954807 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
2018-08-03 22:12:55.955694 I | embed: peerTLS: cert = /etc/kubernetes/ssl/kube-etcd-127-0-0-1.pem, key = /etc/kubernetes/ssl/kube-etcd-127-0-0-1-key.pem, ca = , trusted-ca = /etc/kubernetes/ssl/kube-ca.pem, client-cert-auth = true
2018-08-03 22:12:55.962397 I | embed: listening for peers on https://127.0.0.1:2380
2018-08-03 22:12:55.962527 I | embed: listening for client requests on 127.0.0.1:2379
2018-08-03 22:12:55.963512 C | etcdmain: cannot access data directory: open /var/lib/rancher/etcd/.touch: permission denied
2018-08-03 22:13:21.907721 I | etcdmain: etcd Version: 3.1.12
2018-08-03 22:13:21.907777 I | etcdmain: Git SHA: 918698add
2018-08-03 22:13:21.907784 I | etcdmain: Go Version: go1.8.7
2018-08-03 22:13:21.907793 I | etcdmain: Go OS/Arch: linux/amd64
2018-08-03 22:13:21.907798 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
2018-08-03 22:13:21.908753 I | embed: peerTLS: cert = /etc/kubernetes/ssl/kube-etcd-127-0-0-1.pem, key = /etc/kubernetes/ssl/kube-etcd-127-0-0-1-key.pem, ca = , trusted-ca = /etc/kubernetes/ssl/kube-ca.pem, client-cert-auth = true
2018-08-03 22:13:21.915046 I | embed: listening for peers on https://127.0.0.1:2380
2018-08-03 22:13:21.915166 I | embed: listening for client requests on 127.0.0.1:2379
2018-08-03 22:13:21.916127 C | etcdmain: cannot access data directory: open /var/lib/rancher/etcd/.touch: permission denied
2018-08-03 22:14:13.506025 I | etcdmain: etcd Version: 3.1.12
2018-08-03 22:14:13.506108 I | etcdmain: Git SHA: 918698add
2018-08-03 22:14:13.506116 I | etcdmain: Go Version: go1.8.7
2018-08-03 22:14:13.506121 I | etcdmain: Go OS/Arch: linux/amd64
2018-08-03 22:14:13.506125 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
2018-08-03 22:14:13.507570 I | embed: peerTLS: cert = /etc/kubernetes/ssl/kube-etcd-127-0-0-1.pem, key = /etc/kubernetes/ssl/kube-etcd-127-0-0-1-key.pem, ca = , trusted-ca = /etc/kubernetes/ssl/kube-ca.pem, client-cert-auth = true
2018-08-03 22:14:13.518597 I | embed: listening for peers on https://127.0.0.1:2380
2018-08-03 22:14:13.518677 I | embed: listening for client requests on 127.0.0.1:2379
2018-08-03 22:14:13.520793 C | etcdmain: cannot access data directory: open /var/lib/rancher/etcd/.touch: permission denied

I tried creating the dir /var/lib/rancher/etcd/ and making it owned by my user but that didn't work either. I read some github issues that mention that I might just have to rerun rke up, so I did that again:

robertso@robertso-mac02:~/Nervana/k8-cluster-creation/rancher$ rke up --config local_cluster.yml
WARN[0000] This is not an officially supported version (v0.1.9-rc6) of RKE. Please download the latest official release at https://github.com/rancher/rke/releases/latest
INFO[0000] Building Kubernetes cluster
INFO[0000] [dialer] Setup tunnel for host [localhost]
INFO[0000] [state] Found local kube config file, trying to get state from cluster
INFO[0000] [reconcile] Local config is not vaild, rebuilding admin config
INFO[0000] [reconcile] Rebuilding and updating local kube config
INFO[0000] Successfully Deployed local admin kubeconfig at [./kube_config_local_cluster.yml]
INFO[0000] [state] Fetching cluster state from Kubernetes
INFO[0030] Timed out waiting for kubernetes cluster to get state
INFO[0030] [network] Deploying port listener containers
INFO[0030] [network] Successfully started [rke-etcd-port-listener] container on host [localhost]
INFO[0032] [network] Successfully started [rke-worker-port-listener] container on host [localhost]
INFO[0032] [network] Port listener containers deployed successfully
INFO[0032] [network] Running control plane -> etcd port checks
INFO[0032] [network] Successfully started [rke-port-checker] container on host [localhost]
INFO[0033] [network] Running control plane -> worker port checks
INFO[0033] [network] Successfully started [rke-port-checker] container on host [localhost]
INFO[0034] [network] Running workers -> control plane port checks
INFO[0034] [network] Successfully started [rke-port-checker] container on host [localhost]
INFO[0035] [network] Checking KubeAPI port Control Plane hosts
FATA[0035] [network] Can't access KubeAPI port [6443] on Control Plane host: localhost

Then I checked docker containers again:

robertso@robertso-mac02:~/Nervana/k8-cluster-creation/rancher$ docker ps
CONTAINER ID        IMAGE                                COMMAND                  CREATED              STATUS                         PORTS                                                    NAMES
f9cdd1b761a1        rancher/rke-tools:v0.1.10            "nc -kl -p 1337 -e..."   About a minute ago   Up About a minute              80/tcp, 0.0.0.0:10250->1337/tcp                          rke-worker-port-listener
f23cef0c1154        rancher/rke-tools:v0.1.10            "nc -kl -p 1337 -e..."   About a minute ago   Up About a minute              80/tcp, 0.0.0.0:2379->1337/tcp, 0.0.0.0:2380->1337/tcp   rke-etcd-port-listener
b4e912fac617        rancher/hyperkube:v1.10.5-rancher1   "/opt/rke/entrypoi..."   5 minutes ago        Up 10 seconds                                                                           kube-apiserver
10067fb5587b        rancher/coreos-etcd:v3.1.12          "/usr/local/bin/et..."   5 minutes ago        Restarting (1) 2 minutes ago                                                            etcd

and notice the port listeners are now up. I see the same error in the etcd logs and apiserver

Any help is appreciated! End goal of our project is to see if we can leverage rke to create clusters we need on aws, bare metal and gke, but I can't get past bare metal.

stiks commented 6 years ago

I have had a similar issue when real hostname wasn't matching rke hostname

s1113950 commented 6 years ago

Thanks for the tip! I'll double check that

s1113950 commented 6 years ago

I tried 127.0.0.1 just in case for the address but still no luck. Not sure where to go from here :( I want to create a cluster locally so either 127.0.0.1 or localhost should work in theory

s1113950 commented 6 years ago

Is there an example deployment of a bare metal cluster? https://rancher.com/docs/rke/v0.1.x/en/config-options/nodes/ and https://rancher.com/docs/rke/v0.1.x/en/example-yamls/ don't specify bare metal specifically

stiks commented 6 years ago

my bad, I haven't seen that you trying to run cluster on localhost. Better to use virtual IP (just create new interface) or use VM. I believe when rancher is starting control plane, it actually create proxy service on localhost (just in case you have more that one control plane). Actual control plane wouldn't start because of control place IP in that case also 127.0.0.1.

Try different IP, like 127.0.0.2

s1113950 commented 6 years ago

I created an alias to the loopback adapter as shown here: https://stackoverflow.com/a/624660

And then that resulted in (after the second time running rke up)

$ rke up --config local_cluster.yml
WARN[0000] This is not an officially supported version (v0.1.9-rc6) of RKE. Please download the latest official release at https://github.com/rancher/rke/releases/latest
INFO[0000] Building Kubernetes cluster
INFO[0000] [dialer] Setup tunnel for host [127.0.0.2]
INFO[0000] [state] Found local kube config file, trying to get state from cluster
INFO[0000] [reconcile] Local config is not vaild, rebuilding admin config
INFO[0000] [reconcile] Rebuilding and updating local kube config
INFO[0000] Successfully Deployed local admin kubeconfig at [./kube_config_local_cluster.yml]
INFO[0000] [state] Fetching cluster state from Kubernetes
INFO[0030] Timed out waiting for kubernetes cluster to get state
INFO[0030] [network] Deploying port listener containers
INFO[0031] [network] Successfully started [rke-etcd-port-listener] container on host [127.0.0.2]
INFO[0032] [network] Successfully started [rke-worker-port-listener] container on host [127.0.0.2]
INFO[0032] [network] Port listener containers deployed successfully
INFO[0032] [network] Running control plane -> etcd port checks
INFO[0032] [network] Successfully started [rke-port-checker] container on host [127.0.0.2]
INFO[0033] [network] Running control plane -> worker port checks
INFO[0033] [network] Successfully started [rke-port-checker] container on host [127.0.0.2]
INFO[0034] [network] Running workers -> control plane port checks
INFO[0034] [network] Successfully started [rke-port-checker] container on host [127.0.0.2]
FATA[0035] [network] Host [127.0.0.2] is not able to connect to the following ports: [127.0.0.1:6443]. Please check network policies and firewall rules

Is this because it's not a true virtual IP or is it because of something on my end (I'm at work)? @stiks

moelsayed commented 6 years ago

@s1113950 Unfortunately, local mode is currently supported on Linux based machines only. MacOS is not supported.

s1113950 commented 6 years ago

Thanks for the info! Is there a timeframe for when that would be supported? Maybe I missed a README somewhere that told me this 😕

alena1108 commented 6 years ago

@s1113950 it's not in the immediate plans, although I agree it would be nice to have. Will keep this issue as an enhancement request

stale[bot] commented 3 years ago

This issue/PR has been automatically marked as stale because it has not had activity (commit/comment/label) for 60 days. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

kknd22 commented 3 years ago

local mode is currently supported on Linux based machines only. MacOS is not supported.

still not support? - 3 years passed...