search

rancher / rke

Rancher Kubernetes Engine (RKE), an extremely simple, lightning fast Kubernetes distribution that runs entirely within containers.
Apache License 2.0
3.22k stars 583 forks source link

cluster certificate not generated by default? #546

Closed joesonw closed 6 years ago

joesonw commented 6 years ago

RKE version: v0.1.6-rc2 Docker version: (docker version,docker info preferred)

 Version:         1.13.1
 API version:     1.26 (minimum version 1.12)
 Package version: <unknown>
 Go version:      go1.8.3
 Git commit:      774336d/1.13.1
 Built:           Wed Mar  7 17:06:16 2018
 OS/Arch:         linux/amd64
 Experimental:    false

Operating system and kernel: (cat /etc/os-release, uname -r preferred)

NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"

Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO) KVM cluster.yml file:

# If you intened to deploy Kubernetes in an air-gapped environment,
# please consult the documentation on how to configure custom RKE images.
nodes:
- address: 172.16.10.178
  port: "22"
  internal_address: ""
  role:
  - controlplane
  - worker
  - etcd
  hostname_override: k8s-1
  user: docker
  docker_socket: /var/run/docker.sock
  ssh_key: ""
  ssh_key_path: "" 
  labels: {}
- address: 172.16.10.179
  port: "22"
  internal_address: ""
  role:
  - worker
  hostname_override: k8s-2
  user: docker
  docker_socket: /var/run/docker.sock
  ssh_key: ""
  ssh_key_path: "" 
  labels: {}
- address: 172.16.10.180
  port: "22"
  internal_address: ""
  role:
  - worker
  hostname_override: k8s-3
  user: docker 
  docker_socket: /var/run/docker.sock
  ssh_key: ""
  ssh_key_path: "" 
  labels: {}
services:
  etcd:
    image: rancher/coreos-etcd:v3.1.12
    extra_args: {}
    extra_binds: []
    external_urls: []
    ca_cert: ""
    cert: ""
    key: ""
    path: ""
  kube-api:
    image: rancher/hyperkube:v1.10.1
    #image: rancher/k8s:v1.10.0-rancher1-1
    extra_args:
      admission-control: NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,NodeRestriction,Priority
    extra_binds: []
    service_cluster_ip_range: 10.43.0.0/16
    pod_security_policy: false
  kube-controller:
    image: rancher/hyperkube:v1.10.1
    #image: rancher/k8s:v1.10.0-rancher1-1
    extra_args: {}
    extra_binds: []
    cluster_cidr: 10.42.0.0/16
    service_cluster_ip_range: 10.43.0.0/16
  scheduler:
    image: rancher/hyperkube:v1.10.1
    #image: rancher/k8s:v1.10.0-rancher1-1
    extra_args: {}
    extra_binds: []
  kubelet:
    image: rancher/hyperkube:v1.10.1
    #image: rancher/k8s:v1.10.0-rancher1-1
    extra_args: {}
    extra_binds: []
    cluster_domain: cluster.local
    infra_container_image: rancher/pause-amd64:3.1
    cluster_dns_server: 10.43.0.10
    fail_swap_on: false
  kubeproxy:
    image: rancher/hyperkube:v1.10.1
    #image: rancher/k8s:v1.10.0-rancher1-1
    extra_args: {}
    extra_binds: []
network:
  plugin: calico
  options: {}
authentication:
  strategy: x509
  options: {}
  sans: []
addons: ""
addons_include: []
system_images:
  etcd: ""
  alpine: ""
  nginx_proxy: ""
  cert_downloader: ""
  kubernetes_services_sidecar: ""
  kubedns: ""
  dnsmasq: ""
  kubedns_sidecar: ""
  kubedns_autoscaler: ""
  kubernetes: ""
  flannel: ""
  flannel_cni: ""
  calico_node: ""
  calico_cni: ""
  calico_controllers: ""
  calico_ctl: ""
  canal_node: ""
  canal_cni: ""
  canal_flannel: ""
  wave_node: ""
  weave_cni: ""
  pod_infra_container: ""
  ingress: ""
  ingress_backend: ""
  dashboard: ""
  heapster: ""
  grafana: ""
  influxdb: ""
  tiller: ""
ssh_key_path: /Users/Joesonw/funlearnworld/chinaworld/devops/k8s_cluster_private_key
ssh_agent_auth: false
authorization:
  mode: rbac
  options: {}
ignore_docker_version: false
kubernetes_version: ""
private_registries: []
ingress:
  provider: ""
  options: {}
  node_selector: {}
  extra_args: {}
cluster_name: ""
cloud_provider:
  name: ""
  cloud_config: {}
prefix_path: ""

Steps to Reproduce: rke up Results: Success, But when I was installing istio with automatic sidecar injector. It appears that controller manager signer was not enabled by default. https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/#a-note-to-cluster-administrators

I am far from a vet of k8s, these configurations did confuse me a bit.

How would I able to provide such cert/key?

I tried to generate my own cert/key, put them on master node, tuned my cluster.yml, and pass these extra_args to kube-controller, it did start up, but it then gives me the following error when launching new pods.

Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"default", Name:"sleep-86f6b99f94", UID:"8be4a994-482c-11e8-b466-00163e10651f", APIVersion:"extensions", ResourceVersion:"3621", FieldPath:""}): type: 'Warning' reason: 'FailedCreate' Error creating: Internal error occurred: failed calling admission webhook "sidecar-injector.istio.io": Post https://istio-sidecar-injector.istio-system.svc:443/inject?timeout=30s: x509: certificate signed by unknown authority
galal-hussein commented 6 years ago

@joesonw you can pass any extra arguments to any service using extra_args, for example:

....
services:
  kube-api:
    image: rancher/hyperkube:v1.10.1
    extra_args:
      cluster-signing-cert-file: <path-of-signing-cert>
      cluster-signing-key-file: <path-of-signing-key>
.....
andrexus commented 6 years ago

Any plans to make it configurable also in the UI?

galal-hussein commented 6 years ago

@andrexus opened an issue in rancher/rancher repo, thanks for the suggestion

cgebe commented 6 years ago

Where does this issue continue? I am trying to state a CSR against my API, but certificates are not issued. How can i solve this, either via the UI or via kubectl? thanks

Edit: How can i get my certificates into the rancher container and refer to it in the cluster.yml?

qrtt1 commented 5 years ago

signer is provided by the contoller-manager, you need add cert and its key to kube-controller 

--- cluster.yml 2019-07-22 06:00:29.188527946 +0000
+++ cluster.new.yml 2019-07-22 06:00:18.681650925 +0000
@@ -56,6 +56,9 @@
   kube-controller:
     cluster_cidr: 10.233.64.0/18
     service_cluster_ip_range: 10.233.0.0/18
+    extra_args:
+      cluster-signing-cert-file: /etc/kubernetes/ssl/kube-ca.pem
+      cluster-signing-key-file: /etc/kubernetes/ssl/kube-ca-key.pem
   scheduler:
   kubelet:
     cluster_domain: cluster.local
vitobotta commented 5 years ago

Hi! I am trying to get Robin storage working on a cluster deployed with Rancher, and I have been told that I need to "enable certificate controller manager on the k8s cluster". Is that couple of args what this is about? Can I do this with an existing cluster? And how do I generate the certificate? Thanks

titou10titou10 commented 4 years ago

could the csr signing feature be activated by default in a fresh rke install? thx

themowski commented 3 years ago

signer is provided by the contoller-manager, you need add cert and its key to kube-controller

--- cluster.yml   2019-07-22 06:00:29.188527946 +0000
+++ cluster.new.yml   2019-07-22 06:00:18.681650925 +0000
@@ -56,6 +56,9 @@
   kube-controller:
     cluster_cidr: 10.233.64.0/18
     service_cluster_ip_range: 10.233.0.0/18
+    extra_args:
+      cluster-signing-cert-file: /etc/kubernetes/ssl/kube-ca.pem
+      cluster-signing-key-file: /etc/kubernetes/ssl/kube-ca-key.pem
   scheduler:
   kubelet:
     cluster_domain: cluster.local

We are running Rancher 2.4.5 with K8s 1.18.10 and encountered this issue today. This patch from @qrtt1 was the solution. Note that this patch has to go in the kube-controller section of the YAML -- we accidentally put it in the wrong section (there are multiple with extra_args options), and ended up losing a lot of time because it didn't work.