search

rancher / k3os

Purpose-built OS for Kubernetes, fully managed by Kubernetes.
https://k3os.io
Apache License 2.0
3.5k stars 396 forks source link

`kube-apiserver-arg` does not set up correctly API Server in case of flags that can be set multiple times. #777

Closed mazzy89 closed 2 years ago

mazzy89 commented 2 years ago

Version (k3OS / kernel)

master01 [~]$ k3os --version
k3os version v0.20.11-k3s1r1

VERSION="k3OS v0.20.11-k3s1r1"

Architecture

aarch64

Describe the bug

I've defined multiple times the kube-apiserver-flag with the flag service-account-key-file. A showed here 

  k3s_args:
    - server
    - "--cluster-init"
    - "--flannel-backend=none"
    - "--disable=traefik,servicelb"
    - "--cluster-cidr=10.107.0.0/23"
    - "--service-cidr=10.107.1.0/23"
    - "--kube-apiserver-arg=service-account-key-file=/var/lib/rancher/k3s/server/tls/sa-signer-pkcs8.pub"
    - "--kube-apiserver-arg=service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key"
    - "--kube-apiserver-arg=service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/sa-signer.key"
    - "--kube-apiserver-arg=api-audiences=sts.amazonaws.com"
    - "--kube-apiserver-arg=service-account-issuer=https://s3-eu-central-1.amazonaws.com/my-amazing-bucket"

However how you can see the last occurrence of service-account-key-file overwrite the first. This is how it is bootstraped kube-apiserver

time="2021-10-06T16:55:54.418895840Z" level=info msg="Running kube-apiserver --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=sts.amazonaws.com --authorization-mode=Node,RBAC --bind-address=127.0.0.1 --cert-dir=/var/lib/rancher/k3s/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --enable-admission-plugins=NodeRestriction --etcd-cafile=/var/lib/rancher/k3s/server/tls/etcd/server-ca.crt --etcd-certfile=/var/lib/rancher/k3s/server/tls/etcd/client.crt --etcd-keyfile=/var/lib/rancher/k3s/server/tls/etcd/client.key --etcd-servers=https://127.0.0.1:2379 --feature-gates=ServiceAccountIssuerDiscovery=false --insecure-port=0 --kubelet-certificate-authority=/var/lib/rancher/k3s/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.key --profiling=false --proxy-client-cert-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/k3s/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6444 --service-account-issuer=https://s3-eu-central-1.amazonaws.com/my-amazing-bucket --service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/sa-signer.key --service-cluster-ip-range=10.107.0.0/23 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt --tls-private-key-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key"

To Reproduce

Expected behavior

I would expect that service-account-key-file flag would be defined multiple times.

Actual behavior

Additional context

mazzy89 commented 2 years ago

the guy replied me from k3s. the issue seems not related to the project here. closing and continuing the discussions there.