search

rancherfederal / rke2-aws-tf

MIT License
84 stars 68 forks source link

tls-san entry in server config is ignored #93

Closed ghsbhatia closed 11 months ago

ghsbhatia commented 1 year ago

The rke2 server configuration allows for multiple entries to be specified for tls-san, however the script rke2-init.sh is creating an invalid yaml by creating a new entry for tls-san rather than appending to user provided list.

server config passed to TF module:

# Server Configuration
write-kubeconfig-mode: "0644"
node-label:
  - "name=server"
  - "os=ubuntu"
kube-controller-manager-arg:
  - "bind-address=0.0.0.0"
kube-scheduler-arg:
  - "bind-address=0.0.0.0"
node-taint:
  - "CriticalAddonsOnly=true:NoExecute"
tls-san:
  - k8s.foo-demo.bar.com

server config on server node:

ubuntu@ip-10-1-1-68:~$ sudo cat /etc/rancher/rke2/config.yaml
# Additional user defined configuration
# Server Configuration
write-kubeconfig-mode: "0644"
node-label:
  - "name=server"
  - "os=ubuntu"
kube-controller-manager-arg:
  - "bind-address=0.0.0.0"
kube-scheduler-arg:
  - "bind-address=0.0.0.0"
node-taint:
  - "CriticalAddonsOnly=true:NoExecute"
tls-san:
  - k8s.foo-demo.bar.com
token: FmbtIMwa9TNy5pUHAAx2rs6XlK1qiphqwemAUpsC
cloud-provider-name: "aws"
tls-san:
  - foo-rke2-atv-rke2-cp-8fdaf7078215333b.elb.us-east-1.amazonaws.com

This causes errors when invoking kubectl as follows:

$ kubectl get nodes
Unable to connect to the server: x509: certificate is valid for kubernetes, kubernetes.default, kubernetes.default.svc, kubernetes.default.svc.cluster.local, foo-rke2-atv-rke2-cp-8fdaf7078215333b.elb.us-east-1.amazonaws.com, localhost, ip-10-1-1-68.ec2.internal, not k8s.foo-demo.bar.com
adamacosta commented 1 year ago

It's this:

config() {
  mkdir -p "/etc/rancher/rke2"
  cat <<EOF >> "/etc/rancher/rke2/config.yaml"
# Additional user defined configuration
${config}
EOF
}

and this:

  if [ $TYPE = "server" ]; then
    # Initialize server
    identify

    cat <<EOF >> "/etc/rancher/rke2/config.yaml"
tls-san:
  - ${server_url}
EOF

The module allows the user to pass arbitrary config values, which may include tls-san, but then creates it anyway because the ${server_url} refers to the DNS name assigned by AWS to the control plane load balancer, which isn't known until it gets created. I suspect we'll have to expose a variable to set additional SANs for the API server directly and then explain in bold-face font somewhere that this particular key can't be passed as additional config without breaking the setup. The way this script currently works doesn't make it possible to assign your own DNS name to your API server, which isn't a great idea. We got pretty lucky that it took over three years for someone to encounter this.

ghsbhatia commented 11 months ago

@adamacosta @joshrwolf - Please take a look at the pull request. Thanks!

adamacosta commented 11 months ago

Fixed in v2.4.1