Open nwneisen opened 6 months ago
Just to make sure, can you try to invoke k0sctl v0.17.5 via env -u SSH_AUTH_SOCK k0sctl ...
and see if it makes a difference?
Probably has something to do with https://github.com/k0sproject/rig/pull/162
Just to make sure, can you try to invoke k0sctl v0.17.5 via env -u SSH_AUTH_SOCK k0sctl ... and see if it makes a difference?
This is also with things switched over to a lima VM that works with v0.17.4
nick@amd-t14:~/Downloads: env -u SSH_AUTH_SOCK ./k0sctl-linux-x64 apply -c ~/code/homelab/
charts/ .git/ homelab.yaml values.yaml
Chart.yaml .helmignore Makefile
nick@amd-t14:~/Downloads: env -u SSH_AUTH_SOCK ./k0sctl-linux-x64 apply -c ~/code/homelab/homelab.yaml --debug
DEBU[0000] Loaded configuration:
apiVersion: k0sctl.k0sproject.io/v1beta1
kind: Cluster
metadata:
name: homelab-cluster
spec:
hosts:
# - ssh:
# address: 192.168.1.60
# user: nick
# port: 22
# keyPath: /home/nick/.ssh/id_rsa
# role: single
- ssh:
address: 127.0.0.1
user: nick
port: 60022
keyPath: /home/nick/.lima/_config/user
role: single
k0s:
version: v1.28.7+k0s.0
versionChannel: stable
dynamicConfig: false
config: {}
⠀⣿⣿⡇⠀⠀⢀⣴⣾⣿⠟⠁⢸⣿⣿⣿⣿⣿⣿⣿⡿⠛⠁⠀⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀█████████ █████████ ███
⠀⣿⣿⡇⣠⣶⣿⡿⠋⠀⠀⠀⢸⣿⡇⠀⠀⠀⣠⠀⠀⢀⣠⡆⢸⣿⣿⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀███ ███ ███
⠀⣿⣿⣿⣿⣟⠋⠀⠀⠀⠀⠀⢸⣿⡇⠀⢰⣾⣿⠀⠀⣿⣿⡇⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀███ ███ ███
⠀⣿⣿⡏⠻⣿⣷⣤⡀⠀⠀⠀⠸⠛⠁⠀⠸⠋⠁⠀⠀⣿⣿⡇⠈⠉⠉⠉⠉⠉⠉⠉⠉⢹⣿⣿⠀███ ███ ███
⠀⣿⣿⡇⠀⠀⠙⢿⣿⣦⣀⠀⠀⠀⣠⣶⣶⣶⣶⣶⣶⣿⣿⡇⢰⣶⣶⣶⣶⣶⣶⣶⣶⣾⣿⣿⠀█████████ ███ ██████████
k0sctl v0.17.4 Copyright 2023, k0sctl authors.
Anonymized telemetry of usage will be sent to the authors.
By continuing to use k0sctl you agree to these terms:
https://k0sproject.io/licenses/eula
DEBU[0000] Preparing phase 'Set k0s version'
DEBU[0000] Preparing phase 'Connect to hosts'
INFO[0000] ==> Running phase: Connect to hosts
DEBU[0000] [ssh] 127.0.0.1:60022: HashKnownHosts is set to "yes", will hash newly added keys
DEBU[0000] [ssh] 127.0.0.1:60022: using an unencrypted private key from /home/nick/.lima/_config/user
DEBU[0000] [SSH] 127.0.0.1: failed to connect: ssh dial: dial tcp 127.0.0.1:60022: connect: connection refused
DEBU[0005] retrying, attempt 1 - last error: not connected: client connect: ssh dial: dial tcp 127.0.0.1:60022: connect: connection refused
DEBU[0005] [ssh] 127.0.0.1:60022: HashKnownHosts is set to "yes", will hash newly added keys
DEBU[0005] [SSH] 127.0.0.1: failed to connect: ssh dial: dial tcp 127.0.0.1:60022: connect: connection refused
DEBU[0010] retrying, attempt 2 - last error: not connected: client connect: ssh dial: dial tcp 127.0.0.1:60022: connect: connection refused
DEBU[0010] [ssh] 127.0.0.1:60022: HashKnownHosts is set to "yes", will hash newly added keys
DEBU[0010] [SSH] 127.0.0.1: failed to connect: ssh dial: dial tcp 127.0.0.1:60022: connect: connection refused
DEBU[0015] retrying, attempt 3 - last error: not connected: client connect: ssh dial: dial tcp 127.0.0.1:60022: connect: connection refused
DEBU[0015] [ssh] 127.0.0.1:60022: HashKnownHosts is set to "yes", will hash newly added keys
DEBU[0015] [SSH] 127.0.0.1: failed to connect: ssh dial: dial tcp 127.0.0.1:60022: connect: connection refused
Your last log snippets show network errors ("connection refused"), not SSH handshake failures ("ssh: handshake failed: ssh: unable to authenticate"), btw.
What is the disposition of your keys? I see from the logs there are 2 keys loaded in the ssh agent and one key got loaded from the default configuration:
DEBU[0000] [ssh] 10.252.198.229:22: using all keys (2) from ssh agent because a keypath was not explicitly given
DEBU[0000] [ssh] 10.252.198.229:22: using an unencrypted private key from id_rsa
Is this correct? If so, which of these should have access to the machine? I want to try to reproduce it on a test.
Just to make sure, can you try to invoke k0sctl v0.17.5 via env -u SSH_AUTH_SOCK k0sctl ... and see if it makes a difference?
Running with env -u SSH_AUTH_SOCK
is successful
nick@amd-t14:~/code/homelab (master): env -u SSH_AUTH_SOCK ~/Downloads/k0sctl-linux-x64 apply -c ~/code/homelab/homelab.yaml -d
DEBU[0000] Loaded configuration:
apiVersion: k0sctl.k0sproject.io/v1beta1
kind: Cluster
metadata:
name: homelab-cluster
spec:
hosts:
# - ssh:
# address: 192.168.1.2
# user: root
# port: 22
# keyPath: ~/.ssh/id_ed25519
# role: single
- ssh:
address: 10.252.198.12
user: ubuntu
port: 22
keyPath: ~/id_rsa
role: single
# - ssh:
# address: 127.0.0.1
# user: nick
# port: 60022
# keyPath: /home/nick/.ssh/id_ed25519
# role: single
k0s:
version: v1.28.7+k0s.0
versionChannel: stable
dynamicConfig: false
config: {}
⠀⣿⣿⡇⠀⠀⢀⣴⣾⣿⠟⠁⢸⣿⣿⣿⣿⣿⣿⣿⡿⠛⠁⠀⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀█████████ █████████ ███
⠀⣿⣿⡇⣠⣶⣿⡿⠋⠀⠀⠀⢸⣿⡇⠀⠀⠀⣠⠀⠀⢀⣠⡆⢸⣿⣿⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀███ ███ ███
⠀⣿⣿⣿⣿⣟⠋⠀⠀⠀⠀⠀⢸⣿⡇⠀⢰⣾⣿⠀⠀⣿⣿⡇⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀███ ███ ███
⠀⣿⣿⡏⠻⣿⣷⣤⡀⠀⠀⠀⠸⠛⠁⠀⠸⠋⠁⠀⠀⣿⣿⡇⠈⠉⠉⠉⠉⠉⠉⠉⠉⢹⣿⣿⠀███ ███ ███
⠀⣿⣿⡇⠀⠀⠙⢿⣿⣦⣀⠀⠀⠀⣠⣶⣶⣶⣶⣶⣶⣿⣿⡇⢰⣶⣶⣶⣶⣶⣶⣶⣶⣾⣿⣿⠀█████████ ███ ██████████
k0sctl v0.17.5 Copyright 2023, k0sctl authors.
Anonymized telemetry of usage will be sent to the authors.
By continuing to use k0sctl you agree to these terms:
https://k0sproject.io/licenses/eula
DEBU[0000] Preparing phase 'Set k0s version'
DEBU[0000] Preparing phase 'Connect to hosts'
INFO[0000] ==> Running phase: Connect to hosts
DEBU[0000] [ssh] 10.252.198.12:22: HashKnownHosts is set to "yes", will hash newly added keys
DEBU[0000] [ssh] 10.252.198.12:22: using an unencrypted private key from /home/nick/id_rsa
DEBU[0000] [ssh] 10.252.198.12:22: executing `uname | grep -q Linux`
DEBU[0000] [ssh] 10.252.198.12:22: executing `cat /etc/os-release || cat /usr/lib/os-release`
DEBU[0000] [ssh] 10.252.198.12:22: PRETTY_NAME="Ubuntu 22.04.4 LTS"
DEBU[0000] [ssh] 10.252.198.12:22: NAME="Ubuntu"
DEBU[0000] [ssh] 10.252.198.12:22: VERSION_ID="22.04"
DEBU[0000] [ssh] 10.252.198.12:22: VERSION="22.04.4 LTS (Jammy Jellyfish)"
DEBU[0000] [ssh] 10.252.198.12:22: VERSION_CODENAME=jammy
DEBU[0000] [ssh] 10.252.198.12:22: ID=ubuntu
DEBU[0000] [ssh] 10.252.198.12:22: ID_LIKE=debian
DEBU[0000] [ssh] 10.252.198.12:22: HOME_URL="https://www.ubuntu.com/"
DEBU[0000] [ssh] 10.252.198.12:22: SUPPORT_URL="https://help.ubuntu.com/"
DEBU[0000] [ssh] 10.252.198.12:22: BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
DEBU[0000] [ssh] 10.252.198.12:22: PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
DEBU[0000] [ssh] 10.252.198.12:22: UBUNTU_CODENAME=jammy
DEBU[0000] [ssh] 10.252.198.12:22: executing `[ "$(id -u)" = 0 ]`
DEBU[0000] [ssh] 10.252.198.12:22: executing `sudo -n true`
INFO[0000] [ssh] 10.252.198.12:22: connected
DEBU[0000] Preparing phase 'Detect host operating systems'
INFO[0000] ==> Running phase: Detect host operating systems
INFO[0000] [ssh] 10.252.198.12:22: is running Ubuntu 22.04.4 LTS
DEBU[0000] Preparing phase 'Acquire exclusive host lock'
INFO[0000] ==> Running phase: Acquire exclusive host lock
DEBU[0000] [ssh] 10.252.198.12:22: executing `sudo -s -- test -d /run/lock`
DEBU[0000] [ssh] 10.252.198.12:22: executing `mktemp`
DEBU[0000] [ssh] 10.252.198.12:22: /tmp/tmp.xpThjmAuvI
DEBU[0000] [ssh] 10.252.198.12:22: executing `sudo -s -- cat > "/tmp/tmp.xpThjmAuvI"`
DEBU[0000] [ssh] 10.252.198.12:22: writing 70 bytes to command stdin: 331f512cab4ba61b7d1883acdcf9606bc8a833b996544906657a073a67b19257-43696
DEBU[0000] [ssh] 10.252.198.12:22: executing `sudo -s -- mv -n "/tmp/tmp.xpThjmAuvI" "/run/lock/k0sctl"`
DEBU[0000] [ssh] 10.252.198.12:22: executing `test -f "/tmp/tmp.xpThjmAuvI"`
DEBU[0000] [ssh] 10.252.198.12:22: executing `sudo -s -- rm -f "/tmp/tmp.xpThjmAuvI"`
DEBU[0000] [ssh] 10.252.198.12:22: executing `sudo -s -- test -d /run/lock`
DEBU[0000] [ssh] 10.252.198.12:22: started periodic update of lock file /run/lock/k0sctl timestamp
DEBU[0000] Preparing phase 'Prepare hosts'
INFO[0000] ==> Running phase: Prepare hosts
DEBU[0000] [ssh] 10.252.198.12:22: executing `sudo -s -- command -v -- "curl" 2> /dev/null`
DEBU[0000] [ssh] 10.252.198.12:22: /usr/bin/curl
DEBU[0001] [ssh] 10.252.198.12:22: executing `sudo -s -- command -v -- "hostname" 2> /dev/null`
DEBU[0001] [ssh] 10.252.198.12:22: /usr/bin/hostname
DEBU[0001] [ssh] 10.252.198.12:22: executing `grep 'container=docker' /proc/1/environ 2> /dev/null`
DEBU[0001] Preparing phase 'Gather host facts'
INFO[0001] ==> Running phase: Gather host facts
DEBU[0001] [ssh] 10.252.198.12:22: executing `uname -m`
DEBU[0001] [ssh] 10.252.198.12:22: x86_64
DEBU[0001] [ssh] 10.252.198.12:22: executing `cat /etc/machine-id || cat /var/lib/dbus/machine-id`
DEBU[0001] [ssh] 10.252.198.12:22: cc23a39e4ffb40329de3dcb699bd92bd
DEBU[0001] [ssh] 10.252.198.12:22: executing `hostname 2> /dev/null`
DEBU[0001] [ssh] 10.252.198.12:22: homelab
INFO[0001] [ssh] 10.252.198.12:22: using homelab as hostname
DEBU[0001] [ssh] 10.252.198.12:22: executing `PATH=/usr/local/sbin:/usr/sbin:/sbin:$PATH; (ip route list scope global | grep -E "\b(172|10|192\.168)\.") || (ip route list | grep -m1 default)`
DEBU[0001] [ssh] 10.252.198.12:22: default via 10.252.198.1 dev ens3 proto dhcp src 10.252.198.12 metric 100
INFO[0001] [ssh] 10.252.198.12:22: discovered ens3 as private interface
DEBU[0001] [ssh] 10.252.198.12:22: executing `PATH=/usr/local/sbin:/usr/sbin:/sbin:$PATH ip -o addr show dev ens3 scope global`
DEBU[0001] [ssh] 10.252.198.12:22: 2: ens3 inet 10.252.198.12/24 metric 100 brd 10.252.198.255 scope global ens3\ valid_lft forever preferred_lft forever
DEBU[0001] Preparing phase 'Validate hosts'
INFO[0001] ==> Running phase: Validate hosts
DEBU[0001] [ssh] 10.252.198.12:22: executing `sudo -s -- true`
DEBU[0001] Preparing phase 'Gather k0s facts'
DEBU[0001] [ssh] 10.252.198.12:22: executing `sudo -s -- command -v -- "k0s" 2> /dev/null`
DEBU[0001] [ssh] 10.252.198.12:22: /usr/local/bin/k0s
INFO[0001] ==> Running phase: Gather k0s facts
DEBU[0001] [ssh] 10.252.198.12:22: executing `sudo -s -- /usr/local/bin/k0s version`
DEBU[0001] [ssh] 10.252.198.12:22: v1.28.7+k0s.0
DEBU[0001] [ssh] 10.252.198.12:22: has k0s binary version v1.28.7+k0s.0
DEBU[0001] [ssh] 10.252.198.12:22: executing `sudo -s -- test -e /etc/k0s/k0s.yaml 2> /dev/null`
DEBU[0001] [ssh] 10.252.198.12:22: executing `sudo -s -- cat -- /etc/k0s/k0s.yaml 2> /dev/null`
INFO[0001] [ssh] 10.252.198.12:22: found existing configuration
DEBU[0001] [ssh] 10.252.198.12:22: executing `sudo -s -- stat /run/systemd/system`
DEBU[0001] [ssh] 10.252.198.12:22: File: /run/systemd/system
DEBU[0001] [ssh] 10.252.198.12:22: Size: 80 Blocks: 0 IO Block: 4096 directory
DEBU[0001] [ssh] 10.252.198.12:22: Device: 19h/25d Inode: 284 Links: 3
DEBU[0001] [ssh] 10.252.198.12:22: Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
DEBU[0001] [ssh] 10.252.198.12:22: Access: 2024-04-14 17:48:33.102862615 -0600
DEBU[0001] [ssh] 10.252.198.12:22: Modify: 2024-04-14 17:48:33.078862666 -0600
DEBU[0001] [ssh] 10.252.198.12:22: Change: 2024-04-14 17:48:33.078862666 -0600
DEBU[0001] [ssh] 10.252.198.12:22: Birth: -
DEBU[0001] [ssh] 10.252.198.12:22: executing `sudo -s -- systemctl show -p FragmentPath k0scontroller.service 2> /dev/null | cut -d"=" -f2`
DEBU[0001] [ssh] 10.252.198.12:22: /etc/systemd/system/k0scontroller.service
DEBU[0001] [ssh] 10.252.198.12:22: executing `sudo -s -- /usr/local/bin/k0s status -o json`
DEBU[0001] [ssh] 10.252.198.12:22: {
DEBU[0001] [ssh] 10.252.198.12:22: "Version": "v1.28.7+k0s.0",
DEBU[0001] [ssh] 10.252.198.12:22: "Pid": 3836,
DEBU[0001] [ssh] 10.252.198.12:22: "PPid": 0,
DEBU[0001] [ssh] 10.252.198.12:22: "Role": "controller",
DEBU[0001] [ssh] 10.252.198.12:22: "SysInit": "",
DEBU[0001] [ssh] 10.252.198.12:22: "StubFile": "",
DEBU[0001] [ssh] 10.252.198.12:22: "Output": "",
DEBU[0001] [ssh] 10.252.198.12:22: "Workloads": true,
DEBU[0001] [ssh] 10.252.198.12:22: "SingleNode": true,
DEBU[0001] [ssh] 10.252.198.12:22: "Args": [
DEBU[0001] [ssh] 10.252.198.12:22: "/usr/local/bin/k0s",
DEBU[0001] [ssh] 10.252.198.12:22: "controller",
DEBU[0001] [ssh] 10.252.198.12:22: "--config=/etc/k0s/k0s.yaml",
DEBU[0001] [ssh] 10.252.198.12:22: "--data-dir=/var/lib/k0s",
DEBU[0001] [ssh] 10.252.198.12:22: "--single=true"
DEBU[0001] [ssh] 10.252.198.12:22: ],
DEBU[0001] [ssh] 10.252.198.12:22: "WorkerToAPIConnectionStatus": {
DEBU[0001] [ssh] 10.252.198.12:22: "Message": "",
DEBU[0001] [ssh] 10.252.198.12:22: "Success": true
DEBU[0001] [ssh] 10.252.198.12:22: },
DEBU[0001] [ssh] 10.252.198.12:22: "ClusterConfig": {
DEBU[0001] [ssh] 10.252.198.12:22: "metadata": {
DEBU[0001] [ssh] 10.252.198.12:22: "name": "k0s",
DEBU[0001] [ssh] 10.252.198.12:22: "creationTimestamp": null
DEBU[0001] [ssh] 10.252.198.12:22: },
DEBU[0001] [ssh] 10.252.198.12:22: "kind": "ClusterConfig",
DEBU[0001] [ssh] 10.252.198.12:22: "apiVersion": "k0s.k0sproject.io/v1beta1",
DEBU[0001] [ssh] 10.252.198.12:22: "spec": {
DEBU[0001] [ssh] 10.252.198.12:22: "api": {
DEBU[0001] [ssh] 10.252.198.12:22: "address": "10.252.198.12",
DEBU[0001] [ssh] 10.252.198.12:22: "k0sApiPort": 9443,
DEBU[0001] [ssh] 10.252.198.12:22: "port": 6443,
DEBU[0001] [ssh] 10.252.198.12:22: "sans": [
DEBU[0001] [ssh] 10.252.198.12:22: "10.252.198.12",
DEBU[0001] [ssh] 10.252.198.12:22: "fe80::5054:ff:fe83:7af3",
DEBU[0001] [ssh] 10.252.198.12:22: "127.0.0.1"
DEBU[0001] [ssh] 10.252.198.12:22: ]
DEBU[0001] [ssh] 10.252.198.12:22: },
DEBU[0001] [ssh] 10.252.198.12:22: "controllerManager": {},
DEBU[0001] [ssh] 10.252.198.12:22: "scheduler": {},
DEBU[0001] [ssh] 10.252.198.12:22: "storage": {
DEBU[0001] [ssh] 10.252.198.12:22: "etcd": {
DEBU[0001] [ssh] 10.252.198.12:22: "externalCluster": null,
DEBU[0001] [ssh] 10.252.198.12:22: "peerAddress": "10.252.198.12"
DEBU[0001] [ssh] 10.252.198.12:22: },
DEBU[0001] [ssh] 10.252.198.12:22: "kine": {
DEBU[0001] [ssh] 10.252.198.12:22: "dataSource": "sqlite:/var/lib/k0s/db/state.db?mode=rwc\u0026_journal=WAL\u0026cache=shared"
DEBU[0001] [ssh] 10.252.198.12:22: },
DEBU[0001] [ssh] 10.252.198.12:22: "type": "etcd"
DEBU[0001] [ssh] 10.252.198.12:22: },
DEBU[0001] [ssh] 10.252.198.12:22: "network": {
DEBU[0001] [ssh] 10.252.198.12:22: "calico": null,
DEBU[0001] [ssh] 10.252.198.12:22: "dualStack": {},
DEBU[0001] [ssh] 10.252.198.12:22: "kubeProxy": {
DEBU[0001] [ssh] 10.252.198.12:22: "mode": "iptables",
DEBU[0001] [ssh] 10.252.198.12:22: "metricsBindAddress": "0.0.0.0:10249",
DEBU[0001] [ssh] 10.252.198.12:22: "iptables": {
DEBU[0001] [ssh] 10.252.198.12:22: "syncPeriod": "0s",
DEBU[0001] [ssh] 10.252.198.12:22: "minSyncPeriod": "0s"
DEBU[0001] [ssh] 10.252.198.12:22: },
DEBU[0001] [ssh] 10.252.198.12:22: "ipvs": {
DEBU[0001] [ssh] 10.252.198.12:22: "syncPeriod": "0s",
DEBU[0001] [ssh] 10.252.198.12:22: "minSyncPeriod": "0s",
DEBU[0001] [ssh] 10.252.198.12:22: "tcpTimeout": "0s",
DEBU[0001] [ssh] 10.252.198.12:22: "tcpFinTimeout": "0s",
DEBU[0001] [ssh] 10.252.198.12:22: "udpTimeout": "0s"
DEBU[0001] [ssh] 10.252.198.12:22: }
DEBU[0001] [ssh] 10.252.198.12:22: },
DEBU[0001] [ssh] 10.252.198.12:22: "kuberouter": {
DEBU[0001] [ssh] 10.252.198.12:22: "autoMTU": true,
DEBU[0001] [ssh] 10.252.198.12:22: "mtu": 0,
DEBU[0001] [ssh] 10.252.198.12:22: "metricsPort": 8080,
DEBU[0001] [ssh] 10.252.198.12:22: "hairpin": "Enabled",
DEBU[0001] [ssh] 10.252.198.12:22: "ipMasq": false,
DEBU[0001] [ssh] 10.252.198.12:22: "peerRouterASNs": "",
DEBU[0001] [ssh] 10.252.198.12:22: "peerRouterIPs": ""
DEBU[0001] [ssh] 10.252.198.12:22: },
DEBU[0001] [ssh] 10.252.198.12:22: "nodeLocalLoadBalancing": {
DEBU[0001] [ssh] 10.252.198.12:22: "type": "EnvoyProxy",
DEBU[0001] [ssh] 10.252.198.12:22: "envoyProxy": {
DEBU[0001] [ssh] 10.252.198.12:22: "image": {
DEBU[0001] [ssh] 10.252.198.12:22: "image": "quay.io/k0sproject/envoy-distroless",
DEBU[0001] [ssh] 10.252.198.12:22: "version": "v1.24.12"
DEBU[0001] [ssh] 10.252.198.12:22: },
DEBU[0001] [ssh] 10.252.198.12:22: "apiServerBindPort": 7443,
DEBU[0001] [ssh] 10.252.198.12:22: "konnectivityServerBindPort": 7132
DEBU[0001] [ssh] 10.252.198.12:22: }
DEBU[0001] [ssh] 10.252.198.12:22: },
DEBU[0001] [ssh] 10.252.198.12:22: "podCIDR": "10.244.0.0/16",
DEBU[0001] [ssh] 10.252.198.12:22: "provider": "kuberouter",
DEBU[0001] [ssh] 10.252.198.12:22: "serviceCIDR": "10.96.0.0/12",
DEBU[0001] [ssh] 10.252.198.12:22: "clusterDomain": "cluster.local"
DEBU[0001] [ssh] 10.252.198.12:22: },
DEBU[0001] [ssh] 10.252.198.12:22: "telemetry": {
DEBU[0001] [ssh] 10.252.198.12:22: "enabled": true
DEBU[0001] [ssh] 10.252.198.12:22: },
DEBU[0001] [ssh] 10.252.198.12:22: "installConfig": {
DEBU[0001] [ssh] 10.252.198.12:22: "users": {
DEBU[0001] [ssh] 10.252.198.12:22: "etcdUser": "etcd",
DEBU[0001] [ssh] 10.252.198.12:22: "kineUser": "kube-apiserver",
DEBU[0001] [ssh] 10.252.198.12:22: "konnectivityUser": "konnectivity-server",
DEBU[0001] [ssh] 10.252.198.12:22: "kubeAPIserverUser": "kube-apiserver",
DEBU[0001] [ssh] 10.252.198.12:22: "kubeSchedulerUser": "kube-scheduler"
DEBU[0001] [ssh] 10.252.198.12:22: }
DEBU[0001] [ssh] 10.252.198.12:22: },
DEBU[0001] [ssh] 10.252.198.12:22: "images": {
DEBU[0001] [ssh] 10.252.198.12:22: "konnectivity": {
DEBU[0002] [ssh] 10.252.198.12:22: "image": "quay.io/k0sproject/apiserver-network-proxy-agent",
DEBU[0002] [ssh] 10.252.198.12:22: "version": "v0.1.4"
DEBU[0002] [ssh] 10.252.198.12:22: },
DEBU[0002] [ssh] 10.252.198.12:22: "pushgateway": {
DEBU[0002] [ssh] 10.252.198.12:22: "image": "quay.io/k0sproject/pushgateway-ttl",
DEBU[0002] [ssh] 10.252.198.12:22: "version": "1.4.0-k0s.0"
DEBU[0002] [ssh] 10.252.198.12:22: },
DEBU[0002] [ssh] 10.252.198.12:22: "metricsserver": {
DEBU[0002] [ssh] 10.252.198.12:22: "image": "registry.k8s.io/metrics-server/metrics-server",
DEBU[0002] [ssh] 10.252.198.12:22: "version": "v0.6.4"
DEBU[0002] [ssh] 10.252.198.12:22: },
DEBU[0002] [ssh] 10.252.198.12:22: "kubeproxy": {
DEBU[0002] [ssh] 10.252.198.12:22: "image": "quay.io/k0sproject/kube-proxy",
DEBU[0002] [ssh] 10.252.198.12:22: "version": "v1.28.7"
DEBU[0002] [ssh] 10.252.198.12:22: },
DEBU[0002] [ssh] 10.252.198.12:22: "coredns": {
DEBU[0002] [ssh] 10.252.198.12:22: "image": "quay.io/k0sproject/coredns",
DEBU[0002] [ssh] 10.252.198.12:22: "version": "1.11.1"
DEBU[0002] [ssh] 10.252.198.12:22: },
DEBU[0002] [ssh] 10.252.198.12:22: "pause": {
DEBU[0002] [ssh] 10.252.198.12:22: "image": "registry.k8s.io/pause",
DEBU[0002] [ssh] 10.252.198.12:22: "version": "3.8"
DEBU[0002] [ssh] 10.252.198.12:22: },
DEBU[0002] [ssh] 10.252.198.12:22: "calico": {
DEBU[0002] [ssh] 10.252.198.12:22: "cni": {
DEBU[0002] [ssh] 10.252.198.12:22: "image": "quay.io/k0sproject/calico-cni",
DEBU[0002] [ssh] 10.252.198.12:22: "version": "v3.26.1-1"
DEBU[0002] [ssh] 10.252.198.12:22: },
DEBU[0002] [ssh] 10.252.198.12:22: "node": {
DEBU[0002] [ssh] 10.252.198.12:22: "image": "quay.io/k0sproject/calico-node",
DEBU[0002] [ssh] 10.252.198.12:22: "version": "v3.26.1-1"
DEBU[0002] [ssh] 10.252.198.12:22: },
DEBU[0002] [ssh] 10.252.198.12:22: "kubecontrollers": {
DEBU[0002] [ssh] 10.252.198.12:22: "image": "quay.io/k0sproject/calico-kube-controllers",
DEBU[0002] [ssh] 10.252.198.12:22: "version": "v3.26.1-1"
DEBU[0002] [ssh] 10.252.198.12:22: }
DEBU[0002] [ssh] 10.252.198.12:22: },
DEBU[0002] [ssh] 10.252.198.12:22: "kuberouter": {
DEBU[0002] [ssh] 10.252.198.12:22: "cni": {
DEBU[0002] [ssh] 10.252.198.12:22: "image": "quay.io/k0sproject/kube-router",
DEBU[0002] [ssh] 10.252.198.12:22: "version": "v1.6.0-iptables1.8.9-1"
DEBU[0002] [ssh] 10.252.198.12:22: },
DEBU[0002] [ssh] 10.252.198.12:22: "cniInstaller": {
DEBU[0002] [ssh] 10.252.198.12:22: "image": "quay.io/k0sproject/cni-node",
DEBU[0002] [ssh] 10.252.198.12:22: "version": "1.1.1-k0s.1"
DEBU[0002] [ssh] 10.252.198.12:22: }
DEBU[0002] [ssh] 10.252.198.12:22: },
DEBU[0002] [ssh] 10.252.198.12:22: "default_pull_policy": "IfNotPresent"
DEBU[0002] [ssh] 10.252.198.12:22: },
DEBU[0002] [ssh] 10.252.198.12:22: "extensions": {
DEBU[0002] [ssh] 10.252.198.12:22: "storage": {
DEBU[0002] [ssh] 10.252.198.12:22: "type": "external_storage",
DEBU[0002] [ssh] 10.252.198.12:22: "create_default_storage_class": false
DEBU[0002] [ssh] 10.252.198.12:22: },
DEBU[0002] [ssh] 10.252.198.12:22: "helm": {
DEBU[0002] [ssh] 10.252.198.12:22: "concurrencyLevel": 5,
DEBU[0002] [ssh] 10.252.198.12:22: "repositories": null,
DEBU[0002] [ssh] 10.252.198.12:22: "charts": null
DEBU[0002] [ssh] 10.252.198.12:22: }
DEBU[0002] [ssh] 10.252.198.12:22: },
DEBU[0002] [ssh] 10.252.198.12:22: "konnectivity": {
DEBU[0002] [ssh] 10.252.198.12:22: "adminPort": 8133,
DEBU[0002] [ssh] 10.252.198.12:22: "agentPort": 8132
DEBU[0002] [ssh] 10.252.198.12:22: }
DEBU[0002] [ssh] 10.252.198.12:22: }
DEBU[0002] [ssh] 10.252.198.12:22: },
DEBU[0002] [ssh] 10.252.198.12:22: "K0sVars": {
DEBU[0002] [ssh] 10.252.198.12:22: "AdminKubeConfigPath": "/var/lib/k0s/pki/admin.conf",
DEBU[0002] [ssh] 10.252.198.12:22: "BinDir": "/var/lib/k0s/bin",
DEBU[0002] [ssh] 10.252.198.12:22: "CertRootDir": "/var/lib/k0s/pki",
DEBU[0002] [ssh] 10.252.198.12:22: "DataDir": "/var/lib/k0s",
DEBU[0002] [ssh] 10.252.198.12:22: "EtcdCertDir": "/var/lib/k0s/pki/etcd",
DEBU[0002] [ssh] 10.252.198.12:22: "EtcdDataDir": "/var/lib/k0s/etcd",
DEBU[0002] [ssh] 10.252.198.12:22: "KineSocketPath": "/run/k0s/kine/kine.sock:2379",
DEBU[0002] [ssh] 10.252.198.12:22: "KonnectivitySocketDir": "/run/k0s/konnectivity-server",
DEBU[0002] [ssh] 10.252.198.12:22: "KubeletAuthConfigPath": "/var/lib/k0s/kubelet.conf",
DEBU[0002] [ssh] 10.252.198.12:22: "KubeletVolumePluginDir": "/usr/libexec/k0s/kubelet-plugins/volume/exec",
DEBU[0002] [ssh] 10.252.198.12:22: "ManifestsDir": "/var/lib/k0s/manifests",
DEBU[0002] [ssh] 10.252.198.12:22: "RunDir": "/run/k0s",
DEBU[0002] [ssh] 10.252.198.12:22: "KonnectivityKubeConfigPath": "/var/lib/k0s/pki/konnectivity.conf",
DEBU[0002] [ssh] 10.252.198.12:22: "OCIBundleDir": "/var/lib/k0s/images",
DEBU[0002] [ssh] 10.252.198.12:22: "DefaultStorageType": "kine",
DEBU[0002] [ssh] 10.252.198.12:22: "RuntimeConfigPath": "/run/k0s/k0s.yaml",
DEBU[0002] [ssh] 10.252.198.12:22: "StatusSocketPath": "/run/k0s/status.sock",
DEBU[0002] [ssh] 10.252.198.12:22: "StartupConfigPath": "/etc/k0s/k0s.yaml",
DEBU[0002] [ssh] 10.252.198.12:22: "EnableDynamicConfig": false,
DEBU[0002] [ssh] 10.252.198.12:22: "HelmHome": "/var/lib/k0s/helmhome",
DEBU[0002] [ssh] 10.252.198.12:22: "HelmRepositoryCache": "/var/lib/k0s/helmhome/cache",
DEBU[0002] [ssh] 10.252.198.12:22: "HelmRepositoryConfig": "/var/lib/k0s/helmhome/repositories.yaml"
DEBU[0002] [ssh] 10.252.198.12:22: }
DEBU[0002] [ssh] 10.252.198.12:22: }
INFO[0002] [ssh] 10.252.198.12:22: is running k0s single version v1.28.7+k0s.0
DEBU[0002] [ssh] 10.252.198.12:22: executing `sudo -s -- test -e /var/lib/k0s/pki/admin.conf 2> /dev/null`
DEBU[0002] [ssh] 10.252.198.12:22: executing `sudo -s -- env "KUBECONFIG=/var/lib/k0s/pki/admin.conf" /usr/local/bin/k0s kubectl get -n kube-system namespace kube-system -o template={{.metadata.uid}}`
DEBU[0002] [ssh] 10.252.198.12:22: f6d0f3ba-9a40-4731-aeea-179b84dc8e4b
DEBU[0002] Preparing phase 'Validate facts'
INFO[0002] ==> Running phase: Validate facts
DEBU[0002] Preparing phase 'Download k0s binaries to local host'
DEBU[0002] Preparing phase 'Upload k0s binaries to hosts'
DEBU[0002] Preparing phase 'Download k0s on hosts'
DEBU[0002] [ssh] 10.252.198.12:22: k0s version on target host is already v1.28.7+k0s.0
DEBU[0002] Preparing phase 'Upload files to hosts'
DEBU[0002] Preparing phase 'Install k0s binaries on hosts'
DEBU[0002] Preparing phase 'Prepare ARM nodes'
DEBU[0002] Preparing phase 'Configure k0s'
DEBU[0002] using existing k0s config
DEBU[0002] [ssh] 10.252.198.12:22: executing `mktemp`
DEBU[0002] [ssh] 10.252.198.12:22: /tmp/tmp.WAjV6pMRJA
DEBU[0002] [ssh] 10.252.198.12:22: executing `mktemp 2> /dev/null`
DEBU[0002] [ssh] 10.252.198.12:22: /tmp/tmp.TTSimMYnR4
DEBU[0002] [ssh] 10.252.198.12:22: executing `cat > /tmp/tmp.TTSimMYnR4`
DEBU[0002] [ssh] 10.252.198.12:22: writing 1654 bytes to command stdin
DEBU[0002] [ssh] 10.252.198.12:22: executing `sudo -s -- install -D -m 0600 -- /tmp/tmp.TTSimMYnR4 /tmp/tmp.WAjV6pMRJA`
INFO[0002] [ssh] 10.252.198.12:22: validating configuration
DEBU[0002] [ssh] 10.252.198.12:22: comparing k0s version v1.28.7+k0s.0 with >= v1.23.1+k0s.0
DEBU[0002] [ssh] 10.252.198.12:22: comparison result true
DEBU[0002] [ssh] 10.252.198.12:22: executing `sudo -s -- /usr/local/bin/k0s config validate --config "/tmp/tmp.WAjV6pMRJA"`
DEBU[0002] [ssh] 10.252.198.12:22: configuration will not change
DEBU[0002] [ssh] 10.252.198.12:22: executing `sudo -s -- rm -f -- /tmp/tmp.WAjV6pMRJA 2> /dev/null`
DEBU[0002] Preparing phase 'Restore cluster state'
DEBU[0002] Preparing phase 'Run Before Apply Hooks'
DEBU[0002] Preparing phase 'Initialize the k0s cluster'
DEBU[0002] Preparing phase 'Install controllers'
DEBU[0002] Preparing phase 'Install workers'
DEBU[0002] Preparing phase 'Upgrade controllers'
DEBU[0002] UpgradeControllers phase prep starting
DEBU[0002] 1 controllers in total
DEBU[0002] UpgradeControllers phase prepared, 0 controllers needs upgrade
DEBU[0002] Preparing phase 'Upgrade workers'
DEBU[0002] 0 workers in total
DEBU[0002] UpgradeWorkers phase prepared, 0 workers needs upgrade
DEBU[0002] Preparing phase 'Reset workers'
DEBU[0002] 0 workers in total
DEBU[0002] ResetWorkers phase prepared, 0 workers will be reset
DEBU[0002] Preparing phase 'Reset controllers'
DEBU[0002] 1 controllers in total
DEBU[0002] ResetControllers phase prepared, 0 controllers will be reset
DEBU[0002] Preparing phase 'Run After Apply Hooks'
DEBU[0002] Preparing phase 'Release exclusive host lock'
INFO[0002] ==> Running phase: Release exclusive host lock
DEBU[0002] [ssh] 10.252.198.12:22: stopped lock cycle, removing file
DEBU[0002] [ssh] 10.252.198.12:22: executing `sudo -s -- rm -f -- /run/lock/k0sctl 2> /dev/null`
DEBU[0002] Preparing phase 'Disconnect from hosts'
INFO[0002] ==> Running phase: Disconnect from hosts
INFO[0003] ==> Finished in 2s
INFO[0003] k0s cluster version v1.28.7+k0s.0 is now installed
INFO[0003] Tip: To access the cluster you can now fetch the admin kubeconfig using:
INFO[0003] k0sctl kubeconfig
What is the disposition of your keys?
I copy the multipass key from /var/snap/multipass/common/data/multipassd/ssh-keys/id_rsa
to my home directory and change the ownership to be my logged in user. This is the same setup I have used for previous versions of k0sctl
. I am explicitly setting the path in my config file.
Summary
v0.17.5 can no longer connect to a single host node. v0.17.4 works fine.
Details
I just installed v0.17.5 and created a new config using
k0sctl init
and modified the hosts section to be a single host that pointed to a VM on my local machine. The apply command kept hanging up when I tried to apply the config.I reverted my k0sctl version to v0.17.4 and it worked fine.
Environment
Host: Pop OS VM manager: multipass VM distro: Ubuntu 22.04