search

jnsgruk / concierge

An opinionated utility for provisioning charm development and testing machines.
Apache License 2.0
5 stars 2 forks source link

fix: ensure k8s bootstrap is idempotent #18

Closed jnsgruk closed 1 week ago

jnsgruk commented 1 week ago

Fixes #17

@gruyaume can you give this a test locally and ensure it solves things for you?

gruyaume commented 1 week ago

Fixes #17

@gruyaume can you give this a test locally and ensure it solves things for you?

Yes this solves the issue.

It's not quite related, but in the Canonical K8s I had deployed locally, I didn't have network enabled. Concierge will not enable network and error out when enabling load-balancer (which depends on network).

Logs

guillaume@courge:~/concierge$ sudo ./concierge prepare -p k8s --k8s-channel=1.30-classic/beta --lxd-channel "5.21/stable" --verbose
[sudo] password for guillaume: 
time=2024-11-19T11:49:12.131-05:00 level=INFO msg="Preset selected" preset=k8s
time=2024-11-19T11:49:12.517-05:00 level=DEBUG msg="Filesystem ownership changed" user=guillaume group=1000 path=/home/guillaume/.cache
time=2024-11-19T11:49:12.517-05:00 level=DEBUG msg="Filesystem ownership changed" user=guillaume group=1000 path=/home/guillaume/.cache/concierge/concierge.yaml
time=2024-11-19T11:49:12.517-05:00 level=DEBUG msg="Merged runtime configuration saved" path=.cache/concierge/concierge.yaml
time=2024-11-19T11:49:12.517-05:00 level=DEBUG msg="Running command" command="/usr/bin/apt-get update"
time=2024-11-19T11:49:12.517-05:00 level=DEBUG msg="Installing snap" snap=jq
time=2024-11-19T11:49:13.129-05:00 level=DEBUG msg="Queried snapd API" snap=jq installed=true classic=false
time=2024-11-19T11:49:13.129-05:00 level=DEBUG msg="Running command" command="/usr/bin/snap refresh jq --channel latest/stable"
time=2024-11-19T11:49:13.664-05:00 level=INFO msg="Refreshed snap" snap=jq
time=2024-11-19T11:49:13.664-05:00 level=DEBUG msg="Installing snap" snap=yq
time=2024-11-19T11:49:14.101-05:00 level=DEBUG msg="Queried snapd API" snap=yq installed=true classic=false
time=2024-11-19T11:49:14.101-05:00 level=DEBUG msg="Running command" command="/usr/bin/snap refresh yq --channel latest/stable"
time=2024-11-19T11:49:14.677-05:00 level=INFO msg="Refreshed snap" snap=yq
time=2024-11-19T11:49:14.677-05:00 level=DEBUG msg="Installing snap" snap=rockcraft
time=2024-11-19T11:49:14.830-05:00 level=DEBUG msg="Running command" command="/usr/bin/apt-get install -y python3-pip"
time=2024-11-19T11:49:15.025-05:00 level=DEBUG msg="Queried snapd API" snap=rockcraft installed=true classic=true
time=2024-11-19T11:49:15.025-05:00 level=DEBUG msg="Running command" command="/usr/bin/snap refresh rockcraft --channel latest/stable --classic"
time=2024-11-19T11:49:15.034-05:00 level=INFO msg="Installed apt package" package=python3-pip
time=2024-11-19T11:49:15.034-05:00 level=DEBUG msg="Running command" command="/usr/bin/apt-get install -y python3-venv"
time=2024-11-19T11:49:15.233-05:00 level=INFO msg="Installed apt package" package=python3-venv
time=2024-11-19T11:49:15.475-05:00 level=INFO msg="Refreshed snap" snap=rockcraft
time=2024-11-19T11:49:15.475-05:00 level=DEBUG msg="Installing snap" snap=charmcraft
time=2024-11-19T11:49:15.938-05:00 level=DEBUG msg="Queried snapd API" snap=charmcraft installed=true classic=true
time=2024-11-19T11:49:15.938-05:00 level=DEBUG msg="Running command" command="/usr/bin/snap refresh charmcraft --channel latest/stable --classic"
time=2024-11-19T11:49:16.423-05:00 level=INFO msg="Refreshed snap" snap=charmcraft
time=2024-11-19T11:49:16.423-05:00 level=DEBUG msg="Installing snap" snap=lxd
time=2024-11-19T11:49:16.423-05:00 level=DEBUG msg="Installing snap" snap=k8s
time=2024-11-19T11:49:16.938-05:00 level=DEBUG msg="Queried snapd API" snap=k8s installed=true classic=true
time=2024-11-19T11:49:16.938-05:00 level=DEBUG msg="Running command" command="/usr/bin/snap refresh k8s --channel 1.30-classic/beta --classic"
time=2024-11-19T11:49:17.082-05:00 level=DEBUG msg="Queried snapd API" snap=lxd installed=true classic=false
time=2024-11-19T11:49:17.376-05:00 level=INFO msg="Refreshed snap" snap=k8s
time=2024-11-19T11:49:17.376-05:00 level=DEBUG msg="Installing snap" snap=kubectl
time=2024-11-19T11:49:17.376-05:00 level=DEBUG msg="Running command" command="/usr/bin/snap refresh lxd --channel 5.21/stable"
time=2024-11-19T11:49:17.856-05:00 level=INFO msg="Refreshed snap" snap=lxd
time=2024-11-19T11:49:17.857-05:00 level=DEBUG msg="Running command" command="/snap/bin/lxd waitready"
time=2024-11-19T11:49:17.907-05:00 level=DEBUG msg="Running command" command="/snap/bin/lxd init --minimal"
time=2024-11-19T11:49:17.979-05:00 level=DEBUG msg="Queried snapd API" snap=kubectl installed=true classic=true
time=2024-11-19T11:49:17.979-05:00 level=DEBUG msg="Running command" command="/usr/bin/snap refresh kubectl --channel stable --classic"
time=2024-11-19T11:49:18.031-05:00 level=DEBUG msg="Running command" command="/snap/bin/lxc network set lxdbr0 ipv6.address none"
time=2024-11-19T11:49:18.076-05:00 level=DEBUG msg="Running command" command="/usr/bin/chmod a+wr /var/snap/lxd/common/lxd/unix.socket"
time=2024-11-19T11:49:18.077-05:00 level=DEBUG msg="Running command" command="/usr/sbin/usermod -a -G lxd guillaume"
time=2024-11-19T11:49:18.129-05:00 level=DEBUG msg="Running command" command="/usr/sbin/iptables -F FORWARD"
time=2024-11-19T11:49:18.130-05:00 level=DEBUG msg="Running command" command="/usr/sbin/iptables -P FORWARD ACCEPT"
time=2024-11-19T11:49:18.145-05:00 level=INFO msg="Prepared provider" provider=lxd
time=2024-11-19T11:49:18.446-05:00 level=INFO msg="Refreshed snap" snap=kubectl
time=2024-11-19T11:49:18.447-05:00 level=DEBUG msg="Running command" command="/snap/bin/k8s status"
time=2024-11-19T11:49:19.502-05:00 level=DEBUG msg="Running command" command="/snap/bin/k8s status --wait-ready"
time=2024-11-19T11:49:20.559-05:00 level=DEBUG msg="Running command" command="/snap/bin/k8s set load-balancer.l2-mode=true"
time=2024-11-19T11:49:20.614-05:00 level=DEBUG msg="Running command" command="/snap/bin/k8s set load-balancer.cidrs=10.43.45.0/28"
time=2024-11-19T11:49:20.668-05:00 level=DEBUG msg="Running command" command="/snap/bin/k8s enable load-balancer"
Command: /snap/bin/k8s enable load-balancer
Output:
Enabling load-balancer on the cluster. This may take a few seconds, please wait.
Error: Failed to enable load-balancer on the cluster.

The error was: failed to PUT /k8sd/cluster/config: database transaction to update cluster configuration failed: failed to update cluster configuration: failed to update cluster config: updated cluster configuration is not valid: load-balancer requires network to be enabled
time=2024-11-19T11:49:21.714-05:00 level=DEBUG msg="Running command" command="/snap/bin/k8s enable load-balancer"
Command: /snap/bin/k8s enable load-balancer
Output:
Enabling load-balancer on the cluster. This may take a few seconds, please wait.
Error: Failed to enable load-balancer on the cluster.

The error was: failed to PUT /k8sd/cluster/config: database transaction to update cluster configuration failed: failed to update cluster configuration: failed to update cluster config: updated cluster configuration is not valid: load-balancer requires network to be enabled
time=2024-11-19T11:49:23.758-05:00 level=DEBUG msg="Running command" command="/snap/bin/k8s enable load-balancer"
Command: /snap/bin/k8s enable load-balancer
Output:
Enabling load-balancer on the cluster. This may take a few seconds, please wait.
Error: Failed to enable load-balancer on the cluster.

The error was: failed to PUT /k8sd/cluster/config: database transaction to update cluster configuration failed: failed to update cluster configuration: failed to update cluster config: updated cluster configuration is not valid: load-balancer requires network to be enabled
time=2024-11-19T11:49:27.807-05:00 level=DEBUG msg="Running command" command="/snap/bin/k8s enable load-balancer"
Command: /snap/bin/k8s enable load-balancer
Output:
Enabling load-balancer on the cluster. This may take a few seconds, please wait.
Error: Failed to enable load-balancer on the cluster.

The error was: failed to PUT /k8sd/cluster/config: database transaction to update cluster configuration failed: failed to update cluster configuration: failed to update cluster config: updated cluster configuration is not valid: load-balancer requires network to be enabled
time=2024-11-19T11:49:35.862-05:00 level=DEBUG msg="Running command" command="/snap/bin/k8s enable load-balancer"
Command: /snap/bin/k8s enable load-balancer
Output:
Enabling load-balancer on the cluster. This may take a few seconds, please wait.
Error: Failed to enable load-balancer on the cluster.

The error was: failed to PUT /k8sd/cluster/config: database transaction to update cluster configuration failed: failed to update cluster configuration: failed to update cluster config: updated cluster configuration is not valid: load-balancer requires network to be enabled
time=2024-11-19T11:49:51.920-05:00 level=DEBUG msg="Running command" command="/snap/bin/k8s enable load-balancer"
Command: /snap/bin/k8s enable load-balancer
Output:
Enabling load-balancer on the cluster. This may take a few seconds, please wait.
Error: Failed to enable load-balancer on the cluster.

The error was: failed to PUT /k8sd/cluster/config: database transaction to update cluster configuration failed: failed to update cluster configuration: failed to update cluster config: updated cluster configuration is not valid: load-balancer requires network to be enabled
jnsgruk commented 1 week ago

Hmm that's weird - network should be enabled by default. Either way we can explicitly enable it for sure - can you open another issue and I'll look at it tomorrow/later this week?

gruyaume commented 1 week ago

Hmm that's weird - network should be enabled by default. Either way we can explicitly enable it for sure - can you open another issue and I'll look at it tomorrow/later this week?

Yes, network is enabled by default, but if you have a pre-existing Canonical K8s environment where network was disabled, then you run concierge prepare, concierge will not enable network.

jnsgruk commented 1 week ago

Ah interesting. Yeh in that case I'll add it to the things to enable at least for the presets.

(Edit: See #19)