Starttoaster
commented
10 months ago In case it is useful, here is my k0sctl config (note that I do not believe this is an error with k0sctl though):
apiVersion: k0sctl.k0sproject.io/v1beta1
kind: Cluster
metadata:
name: cluster1
spec:
hosts:
- openSSH:
address: 100.85.216.39
user: bb
port: 22
role: controller+worker
installFlags:
- --no-taints
- openSSH:
address: 100.72.143.21
user: bb
port: 22
role: controller+worker
installFlags:
- --no-taints
- openSSH:
address: 100.81.77.109
user: bb
port: 22
role: controller+worker
installFlags:
- --no-taints
k0s:
config:
spec:
network:
nodeLocalLoadBalancing:
enabled: true
type: EnvoyProxy
version: 1.26.7+k0s.0
dynamicConfig: false
twz123
Before creating an issue, make sure you've checked the following:
Platform
Version
v1.26.7+k0s.0
Sysinfo
`k0s sysinfo`
Machine ID: "f1028d6abf81c1b8b72ad4e97b70aebac34433f8629b2767f04264d6e9877937" (from machine) (pass) Total memory: 15.2 GiB (pass) Disk space available for /var/lib/k0s: 1.2 TiB (pass) Operating system: Linux (pass) Linux kernel release: 5.15.0-88-generic (pass) Max. file descriptors per process: current: 1048576 / max: 1048576 (pass) Executable in path: modprobe: /usr/sbin/modprobe (pass) /proc file system: mounted (0x9fa0) (pass) Control Groups: version 2 (pass) cgroup controller "cpu": available (pass) cgroup controller "cpuacct": available (via cpu in version 2) (pass) cgroup controller "cpuset": available (pass) cgroup controller "memory": available (pass) cgroup controller "devices": available (assumed) (pass) cgroup controller "freezer": available (assumed) (pass) cgroup controller "pids": available (pass) cgroup controller "hugetlb": available (pass) cgroup controller "blkio": available (via io in version 2) (pass) CONFIG_CGROUPS: Control Group support: built-in (pass) CONFIG_CGROUP_FREEZER: Freezer cgroup subsystem: built-in (pass) CONFIG_CGROUP_PIDS: PIDs cgroup subsystem: built-in (pass) CONFIG_CGROUP_DEVICE: Device controller for cgroups: built-in (pass) CONFIG_CPUSETS: Cpuset support: built-in (pass) CONFIG_CGROUP_CPUACCT: Simple CPU accounting cgroup subsystem: built-in (pass) CONFIG_MEMCG: Memory Resource Controller for Control Groups: built-in (pass) CONFIG_CGROUP_HUGETLB: HugeTLB Resource Controller for Control Groups: built-in (pass) CONFIG_CGROUP_SCHED: Group CPU scheduler: built-in (pass) CONFIG_FAIR_GROUP_SCHED: Group scheduling for SCHED_OTHER: built-in (pass) CONFIG_CFS_BANDWIDTH: CPU bandwidth provisioning for FAIR_GROUP_SCHED: built-in (pass) CONFIG_BLK_CGROUP: Block IO controller: built-in (pass) CONFIG_NAMESPACES: Namespaces support: built-in (pass) CONFIG_UTS_NS: UTS namespace: built-in (pass) CONFIG_IPC_NS: IPC namespace: built-in (pass) CONFIG_PID_NS: PID namespace: built-in (pass) CONFIG_NET_NS: Network namespace: built-in (pass) CONFIG_NET: Networking support: built-in (pass) CONFIG_INET: TCP/IP networking: built-in (pass) CONFIG_IPV6: The IPv6 protocol: built-in (pass) CONFIG_NETFILTER: Network packet filtering framework (Netfilter): built-in (pass) CONFIG_NETFILTER_ADVANCED: Advanced netfilter configuration: built-in (pass) CONFIG_NETFILTER_XTABLES: Netfilter Xtables support: module (pass) CONFIG_NETFILTER_XT_TARGET_REDIRECT: REDIRECT target support: module (pass) CONFIG_NETFILTER_XT_MATCH_COMMENT: "comment" match support: module (pass) CONFIG_NETFILTER_XT_MARK: nfmark target and match support: module (pass) CONFIG_NETFILTER_XT_SET: set target and match support: module (pass) CONFIG_NETFILTER_XT_TARGET_MASQUERADE: MASQUERADE target support: module (pass) CONFIG_NETFILTER_XT_NAT: "SNAT and DNAT" targets support: module (pass) CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: "addrtype" address type match support: module (pass) CONFIG_NETFILTER_XT_MATCH_CONNTRACK: "conntrack" connection tracking match support: module (pass) CONFIG_NETFILTER_XT_MATCH_MULTIPORT: "multiport" Multiple port match support: module (pass) CONFIG_NETFILTER_XT_MATCH_RECENT: "recent" match support: module (pass) CONFIG_NETFILTER_XT_MATCH_STATISTIC: "statistic" match support: module (pass) CONFIG_NETFILTER_NETLINK: module (pass) CONFIG_NF_CONNTRACK: Netfilter connection tracking support: module (pass) CONFIG_NF_NAT: module (pass) CONFIG_IP_SET: IP set support: module (pass) CONFIG_IP_SET_HASH_IP: hash:ip set support: module (pass) CONFIG_IP_SET_HASH_NET: hash:net set support: module (pass) CONFIG_IP_VS: IP virtual server support: module (pass) CONFIG_IP_VS_NFCT: Netfilter connection tracking: built-in (pass) CONFIG_NF_CONNTRACK_IPV4: IPv4 connetion tracking support (required for NAT): unknown (warning) CONFIG_NF_REJECT_IPV4: IPv4 packet rejection: module (pass) CONFIG_NF_NAT_IPV4: IPv4 NAT: unknown (warning) CONFIG_IP_NF_IPTABLES: IP tables support: module (pass) CONFIG_IP_NF_FILTER: Packet filtering: module (pass) CONFIG_IP_NF_TARGET_REJECT: REJECT target support: module (pass) CONFIG_IP_NF_NAT: iptables NAT support: module (pass) CONFIG_IP_NF_MANGLE: Packet mangling: module (pass) CONFIG_NF_DEFRAG_IPV4: module (pass) CONFIG_NF_CONNTRACK_IPV6: IPv6 connetion tracking support (required for NAT): unknown (warning) CONFIG_NF_NAT_IPV6: IPv6 NAT: unknown (warning) CONFIG_IP6_NF_IPTABLES: IP6 tables support: module (pass) CONFIG_IP6_NF_FILTER: Packet filtering: module (pass) CONFIG_IP6_NF_MANGLE: Packet mangling: module (pass) CONFIG_IP6_NF_NAT: ip6tables NAT support: module (pass) CONFIG_NF_DEFRAG_IPV6: module (pass) CONFIG_BRIDGE: 802.1d Ethernet Bridging: module (pass) CONFIG_LLC: module (pass) CONFIG_STP: module (pass) CONFIG_EXT4_FS: The Extended 4 (ext4) filesystem: built-in (pass) CONFIG_PROC_FS: /proc file system support: built-in (pass)What happened?
I have a 3-node cluster that is comprised of 3 nodes with the Controller+Worker roles, and just tried out the relatively new node local loadbalancing feature. I see my first node is the current "leader" looking at my current leases in the kube-node-lease namespace. And I see that a single envoy Pod was deployed in kube-system for the third node in my cluster.
So I just shut down the 3rd node to test this out, which of course made envoy completely unreachable. And after about 15 minutes of waiting, the envoy loadbalancer Pod is still stuck in Terminating according to kubectl, without a new Pod being deployed. But I'm currently of the belief, looking at the documentation, that what was intended here was for all 3 of my Controller+Worker nodes to deploy their own envoy Pod. As the documentation states that all Worker nodes are meant to deploy one.
I'm not sure if this is at all relevant, but I was originally using an externalAddress loadbalancer for this cluster, and transitioned to envoy, in case that is the tiny detail that leads to what I believe is an unintended state.
Steps to reproduce
Expected behavior
I'd have expected all 3 of my "Workers", though they also serve the Controller role, to deploy an Envoy Pod.
Actual behavior
Only the 3rd node in my cluster has the Envoy Pod even though the other two nodes are also workers.
Screenshots and logs
No response
Additional context
I checked
/etc/k0s/k0s.yamlon all 3 nodes and found thenodeLocalLoadBalancingsetting configured in them.