techno-tim / k3s-ansible

The easiest way to bootstrap a self-hosted High Availability Kubernetes cluster. A fully automated HA k3s etcd install with kube-vip, MetalLB, and more. Build. Destroy. Repeat.
https://technotim.live/posts/k3s-etcd-ansible/
Apache License 2.0
2.41k stars 1.05k forks source link

cilium_exportPodCIDR variable missing? Should this be defined in the all.yml under group_vars? #496

Closed byrn-baker closed 6 months ago

byrn-baker commented 6 months ago

cilium_exportPodCIDR variable missing? Should this be defined in the all.yml under group_vars?

Expected Behavior

Enabling cilium the playbook runs and enables k3s with cilium

Current Behavior

fails because it is looking for a boolean cilium_exportPodCIDR

Steps to Reproduce

  1. comment out flannel iface and un-comment cillium interface
  2. run site playbook

Context (variables)

Operating system: Ubuntu 22.04

Hardware: Virtualized

Variables Used

all.yml

# interface which will be used for flannel
# flannel_iface: "eth0"

# uncomment calico_iface to use tigera operator/calico cni instead of flannel https://docs.tigera.io/calico/latest/about
# calico_iface: "eth0"
calico_ebpf: false           # use eBPF dataplane instead of iptables
calico_tag: "v3.27.2"        # calico version tag

# uncomment cilium_iface to use cilium cni instead of flannel or calico
# ensure v4.19.57, v5.1.16, v5.2.0 or more recent kernel
cilium_iface: "eth0"
cilium_mode: "routed"        # native when nodes on same subnet or using bgp, else set routed
cilium_tag: "v1.15.1"        # cilium version tag
cilium_hubble: true          # enable hubble observability relay and ui

# if using calico or cilium, you may specify the cluster pod cidr pool
cluster_cidr: "10.52.0.0/16"

# enable cilium bgp control plane for lb services and pod cidrs. disables metallb.
cilium_bgp: true

# bgp parameters for cilium cni. only active when cilium_iface is defined and cilium_bgp is true.
cilium_bgp_my_asn: "64513"
cilium_bgp_peer_asn: "200"
cilium_bgp_peer_address: "172.16.221.1"
cilium_bgp_lb_cidr: "172.16.228.0/24"   # cidr for cilium loadbalancer ipam

# apiserver_endpoint is virtual ip-address which will be configured on each master
apiserver_endpoint: "172.16.221.222"

# k3s_token is required  masters can talk together securely
# this token should be alpha numeric only
k3s_token: "supersecret123"

# The IP on which the node is reachable in the cluster.
# Here, a sensible default is provided, you can still override
# it for each of your hosts, though.
k3s_node_ip: "{{ ansible_facts[(cilium_iface | default(calico_iface | default(flannel_iface)))]['ipv4']['address'] }}"

# Disable the taint manually by setting: k3s_master_taint = false
k3s_master_taint: "{{ true if groups['node'] | default([]) | length >= 1 else false }}"

# these arguments are recommended for servers as well as agents:
extra_args: >-
  {{ '--flannel-iface=' + flannel_iface if calico_iface is not defined and cilium_iface is not defined else '' }}
  --node-ip={{ k3s_node_ip }}

# change these to your liking, the only required are: --disable servicelb, --tls-san {{ apiserver_endpoint }}
# the contents of the if block is also required if using calico or cilium
extra_server_args: >-
  {{ extra_args }}
  {{ '--node-taint node-role.kubernetes.io/master=true:NoSchedule' if k3s_master_taint else '' }}
  {% if calico_iface is defined or cilium_iface is defined %}
  --flannel-backend=none
  --disable-network-policy
  --cluster-cidr={{ cluster_cidr | default('10.52.0.0/16') }}
  {% endif %}
  --tls-san {{ apiserver_endpoint }}
  --disable servicelb
  --disable traefik

extra_agent_args: >-
  {{ extra_args }}

# image tag for kube-vip
kube_vip_tag_version: "v0.7.2"

# tag for kube-vip-cloud-provider manifest
# kube_vip_cloud_provider_tag_version: "main"

# kube-vip ip range for load balancer
# (uncomment to use kube-vip for services instead of MetalLB)
# kube_vip_lb_ip_range: "172.16.221.80-172.16.221.90"

# metallb type frr or native
metal_lb_type: "native"

# metallb mode layer2 or bgp
metal_lb_mode: "layer2"

# bgp options
# metal_lb_bgp_my_asn: "64513"
# metal_lb_bgp_peer_asn: "64512"
# metal_lb_bgp_peer_address: "172.16.221.1"

# image tag for metal lb
metal_lb_speaker_tag_version: "v0.14.3"
metal_lb_controller_tag_version: "v0.14.3"

# metallb ip range for load balancer
metal_lb_ip_range: "172.16.221.80-172.16.221.90"

# Only enable this if you have set up your own container registry to act as a mirror / pull-through cache
# (harbor / nexus / docker's official registry / etc).
# Can be beneficial for larger dev/test environments (for example if you're getting rate limited by docker hub),
# or air-gapped environments where your nodes don't have internet access after the initial setup
# (which is still needed for downloading the k3s binary and such).
# k3s's documentation about private registries here: https://docs.k3s.io/installation/private-registry
custom_registries: false
# The registries can be authenticated or anonymous, depending on your registry server configuration.
# If they allow anonymous access, simply remove the following bit from custom_registries_yaml
#   configs:
#     "registry.domain.com":
#       auth:
#         username: yourusername
#         password: yourpassword
# The following is an example that pulls all images used in this playbook through your private registries.
# It also allows you to pull your own images from your private registry, without having to use imagePullSecrets
# in your deployments.
# If all you need is your own images and you don't care about caching the docker/quay/ghcr.io images,
# you can just remove those from the mirrors: section.
custom_registries_yaml: |
  mirrors:
    docker.io:
      endpoint:
        - "https://registry.domain.com/v2/dockerhub"
    quay.io:
      endpoint:
        - "https://registry.domain.com/v2/quayio"
    ghcr.io:
      endpoint:
        - "https://registry.domain.com/v2/ghcrio"
    registry.domain.com:
      endpoint:
        - "https://registry.domain.com"

  configs:
    "registry.domain.com":
      auth:
        username: yourusername
        password: yourpassword

# On some distros like Diet Pi, there is no dbus installed. dbus required by the default reboot command.
# Uncomment if you need a custom reboot command
# custom_reboot_command: /usr/sbin/shutdown -r now

# Only enable and configure these if you access the internet through a proxy
# proxy_env:
#   HTTP_PROXY: "http://proxy.domain.local:3128"
#   HTTPS_PROXY: "http://proxy.domain.local:3128"
#   NO_PROXY: "*.domain.local,127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"

Hosts

host.yml

all:
  children:
    k3s_cluster:
      children:
        master:
          hosts:
            lab-k3s01:
              ansible_host: 192.168.18.38
              ansible_username: ansibleuser
              vmid: 3001
              storage: "ssd02-4T"
              vcpus: 4
              memory: 4096
              data_vlan: 221
              data: 172.16.221.38
              mgmt_vlan: 18
            lab-k3s02:
              ansible_host: 192.168.18.39
              ansible_username: ansibleuser
              vmid: 3002
              storage: "ssd02-4T"
              vcpus: 4
              memory: 4096
              data_vlan: 221
              data: 172.16.221.39
              mgmt_vlan: 18
            lab-k3s03:  
              ansible_host: 192.168.18.40
              ansible_username: ansibleuser
              vmid: 3003
              storage: "ssd02-4T"
              vcpus: 4
              memory: 4096
              data_vlan: 221
              data: 172.16.221.40
              mgmt_vlan: 18
        node:
          hosts:
            west-lab-k3s01: 
              ansible_host: 192.168.18.41
              ansible_username: ansibleuser
              vmid: 3004
              storage: "ssd02-4T"
              vcpus: 4
              memory: 4096
              data_vlan: 222
              data: 172.16.222.41
              mgmt_vlan: 18
            east-lab-k3s01: 
              ansible_host: 192.168.18.42
              ansible_username: ansibleuser
              vmid: 3005
              storage: "ssd02-4T"
              vcpus: 4
              memory: 4096
              data_vlan: 223
              data: 172.16.223.42
              mgmt_vlan: 18

Possible Solution

Define cilium_exportPodCIDR boolean in the all.yml