canonical / cluster-api-bootstrap-provider-microk8s

This project offers a cluster API bootstrap provider controller that manages the node provision of a MicroK8s cluster.
https://microk8s.io
21 stars 15 forks source link
cluster-api kubernetes microk8s

Cluster API bootstrap provider for MicroK8s

Cluster API provides declarative APIs to provision, upgrade, and operate Kubernetes clusters.

The bootstrap provider controller in cluster API is responsible for initializing the control plane and worker nodes of the provisioned cluster.

This project offers a cluster API bootstrap provider controller that manages the node provision of a MicroK8s cluster. It is expected to be used along with the respective MicroK8s specific control plane provider.

Prerequisites

Installation

To to configure clusterctl with the two MicroK8s providers edit ~/.cluster-api/clusterctl.yaml and add the following:

providers:
  - name: "microk8s"
    url: "https://github.com/canonical/cluster-api-bootstrap-provider-microk8s/releases/latest/bootstrap-components.yaml"
    type: "BootstrapProvider"
  - name: "microk8s"
    url: "https://github.com/canonical/cluster-api-control-plane-provider-microk8s/releases/latest/control-plane-components.yaml"
    type: "ControlPlaneProvider"

You will now be able now to initialize clusterctl with the MicroK8s providers:

clusterctl init --bootstrap microk8s --control-plane microk8s -i <infra-provider-of-choice>

Alternatively, you can build the providers manually as described in the following section.

Building from source

Usage

As soon as the bootstrap and control-plane controllers are up and running you can apply the cluster manifests describing the desired specs of the cluster you want to provision. Each machine is associated with a MicroK8sConfig through which you can set the cluster's properties. Please review the available options in the respective definitions file. You may also find useful the example manifests found under the examples directory. Note that the configuration structure followed is similar to the the one of kubeadm, in the MicroK8sConfig you will find a CLusterConfiguration and an InitConfiguration sections. When targeting a specific infrastructure you should be aware of which ports are used by MicroK8s and allow them in the network security groups on your deployment.

Two workload cluster templates are available under the templates folder, which are actively used to validate releases:

Before generating a cluster, ensure that you have followed the instructions to set up the infrastructure provider that you will use.

AWS

NOTE: Ensure that you have properly deployed the AWS infrastructure provider prior to executing the commands below. See Initialization for common providers

Generate a cluster template with:

# review list of variables needed for the cluster template
clusterctl generate cluster microk8s-aws --from ./templates/cluster-template-aws.yaml --list-variables

# set environment variables (edit the file as needed before sourcing it)
source ./templates/cluster-template-aws.rc

# generate cluster
clusterctl generate cluster microk8s-aws --from ./templates/cluster-template-aws.yaml > cluster-aws.yaml

Then deploy the cluster with:

microk8s kubectl apply -f cluster-aws.yaml

The MicroK8s cluster will be deployed shortly afterwards. You can see the provisioned machines with:

# microk8s kubectl get machines
NAME                                 CLUSTER        NODENAME          PROVIDERID                              PHASE     AGE   VERSION
microk8s-aws-control-plane-wd94k     microk8s-aws   ip-10-0-95-69     aws:///us-east-1a/i-0051fa0d99ae18e43   Running   10m   v1.23.0
microk8s-aws-md-0-766784ddc4-75m6q   microk8s-aws   ip-10-0-120-139   aws:///us-east-1a/i-0eb890a181c9b8215   Running   14m   v1.23.0
microk8s-aws-control-plane-nkx2b     microk8s-aws   ip-10-0-218-228   aws:///us-east-1c/i-0a28bd5ac4ee1ce5d   Running   10m   v1.23.0
microk8s-aws-control-plane-f4tp2     microk8s-aws   ip-10-0-100-255   aws:///us-east-1a/i-0280ea368cd050dd2   Running   10m   v1.23.0
microk8s-aws-md-0-766784ddc4-n8csl   microk8s-aws   ip-10-0-69-166    aws:///us-east-1a/i-01cbbd4c07df2eb8c   Running   14m   v1.23.0

With clusterctl you can then retrieve the kubeconfig file for the workload cluster:

clusterctl get kubeconfig microk8s-aws > kubeconfig

And then use it to access the cluster:

# KUBECONFIG=./kubeconfig kubectl get node
NAME              STATUS   ROLES    AGE     VERSION
ip-10-0-95-69     Ready    <none>   9m11s   v1.23.9-2+88a2c6a14e7008
ip-10-0-120-139   Ready    <none>   2m45s   v1.23.9-2+88a2c6a14e7008
ip-10-0-69-166    Ready    <none>   2m22s   v1.23.9-2+88a2c6a14e7008
ip-10-0-100-255   Ready    <none>   108s    v1.23.9-2+88a2c6a14e7008
ip-10-0-218-228   Ready    <none>   110s    v1.23.9-2+88a2c6a14e7008

To discard the cluster, delete the microk8s-aws cluster.

microk8s kubectl delete cluster microk8s-aws

NOTE: This command might take a while, because it ensures that the providers properly clean up any resources (Virtual Machines, Load Balancers, etc) that were provisioned while setting up the cluster

... and also the secrets created:

# microk8s kubectl delete secret microk8s-aws-jointoken  microk8s-aws-ca  microk8s-aws-kubeconfig
secret "microk8s-aws-jointoken" deleted
secret "microk8s-aws-ca" deleted
secret "microk8s-aws-kubeconfig" deleted

Note: if you want to provide your own CA and/or the join token used to form a cluster you will need to create the respective secrets (<cluster-name>-ca and <cluster-name>-jointoken) before applying the cluster manifests.

Note: the default cluster template for AWS ensures that the default security groups created by the AWS infrastructure provider are sufficient for the cluster to work as expected. For more complex scenarios, you might have to configure your own security groups and set the AWSCluster spec accordingly. For more details, refer to the upstream AWS provider documentation.

OpenStack

NOTE: Ensure that you have properly deployed the OpenStack infrastructure provider prior to executing the commands below. See Initialization for common providers

Create a cloud-config secret for the OpenStack API in the management cluster with the following command. Make sure to replace the OpenStack credentials in the command below to match your OpenStack cloud. If unsure, consult "Horizon" > "API Access" > "Download OpenStack RC File" > "OpenStack clouds.yaml File".

sudo microk8s kubectl create -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: cloud-config
  labels:
    clusterctl.cluster.x-k8s.io/move: "true"
stringData:
  cacert: ''
  clouds.yaml: |
    clouds:
      openstack:
        auth:
          auth_url: "${OS_AUTH_URL}"
          username: "${OS_USERNAME}"
          password: "${OS_PASSWORD}"
          project_name: "${OS_PROJECT_NAME}"
          user_domain_name: "${OS_USER_DOMAIN_NAME}"
          project_domain_name: "${OS_PROJECT_DOMAIN_NAME}"
        region_name: "${OS_REGION_NAME}"
        interface: "public"
        verify: false
        identity_api_version: 3
EOF

Generate a cluster template with:

# review list of variables needed for the cluster template
clusterctl generate cluster microk8s-openstack --from ./templates/cluster-template-openstack.yaml --list-variables

# set environment variables (edit the file as needed before sourcing it)
source ./templates/cluster-template-openstack.rc

# generate cluster
clusterctl generate cluster microk8s-openstack --from ./templates/cluster-template-openstack.yaml > cluster-openstack.yaml

Then deploy the cluster with:

microk8s kubectl apply -f cluster-openstack.yaml

Azure

NOTE: Ensure that you have properly deployed the Azure infrastructure provider prior to executing the commands below. See Initialization for common providers

Generate a cluster template with:

# review list of variables needed for the cluster template
clusterctl generate cluster microk8s-azure --from ./templates/cluster-template-azure.yaml --list-variables

# set environment variables (edit the file as needed before sourcing it)
source ./templates/cluster-template-azure.rc

# generate cluster
clusterctl generate cluster microk8s-azure --from ./templates/cluster-template-azure.yaml > cluster-azure.yaml

Then deploy the cluster with:

microk8s kubectl apply -f cluster-azure.yaml

Note: Make sure you have the secret to include the password of the Service Principal identity. This secret will be referenced by the AzureClusterIdentity used by the AzureCluster.

GCP

NOTE: Ensure that you have properly deployed the GCP infrastructure provider prior to executing the commands below. See Initialization for common providers

Prior to generate a cluster template, you need to create a VM image for use in the cluster. The MicroK8s provider works with any stock Ubuntu image. Use the Ubuntu 22.04 LTS image with:

gcloud compute images create ubuntu-2204 --source-image-project ubuntu-os-cloud --source-image-family ubuntu-2204-lts

Make note of the name of the image ubuntu-2204, which we then feed into the cluster template.

Generate a cluster template with:

# review list of variables needed for the cluster template
clusterctl generate cluster microk8s-gcp --from ./templates/cluster-template-gcp.yaml --list-variables

# set environment variables (edit the file as needed before sourcing it)
source ./templates/cluster-template-gcp.rc

# generate the cluster
clusterctl generate cluster microk8s-gcp --from ./templates/cluster-template-gcp.yaml > cluster-gcp.yaml

Then, deploy the cluster with:

microk8s kubectl apply -f cluster-gcp.yaml

Development

The two MicroK8s CAPI providers, the bootstrap and control plane, serve distinct purposes:

The bootstrap provider

The control plane provider

Deployment of components

Interactions among controllers