search

olga-mir / k8s-multi-cluster

Experiments with multi cluster patterns
Apache License 2.0
7 stars 0 forks source link

Use ClusterResourceSet to install CNI during CAPI cluster bootstrap. #5

Closed olga-mir closed 2 years ago

olga-mir commented 2 years ago

ClusterResourceSet is experimental feature in CAPI: https://cluster-api.sigs.k8s.io/tasks/experimental-features/experimental-features.html?highlight=feature-gates#experimental-features

This PR enables this feature gate and uses it to pass calico CNI manifests in order to install it during cluster bootstrap instead of waiting when it is right time to apply and re-trying in script.

KubeadmControlPlane (kcp) goes from UNAVAILABLE to AVAILABLE without kubectl apply calico in the script.

% k get kcp
NAME                 CLUSTER   INITIALIZED   API SERVER AVAILABLE   REPLICAS   READY   UPDATED   UNAVAILABLE   AGE     VERSION
mgmt-control-plane   mgmt      true          true                   1          1       1         0             6m19s   v1.22.8
%
% k get clusterresourceset
NAME         AGE
crs-calico   7m21s
%
% k get clusterresourceset crs-calico -o yaml | yq e '.status' -
conditions:
  - lastTransitionTime: "2022-05-29T11:48:08Z"
    status: "True"
    type: ResourcesApplied

After installing Flux on AWS mgmt cluster:

NAME                            REVISION                                SUSPENDED       READY   MESSAGE
gitrepository/flux-system       feature/cluster-resource-set/2a9c8d9    False           True    stored artifact for revision 'feature/cluster-resource-set/2a9c8d96ed75ed79924bad0d56dce6e29afc2a3c'

NAME                            REVISION                                SUSPENDED       READY   MESSAGE
kustomization/flux-system       feature/cluster-resource-set/2a9c8d9    False           True    Applied revision: feature/cluster-resource-set/2a9c8d9
kustomization/infrastructure    feature/cluster-resource-set/2a9c8d9    False           True    Applied revision: feature/cluster-resource-set/2a9c8d9