Clean up and re-org config, template Cilium and CAPI resources and fix the complete 3 cluster setup e2e
Installation
deploy permanent management cluster on AWS (using temp kind cluster and then pivot)
./scripts/deploy.sh
flux on management cluster will apply CAPI manifests that are currently present in the repo.
When script is complete run script to finalize workload clusters (install cilium which currently is not vi CRS - due to dynamic KAS address) and flux secret (WIP to eliminate this step).
This script without arguments will discover all workload clusters and perform all necessary adjustments:
./scripts/workload-cluster.sh
Adding a new cluster
Hands free with just one script!
To add a new cluster create config env for it by copying existing file (./config/cluster-<num>.env) and modifying values. This is intended to be manual as script can't or shouldn't guess this values (or too difficult in bash like calc next CIDR)
./scripts/workload-cluster.sh -n cluster-02
This will generate all necessary files and add the cluster to mgmt kustomization list too. Then it will be pushed to the repo (example commit from the script: https://github.com/olga-mir/k8s-multi-cluster/pull/10/commits/92ee7e094881969736ed666a0e732f073ebc53c6), where flux will apply it and capi will provision it. The ./scripts/workload-cluster.sh is still waiting for the cluster to come up and finalize the installation.
on mgmt cluster:
% k get cluster -A
NAMESPACE NAME PHASE AGE VERSION
cluster-01 cluster-01 Provisioned 12m
cluster-02 cluster-02 Provisioning 60s
cluster-mgmt cluster-mgmt Provisioned 13m
Clean up and re-org config, template Cilium and CAPI resources and fix the complete 3 cluster setup e2e
Installation
deploy permanent management cluster on AWS (using temp
kindcluster and then pivot)flux on management cluster will apply CAPI manifests that are currently present in the repo.
When script is complete run script to finalize workload clusters (install cilium which currently is not vi CRS - due to dynamic KAS address) and flux secret (WIP to eliminate this step). This script without arguments will discover all workload clusters and perform all necessary adjustments:
Adding a new cluster
Hands free with just one script!
To add a new cluster create config env for it by copying existing file (
./config/cluster-<num>.env) and modifying values. This is intended to be manual as script can't or shouldn't guess this values (or too difficult in bash like calc next CIDR)This will generate all necessary files and add the cluster to mgmt kustomization list too. Then it will be pushed to the repo (example commit from the script: https://github.com/olga-mir/k8s-multi-cluster/pull/10/commits/92ee7e094881969736ed666a0e732f073ebc53c6), where flux will apply it and capi will provision it. The
./scripts/workload-cluster.shis still waiting for the cluster to come up and finalize the installation.on mgmt cluster:
Hubble
Current setup enables hubble relay and UI.
https://docs.cilium.io/en/v1.11/gettingstarted/hubble/#hubble-ui or in CLI with instructions from here: https://docs.cilium.io/en/v1.11/gettingstarted/hubble_setup/