experiment with cluster config (2 day spike)

[antonyquant123]

https://github.com/weaveworks/clusters-config

see if this can be utilised for development alongside or instead of tilt

[ranatrk]

resources/docs:

• https://github.com/weaveworks/clusters-config/blob/main/README.md
• https://github.com/weaveworks/clusters-config/blob/main/docs/cluster.md#requesting-a-cluster

Notes:

Installation/Setup

• Install : aws cli, eksctl, gsts (need to add unalias gsts in ./zshrc & installed with npm), pre-commit
• Setup
  • create issue to request permissions in corp
  • login:

    export AWS_PROFILE=sts
    export AWS_ROLE_EKS="arn:aws:iam::894516026745:role/WeaveEksEditor"
    export GOOGLE_IDP_ID=C0203uytv
    export GOOGLE_SP_ID=656726301855
    gsts --aws-role-arn "$AWS_ROLE_EKS" --sp-id "$GOOGLE_SP_ID" --idp-id "$GOOGLE_IDP_ID" --username <EMAIL>

    • first time will open browser to login with the gmail, after that login doesn't prompt anything

Create cluster

• make request-cluster ARGS="--team pesto --cluster-name <CLUSTER_NAME> --weave-branch <BRANCH_NAME> --weave-mode enterprise --delete-after 10"
• the previous command will create the required files, and checkout to a new branch
• commit the new files and push to the remote new branch
• the push will trigger the creation of the cluster, that can be monitored in the actions tab
• slack channel : clusters-config created clusters will send notification there
• download kubeconfig for created cluster eksctl utils write-kubeconfig --region eu-north-1 --cluster <CLUSTER_NAME> --kubeconfig=$HOME/.kube/config

  UI:

• Frontend WGE: kubectl port-forward -n flux-system svc/clusters-service 9001:8000
• can also be accessed via CLUSTER_NAME.eng-sandbox.weave.works/templates/aws-eks/create
• login-> username: wego-admin , password: password

Templates:

• working on creating clusters from sample template provided in repo (created GitopsCluster with CAPI template)

• some points to test further on the cluster created

  • [ ] template profiles
  • [ ] adding applications
  • [ ] multitenancy
  • [ ] using secrets in leaf clusters

[ranatrk]

https://gist.github.com/ranatrk/d13b831ae4b9202926cba580cc06385f