Open joshuasimon-taulia opened 1 year ago
Hi, thanks for taking my idea to the table, I'm so happy because of that :) I hope I will be able to help you somehow with it soon. Atm I can jump into the discussion at least.
In my opinion the target behavior could be to not render the YAML's and to use helmfiles in ArgoCD directly. That would allow us to remove a lot of the jx gitops
commands code, have less to maintain - the jx
cli already has too many commands.
Secondly I think we should go into a direction to separate two groups of resources from each other and treat them separately.
a) First group is for stateful resources and resources that have recursive dependencies. Eamples: StatefulSet, Namespace, PVC. Those kinds of resources should be protected from accidential pruning using ArgoCD's built-in mechanisms
b) Second group could be the rest of resources that should be pruned automatically and synced automatically.
kind: AppProject
objects with restrictions like following examples:
That allows to raise an alert if we accidently put an object to a place where it should not be.
Going further;
I think that we could try to remove project's configuration in ConfigMap form from the bootrepo and replace them with CRD's eg. JXProject
. When ArgoCD will sync those CRD's to the cluster then:
JXProject
JXPRoject
from the cluster into a configmap that the Lighthouse will understand and apply to the clusterThat would result in:
kubectl
on the cluster and playing with the configuration as part of testing somethingRegarding the ArgoCD installation I would leave a possibility to have a custom way of installation, because there are multiple ways of installing ArgoCD, including by kustomize, by helm, using operator.
Probably best would be to split it in docs into two steps:
The second point should be universal and independent of the type of ArgoCD installation. I have worked in a project, where we were using an ArgoCD operator and installing dozens of instances with its specific settings.
I just had loose a thought about the JXProject
CRD - introducing it we could give users the possibility to extend Jenkins X with their own operators.
For example people will be able to create operators reacting on project creation/deletion to e.g. register it in monitoring, or even in tasks management software (like pipeline status integration in some external systems).
And more simpler form - ArgoCD has a post-sync hooks, a post-sync hook can be used to run e.g. Python script evertime some project in ArgoCD is synced.
I see a lot of benefits of moving from bootjob to ArgoCD :smile:
@ankitm123 @msvticket @tomhobson how do you feel about
chore: regenerate
commit backs to the gitops repo with pr comments that contain the output helmfile diff
or argo diff
. ex: bot and a github action that post the output of as a PR commentApplication
object per k8s namespace which contains all k8s objects rendered by the respective helmfile. there is currently no good way to loop through the releases
array in helmfile.yaml in the an argo ApplicationSets
or split the output of the ConfigManagementPlugin
into multiple argo applications. ideally for organizational purposes, we would create 1 argo Application
per helm chart release in helmfile.yaml. you can see what i mean in the above screenshot
The release pipeline on the gitops cluster repo (bootjob) has a few issues
We run ArgoCD to sync the our non-jenkins-x config repo to our preprod environment and it is very intuitive and flexible. There is a proposal in the kubernetes slack #jenkins-x-dev channel to replace the above process with Argo CD
The most basic POC could look something like
install the ArgoCD helm chart via terraform with lifecycle configuration to ignore all future changes. normally, we would manage helm releases via helmfile, but because we need to bootstrap the cluster and run a first ArgoCD sync, we can use the terraform helm provider
argo-cd-apps
or somethinghelmfile diff
orargo diff
as a PR commenti have some rough ideas here: https://github.com/jenkins-x/terraform-google-jx/pull/228 https://github.com/joshuasimon-taulia/helm-argo-cd-apps/commit/4572c611887190bc4f9640e177a8e902ff7b6558
this is what the demo appset generates in my
atlantis
project when you click on the "namespace" application, the ui drills down into your actual k8s objects