An ApplicationSync Custom resource

[empath]

(This is something I'll work on myself if there's interest)

Tekton pipelines has a custom resource called a "TaskRun" which triggers a task in their pipeline, and returns a status.

Inspired by that, I suggest implementing an ApplicationSync custom resource that triggers an Application Sync and returns status-- this would, I think, cover the only functionality of argo that requires an api call currently rather than applying a kubernetes resource.

So in order to trigger an application sync, one would just do something like this:

application-sync.yaml

kind: ApplicationSync
metadata:
  generateName: Sync-node-front-end
spec:
  application: node-front-end
  force: true
  prune: true
  revision: HEAD
  strategy: hook
  timeout: 600

and running it would be just kubectl create -n argocd -f application-sync.yaml

then it would sync the application and return a status object with the result of the sync, the time it ran, etc.

I'm not sure if you'd want to create these every time argo does a sync, that might be a little bit heavy, but it would give you a history of syncs that argo could leverage for the ui, if you wanted.

Thoughts?

[jessesuen]

This is actually a very interesting proposal. The concept is analogous to batch/Jobs, or even argoproj.io/Workflows, but specific to application syncs. It would involve a separate, "sync" controller which would essentially run these SyncJobs to completion. The current application-controller, would only be responsible for reporting on health and sync status.

Perhaps if we were starting from the beginning and had thought of this approach earlier, we may have gone this direction (which is ironic, having implemented the workflow controller).

I'm not sure if you'd want to create these every time argo does a sync, that might be a little bit heavy,

I think if we were to do this, we should flip the entire implementation of sync to this approach, and not have both the current way to perform syncs (which is an operation field in the app object), and the custom resource way. I don't believe the approach to be too heavy, as long as we have a GC mechanism.

The benefits I see to the custom resource sync are:

1. ability to launch sync jobs outside of argocd api server
2. the ability to see a history of sync jobs
3. better separation of concerns inside the application-controller (reconciliation vs. sync)
4. ability to "queue" up sync jobs -- currently we do not allow more than one operation to happen at a time.
5. extensibility -- in the future, this object could support other operations aside from sync

If we do this, it would be quite undertaking and carries quite a bit risk. Here would be where the work and areas concerns may be:

1. UI work to support history listing instead of last sync operation.
2. splitting the application-controller logic into two (it would still single deployment, but multiple controllers in same process). In terms of current implementation, I actually believe the actual logic in the controller is actually already split somewhat cleanly for the reconciliation logic vs. the sync operation logic, so it may not be too terrible to split.
3. auto-sync would need to be very careful since it could potentially spam sync jobs
4. deletion of apps while performing sync
5. terminating a sync job
6. sync metrics would be moved to new controller

In general, I'm quite open to this proposal, but we may not be able to prioritize this any time soon, given that what we have currently works pretty well, and theres some other features we are targeting first.

I also do believe that it's possible for this CRD and controller to be built out separately, without any changes needed in Argo CD.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jessesuen]

Revisiting this again and expanding the scope of the proposal to allow the ability to sync multiple applications in a ordered manner. @jannfis will work on the spec/proposal and update with a working document.

[jwenz723]

Is there any details that are documented or branches that can be used to track the progress of this issue?

I'm interested in setting up a particular argo application to be deployed to multiple environments in sequential order. I attempted to set this behavior up using sync-waves with the app-of-apps pattern, but that doesn't work reliably as @jessesuen mentioned in #4577, so hoping to be able to achieve the same behavior using this feature.

[lnattrass]

Hey team! Is there any progress on this piece, or do you have any idea where it fits in the roadmap for ArgoCD?

[ghostsquad]

I think that https://github.com/maruina/argocd-progressive-rollout-controller this is a POC of that

[nitishfy]

@jannfis Is this issue still open to work on or should we close it?

[andrii-korotkov-verkada]

We already have a sync information in the application status. I'm not convinced a separate object is worth it. Though maybe at the time the status didn't have that info.