Multiple application resource tracking

[ItayZiv]

Summary

Allow a resource to be managed by multiple applications, especially when only a partial manifest is provided (via server-side applying)

Motivation

Many operators use large CRs which can have certain configuration parameters which are related to different "logical" concerns. For example I can have a IngressController configuration, where I have the actual set up of the controller, default SSL configuration, and audit logging separated into different apps. In this case I can use server-side apply to properly create and apply changes as needed, but there will always be a diff because they belong to different applications so they will override the resource tracking label/annotation. The only way to make it work without diffs would be to create a sync hook job that patches the changes, but this doesn't really align with the whole gitops idea.

Proposal

Either support multiple resource tracking annotations (for different applications), or multiple applications in the same annotation. Another option is to somehow have "secondary" applications where they don't add the normal resource tracking annotation so cascade deletion won't delete those resources, only deletion of the app which is the "primary" resource owner would delete it

[andrii-korotkov-verkada]

Why not have separate resources in this case?

[ItayZiv]

I am talking about situations where there is a single resource. A prominent example is OpenShift clusters, but this is not limited to them, there are operators that have "Singleton" resources or in general resources that handle a lot of logic. Depending on your ArgoCD usage, might be better fit in separate apps. While technically you could merge everything into 1 big app, and use some config management plugin to control different parts of the resource, this can very easily get out of hand and have all your configuration become a single app.

[andrii-korotkov-verkada]

You can also just have one separate apps for such singletons.

[ItayZiv]

While this is possible there are cases where this gets quite messy. For example in OpenShift configuring authentication is done via a Singleton OAUTH resource that has a list of identity providers. If I have an app that deploys a new auth provider (like Keycloak), it would make sense to have it add itself to the resource and not have to change it in the "Auth" app.

Another example I ran into in a few different cases is that I have the apps that setup the normal configuration of resources, but have an additional app to apply hardening policies and extract audit logs.

In general this limitation feels quite artificial since from a technical perspective nothing prevents a resource from existing in both apps, other than the fact that either app will be out of sync (this can be solved by ignoring diffs on both apps, but the the owning apps will change every sync)