Pluggable Health Checks & community health check catalog

[sbose78]

Summary

Introduce a HealthCheck CRD for defining healthchecks for non-standard Kubernetes types ( and overrides to standard Kubernetes types). This would make custom health checks pluggable and more manage-able for a specific deployment of ArgoCD

Motivation

• As an admin, I would like to decouple custom health checks from instance configuration ( in argocd-cm )
• From a codebase maintenance perspective, I would like to separate custom community-contributed health checks from the rest of the codebase.

Proposal

• Introduce the cluster-scoped HealthCheck CRD that would be used to declare the lua script for a specific "GroupVersionKind".
• Gradually deprecate declaration of health checks in argocd-cm.yaml
• Gradually deprecate https://github.com/argoproj/argo-cd/tree/master/resource_customizations for non-standard Kubernetes types
• Create a new repo called community-health-checks where all health checks ( and their tests ) in https://github.com/argoproj/argo-cd/tree/master/resource_customizations would be migrated over as HealthCheck CRs.
• Release the custom health check catalog as a "add-on".

[alexmt]

We've faced several health check issues after upgrading to v2.0: After https://github.com/argoproj/argo-cd/issues/6039 and https://github.com/argoproj/argo-cd/issues/6035. The fix requires Argo CD release although health check is just a Lua script.

At the same time, I really don't want to make Argo CD similar to Istio and ask users to install dozens of CRDs. What do you think about the following compromise solution:

• implement https://github.com/argoproj/argo-cd/issues/5439 and let Argo CD keep using ConfigMap
• create HealthCheck that "manages" resource keys in Argo CD ConfigMap.
• Eventually create community-health-checks and move built-in health checks

This way we can keep the number of required CRDs small. If later we learn that everyone install HealthCheck then we can merge it into Argo CD

What do you think @sbose78 ?

[sbose78]

That sounds like a great approach!

[jessesuen]

@sbose78 we'd like to fold this into the Argo CD extensions proposal, which will replace current mechanism of health checks in the argocd configmap: https://github.com/argoproj/argo-cd/pull/6240

[sbose78]

Thank you, I'll review it, Jesse!