Manifest post processors

[alexmt]

Summary

Support post-processors that modify manifests generated by argocd-repo-server.

Motivation

Postprocessors might solve the following use-cases:

• Injecting secrets. The https://github.com/IBM/argocd-vault-plugin/ implements secret injection. Currently, the project has to be used as a config management plugin which has a disadvantage: user no longer benefits from Helm/Kustomize first-class approach.

• Injecting mandatory label/annotations; sidecar injection etc.

• Linting: post-processors might fail manifest generation and produce the error

Proposal

TBD

[jannfis]

I do like this proposal. I feel manifest post-processors could be used for a variety of great use-cases.

This could also help image updater to provide a mechanism to update images in other types of applications than Kustomize or Helm.

If you submit an enhancement proposal for this, feel free to add me as sponsor :)

[seizadi]

I think it is implied but just to be clear, the changes also would be part of the logic to compare deployed manifest with Git Repo so that mutations are made before comparison. Also the API/UI would show The Git Repo, what is deployed and the changes made due to the mutations.

[jessesuen]

Not sure I like this idea... It seems like we're straying into the business of config management but also validation/mutating webhooks, and will probably do a worse job of it. The Secrets use case seems difficult to achieve in a multi-tenant way.

What would post processors look like? Scripts that accept STDIN? patches? Are we proposing for post-processing to happen at a global scope? project scope? Will there be variable support in post-processing (e.g. application name, project name, etc...)?

Keep in mind Helm and Kustomize already have their post-processors, so we would be adding our own flavor of it.

• https://helm.sh/docs/topics/advanced/#post-rendering
• Kustomize itself would be considered a post-processor

[seizadi]

I consider mutating web hooks being part of configuration management, and changes made by them should indicate the desired state. There are some examples of our use cases for using them here: https://github.com/argoproj/argo-cd/issues/7627

What would post processors look like? Scripts that accept STDIN? patches? I think post-processors should be written as plugins using the pattern in kubernetes. You can see a sample code for one of the web hooks we use here: https://github.com/jainishshah17/tugger/blob/master/cmd/tugger/main.go#L104

Are we proposing for post-processing to happen at a global scope? project scope? Will there be variable support in post-processing (e.g. application name, project name, etc...)? Generally the web hooks run on the cluster and have global scope, but you could see use cases where some project/namespaces are listed as exception. Here is sample web hook that has exception for namespaces: https://github.com/jainishshah17/tugger/blob/master/cmd/tugger/main.go#L53

Keep in mind Helm and Kustomize already have their post-processors, so we would be adding our own flavor of it. It would be desirable to make all changes and save that state to Github. We have found in some cases e.g. change to docker registry that this process take time and require coordination from a large group of application developers. The mutating web hook pattern allows us to quickly make changes while we coordinate with development teams to make sure they make appropriate changes and we can remove a mutation from system.

[iMartyn]

Just to add my input here :

In my opinion secrets injection is the big thing here. Both the methods I would like to use require "faking" helm in the argocd-repo-server container in a very hacky way.

• First would be with something like sops and helm-secrets, which requires the encrypted secrets in the repo, and requires kludging in the helm binary with a wrapper manually.
• Secondly, with a real plugin like argocd-vault-plugin where I also have to add stuff to the heml-repo-server to register the plugin but then I also loose all the helm functionality (examining/editing the parameters, providing a values,yaml block)

Whilst I appreciate not wanting to stray into mutating webhooks, but mutating webhooks do not solve the problem of third-party charts which need a secret in the values.

My use case : I want to be able to create a secret in vault, reference it in the parameters of the argocd application via the gui, and install it.

That usecase does not work for a mutating webhook unless I write a brand new mutating webhook for every application I find on the internet I want to test in a cluster.

[beekhof]

Keep in mind Helm and Kustomize already have their post-processors, so we would be adding our own flavor of it.

Alternatively, we could create an interface for driving those capabilities that is compatible with the existing Helm/Kustomize first-class approaches (unlike using plugins today).

[jan-hudec]

Keep in mind Helm and Kustomize already have their post-processors, so we would be adding our own flavor of it.

* https://helm.sh/docs/topics/advanced/#post-rendering

* Kustomize itself would be considered a post-processor

For the use-cases I have in mind, and most of the others I've noticed above, the helm post-rendering is fully appropriate. The problems using it with ArgoCD as is are basically:

• There does not seem to be an option to invoke the post-render (it can only be specified with a command-line argument and I didn't see any option for ‘additional helm arguments’ in the application definition.
• Only helm has that feature at all, but it would be very useful to mix&match with the other tools.

What would post processors look like? Scripts that accept STDIN? patches?

Yaml on STDIN, modified yaml on STDOUT. Just pipe the output of the primary generator through it (which is exactly what helm does).

Are we proposing for post-processing to happen at a global scope? project scope?

Project scope. It'd be just combining multiple tools so they can be installed separately. Admission webhooks sound better for the global scope.

Will there be variable support in post-processing (e.g. application name, project name, etc...)?

The post-processor should get the environment variables just like the primary generator.

Basically instead of writing a new config management plugin that does helm-expand | kbld | vault | … one would specify helm as plugin and kbld, vault etc. as post-processors and deploy as separate side-cars so that images published by upstream can be used as they are.

[jdeus]

Any news?