Feature request: passing (different) custom values to a series of helm charts deployed with ApplicationSet

[marcozov]

First, thanks a lot for this great project you have been working on. It's truly amazing to see a feature like this being onboarded.

Here is a description of the the issue that I'm having while working with Argo ApplicationSets. My use case is very similar to this one: https://argocd-applicationset.readthedocs.io/en/stable/Use-Cases/#use-case-self-service-of-argo-cd-applications-on-multitenant-clusters . Basically, we want to be able to allow multiple teams to deploy any application they want, but we want to restrict their "capabilities". This is a perfect match with ApplicationSets, since all we need to do is to enforce the project field in the application template. However, the challenge that I'm facing is for deploying helm charts with this approach: we heavily rely on Helm, and most of the applications are gonna be deployments of Helm charts. Therefore I was wondering if it is possible at all to pass a set of custom values for each application that is created via the template. I tried to pass a block of values to the configuration file that can be controlled by the user (in the example it is called config.json) but it is not accepting it.

Example

There are N helm charts stored in a repository (let's say a dedicated git repostory that stores helm charts, but it's not very relevant). A team (called teamx) wants to deploy them by simply adding a parameters file inside the folder of a specific application (e.g., config.json).

A basic configuration would look like this:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: teamx-poc
  namespace: argocd
spec:
  generators:
  - git:
      repoURL: https://my-git-repo.io
      revision: master
      files:
      - path: "path/to/teamx/folder/**/config.json"
  template:
    metadata:
      name: 'teamx-{{app.name}}'
      namespace: argocd
    spec:
      project: teamx-project # enforce the project for `teamx`
      source:
        repoURL: https://my-git-helm-repo.io # the URL for the git repo that stores helm charts
        targetRevision: master
        path: '{{chart-path}}' # the path of the helm chart, specified in `config.json` in the app folder
        helm:
          values: |-
            ???
      destination:
        server: https://kubernetes.default.svc
        namespace: '{{namespace}}' # the destination namespace, specified in `config.json` in the app folder

The only part that would be missing in this scenario would be the possibility of passing application-specific values to the helm chart, from the config.json file (passing a block with multiple values doesn't seem to be possible at the moment). The ideal solution, in my opinion, would be to implement https://github.com/argoproj/argo-cd/issues/2789 and have a way to pass an external file as a set of values. But this issue is not really about this, it is more about being able to customize helm charts deployments in some way.

Is there any thought about implementing this use case? Do you think it would make sense to have this feature as well?

If this requirement makes sense to you, I would be happy to discuss this more and contribute to the feature.

[nfillot]

Hello @marcozov

But this issue is not really about this, it is more about being able to customize helm charts deployments in some way.

I am using the following syntax to have custom values per environment but values files are indeed in the same repo as the helm chart

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: apm-server
spec:
  generators:
  - list:
      elements:
      - cluster: mycluster1
        url: https://url1
      - cluster: mycluster2
        url: https://url2
  template:
    metadata:
      name: '{{cluster}}-apm-server'
    spec:
      project: default
      source:
        helm:
          valueFiles:
            - values-{{cluster}}.yaml
          releaseName: apm-server
        path: apm-server
        repoURL: https://mycustomrepo/helm.git

[andrejpetras]

Hi,

I solve it differently.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: test
spec:
  generators:
  - git:
      repoURL: git@gitlab.com:1000kit/playground/local/env.git
      revision: test2
      files:
      - path: '*.yaml'
  template:
    metadata:
      name: '{{path.basename}}-test-local'
    spec:
      project: default
      source:
        repoURL: 'https://harbor.1000kit.org/chartrepo/1000kit/'
        targetRevision: '{{version}}'
        chart: '{{chart}}'
        helm:
          releaseName: '{{path.basename}}'
          values: |
            {{values}}
      destination:
        server: https://kubernetes.default.svc
        namespace: 'test'
      syncPolicy:
        automated:
          prune: true
          selfHeal: true

https://github.com/lorislab/applicationset/tree/yaml

I put a short description in the README file. The generator in the ApplicationSet is the git generator which will check if file ends with .yaml Before the template is generated from the yaml file it will be merged with global.yml file if exists. Template parameters are [root key] = string value of the merged yaml.

Source code: https://github.com/lorislab/applicationset/blob/yaml/pkg/generators/git.go#L221

[marcozov]

Thanks for you replies.

@nfillot indeed it can work that way, but I would like to be able to decouple the development of the helm chart from its deployment. That's why I'm looking for a different approach. @andrejpetras this looks exactly like what I was looking for. I was wondering if you're planning to open a PR for this in order to integrate with the next releases of the project. If there is anything I can do to help, you can count me in.

[jgwest]

I think a good first step would be implementing https://github.com/argoproj-labs/applicationset/issues/106 with the same functionality as the JSON parsing, and then we can next discuss how to extend that re: the other changes in lorislab yaml branch, including: per-directory values (the global.yml), changes to how file values are flattened, and addition of new path and path.basename parameters.

[andrejpetras]

@jgwest I agree, my solution is only PoC for my use case.

global.yml is a temporary solution.

JSON flattened doesn't work for me. I want to add a "values" structure like in the yaml file. So I convert the yaml substructure back to a string: https://github.com/lorislab/applicationset/blob/yaml/pkg/generators/git.go#L266

Would it be possible to have some sort of function on the template that could do the conversion?

{{ object.key.sub-key | function }}

Or we can create SDK to write own customApplicationSet controller for ArgoCD.

[marcozov]

So I convert the yaml substructure back to a string: https://github.com/lorislab/applicationset/blob/yaml/pkg/generators/git.go#L266

@andrejpetras that part is for passing those values as a string to the field values of the application, right? Why don't you like the current approach of your PoC? I'm not 100% sure I understand the other approaches that you are suggesting.

[JTarball]

The changes in https://github.com/argoproj-labs/applicationset/pull/211 seem to work if you specify a specific key.

Do we have an eta for being able to do something like below with the values when we need more flexibility?

    helm:
      releaseName: '{{path.basename}}'
      values: |
        {{values}}

[mksha]

even more than that we need a capability to pass the values file as pure yaml to values: currently, it flattens the yaml and json so we can not pass the nested values as map.

But it would be really helpful if we make application set generator as normal go template that allows to pass any kind of yaml values and then we use it through variable inside application set itself.

for example in some-values.yaml values: a: b c:

• 1
• 2

• 2

  then in application set just use values: {{values}}

@jgwest is someone working on it ?

[mksha]

let me know if i can help

[andrejpetras]

@marcozov is working for me. The downside is that you can not do this: {{ parent.value.attribute }}. You have only {{ parent }} which represents string.

@mksha I like go template but I am no sure if it is the right solution for this case, for example it could be in conflict with helm. But I agree we need a more powerful template engine.

So I convert the yaml substructure back to a string: https://github.com/lorislab/applicationset/blob/yaml/pkg/generators/git.go#L266

@andrejpetras that part is for passing those values as a string to the field values of the application, right? Why don't you like the current approach of your PoC? I'm not 100% sure I understand the other approaches that you are suggesting.

[pkrishnath]

I solved as

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
 name: apm-server
spec:
 generators:
 - list:
     elements:
     - cluster: mycluster1
       url: https://url1
       values:
         override: |
            <your values>
     - cluster: mycluster2
       url: https://url2
       values:
          override: |
              <your values>
 template:
   metadata:
     name: '{{cluster}}-apm-server'
   spec:
     project: default
     source:
       helm:
         values: | 
              {{values.override}}
         releaseName: apm-server
       path: apm-server
       repoURL: https://mycustomrepo/helm.git

[diogokiss]

The changes in #211 seem to work if you specify a specific key.

Do we have an eta for being able to do something like below with the values when we need more flexibility?

    helm:
      releaseName: '{{path.basename}}'
      values: |
        {{values}}

Any updates about this being feasible?

[zbialik]

I agree this would be really nice. I think a main issue is that these parameters all get flattened into string values. If we could somehow pipe the string to an array/list/dict OR just handle the parameters as these type of objects directly, that'd be great.

[ValentinoUberti]

Hi, is there any update?

[vl-kp]

any update?