Ability to read additional environment variables within the argocd-repo-server

[shanproofpoint]

Summary

When specifying an Application yaml, we would like the argocd build envirionment to include all os level environment variables or can be restricted to any custom os environment variable prefixed with ARGOCD_

Motivation

we are setting up many clusters that poll a single git repo for their entire list of applications and their configuration. however, each cluster has an identity that is used in parameterizing some deployments and custom resources either through additional annotations, environment variables or other resources like disks. we store this as a configmap of values inside the argocd namespace which then can me mounted in the argocd-repo-server as environment variables.

Proposal

just read additional os level environment variables into the argocd build environment so we can do the following:

kind: Application
metadata:
  name: argocd-app-sops
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://isindir.github.io/sops-secrets-operator/
    chart: sops-secrets-operator
    targetRevision: 0.9.2
    helm:
      parameters:
        - name: "gcp.enabled"
          value: "true"
        - name: "gcp.existingSecretName"
          value: "sops-decrypter-access"
        - name: "podAnnotations.i-was-here/ok"
          value: $ARGOCD_MY_CUSTOM_ENV_VAR
        - name: "podAnnotations.i-was-here2/ok"
          value: $ARGOCD_APP_NAME

where ARGOCD_MY_CUSTOM_ENV_VAR would be read from a configmap into the argocd-repo-server container

[Jmainguy]

We are currently using a plugin to get these vars, but that stops us from using the helm: block. So this would be a great feature to add.

[crenshaw-dev]

I think this makes sense. Rather than prefixing with ARGOCD_ (there may be collisions with Argo CD env vars added in the future), maybe there could be an allow-list configurable via repo-server ConfigMap or env var?

[mstrYoda]

Hi, I would like to work on this issue. Can you assign it to me @crenshaw-dev ? Thanks.

[shanproofpoint]

Hi, I would like to work on this issue. Can you assign it to me @crenshaw-dev ? Thanks.

thank you!

[jselleck-sans]

I'd like this too! In my case I want to override application placeholders with global (or config map) env vars. Our use case is gitops - we want to use the same manifests but pull from different branches depending on a cluster name variable (changing branch Revision). I also see us setting parameters in apps for helm charts - dependent on global params..

[jselleck-sans]

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: guestbook
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/argoproj/argocd-example-apps.git
    targetRevision: $MY_CUSTOM_ENV_VAR
    path: guestbook
  destination:
    server: https://kubernetes.default.svc
    namespace: guestbook

• maybe add an annotation to the Application manifest to point to a CM for variables.

[SebastienTolron]

+1 Would be so good !

Trying to achieve the same goal. Having multiple cluster and 1 applicationSet. So , for ingress route, it would be pretty handfull.

[mstrYoda]

I finally find time to work on this. I will schedule it to develop with our Golang community 🙏

[shanproofpoint]

Thank you!

[revaspeho]

Hi I am the head DevOps Engineer for the federal government. We need to be able to pass in our gitlab auth token into the url string for helm.valueFiles -https://$AUTH_TOKEN. This would be great!

[danhawkins]

👍 for this, I am trying to have a dynamic targetRevision also

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: letsencrypt
  namespace: argocd
spec:
  project: core-services
  source:
    path: services/letsencrypt/manifests
    repoURL: <repo>
    targetRevision: $TARGET_REVISION

[krzwiatrzyk]

@revaspeho theoretically you can achieve it with helm-secrets plugin - https://github.com/argoproj/argo-cd/issues/7819 practically, environment substation doesn't work for me (but still trying to make it work)

[danhawkins]

@mstrYoda are you actively working on this?

[danhawkins]

I was able to get something to work here, I built my own docker image, removing this line from applicationset utils to allow access to the env function. Then using and applicationset with a go template

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: letsencrypt
  namespace: argocd
spec:
  goTemplate: true
  generators:
    - clusters: {}
  template:
    metadata:
      name: '{{.name}}-letsencrypt'
    spec:
      project: core-services
      source:
        path: services/letsencrypt/manifests
        repoURL: https://github.com/xxxxxxxxxxxx/argolab.git
        targetRevision: '{{ env "TARGET_REVISION" }}'
        directory:
          recurse: true
      destination:
        server: https://kubernetes.default.svc
        namespace: cert-manager
      syncPolicy:
        syncOptions:
          - CreateNamespace=true
        automated:
          selfHeal: true

This works, but obviously there is some security concern. But I think if we are able to set an allowlist of vars somewhere like suggested earlier on, we can perhaps just filter out env values not on the list? Something like that. I mean I am sure there could be more elegant ways to solve this. But I am happy with this so far.

[danhawkins]

I have a PR that solves this, will submit it soon for review

[shanproofpoint]

Thank you ! @danhawkins

[danhawkins]

@crenshaw-dev you can assign this to me, I made a PR, any feedback to get it merged would be appreciated.

[leoluz]

As mentioned in Contributor's meeting today, we aim to address this issue by the functionality implemented in this PR: https://github.com/argoproj/argo-cd/pull/12508

[pierluigilenoci]

@leoluz I looked at PR #12508 and needed help understanding how that can solve the problem proposed in this issue. @shanproofpoint, what do you think about it?

[shanproofpoint]

@leoluz i also feel it does not directly address this need. We need something like a parameterized kustomization.yaml file each cluster that reads the same application yaml. Can you explain how multiple repos get merged?

[leoluz]

@shanproofpoint @pierluigilenoci With the functionality implemented in the multi-source PR a new concept of chained manifest generation will be possible. For example, this means that you will be able to rely on a helm chart to generate your basic manifests and then implement a small CMP plugin that does the last mile manifest manipulation required by your company.

[shanproofpoint]

Fantastic...

Shan Tang

Staff Engineer

Proofpoint Global Points of Presence

---

From: Leonardo Luz Almeida @.> Sent: Tuesday, May 23, 2023 11:47 AM To: argoproj/argo-cd @.> Cc: Shan Tang @.>; Mention @.> Subject: Re: [argoproj/argo-cd] Ability to read additional environment variables within the argocd-repo-server (#7189)

@shanproofpoint @pierluigilenoci With the functionality implemented in the multi-source PR a new concept of chained manifest generation will be possible. For example, this means that you will be able to rely on a helm chart to generate your

@shanproofpointhttps://urldefense.com/v3/__https://github.com/shanproofpoint__;!!ORgEfCBsr282Fw!uEF6M75yEPur8tGe6OBKFjDoqJDyHolPHRbRHGz_VW-QirJGMduvGt_jp8kmqGQLH_W_jY_BJLP6s_UeC-sua-wF$ @pierluigilenocihttps://urldefense.com/v3/__https://github.com/pierluigilenoci__;!!ORgEfCBsr282Fw!uEF6M75yEPur8tGe6OBKFjDoqJDyHolPHRbRHGz_VW-QirJGMduvGt_jp8kmqGQLH_W_jY_BJLP6s_UeC2SWmjEy$ With the functionality implemented in the multi-source PRhttps://urldefense.com/v3/__https://github.com/argoproj/argo-cd/pull/12508__;!!ORgEfCBsr282Fw!uEF6M75yEPur8tGe6OBKFjDoqJDyHolPHRbRHGz_VW-QirJGMduvGt_jp8kmqGQLH_W_jY_BJLP6s_UeC_QXI91x$ a new concept of chained manifest generation will be possible. For example, this means that you will be able to rely on a helm chart to generate your basic manifests and then implement a small CMP plugin that does the last mile manifest manipulation required by your company.

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/argoproj/argo-cd/issues/7189*issuecomment-1559963888__;Iw!!ORgEfCBsr282Fw!uEF6M75yEPur8tGe6OBKFjDoqJDyHolPHRbRHGz_VW-QirJGMduvGt_jp8kmqGQLH_W_jY_BJLP6s_UeC68jjRX2$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AMB7ACI64J5I46FE2ZUBQJ3XHUA3HANCNFSM5DXZU7MA__;!!ORgEfCBsr282Fw!uEF6M75yEPur8tGe6OBKFjDoqJDyHolPHRbRHGz_VW-QirJGMduvGt_jp8kmqGQLH_W_jY_BJLP6s_UeC8pMyxfZ$. You are receiving this because you were mentioned.Message ID: @.***>

[krzwiatrzyk-lgd]

@leoluz I think that chain manifest generation or CMP is an overkill for such need.

Showing another example: I am managing a fleet of EKS cluster and want to manage "infrastracture" apps via app-of-apps pattern. ArgoCD is deployed once on the cluster create and right away app-of-apps manifest pointing to infrastructure manfiests is deployed.

Infrastructure manifests contains an application manifest for cluster autoscaler which requires an unique clusterName to distinguish which autoscaling groups can be targetted for autoscaling.

Having environment substition in application manifest this will be very straightforward:

1. Set environment on ArgoCD deployment
2. Specify environment in application manifest.

Environment variables substitution is already there... but only for a small set of defined variables.

[AndreCbrera]

may can you try envsubst '$ARGOCD_APP_NAME' < application.yaml | k apply -f - declaring before the variable export ARGOCD_APP_NAME="hellotherepoc" all this in argocd namespace

[maeghan-porter]

I'm looking in to replacing fluxcd with argocd and I'm struggling with argocd not having the same idea of postBuild variable substitution that flux has. We have multiple multi-tenancy eks clusters for different environments (dev, beta, stable nonprod, stable prod), and we use variables such as the name of the cluster (needed for aws-specific helm charts), the environment, the stage (nonprod/prod), the vpc id the cluster is in, etc. via postBuild variables with flux. It would be nice if, in argocd, we could set some variables like this on the cluster objects and they could be rendered in the values for a helmchart or in kustomizations depending on which cluster it's being deployed to.

[AndreCbrera]

@maeghan-porter may you can use some approach like envsubst command, I do this using helm values:

envsubst '$ARGOCD_APP_NAME' < values.yaml | helm install app-test .

[maeghan-porter]

@maeghan-porter may you can use some approach like envsubst command, I do this using helm values:

envsubst '$ARGOCD_APP_NAME' < values.yaml | helm install app-test .

@AndreCbrera It is not clear to me how that would work with argo? I'm new to this so maybe I'm missing something... Edit: your exact example confused me, but I see what you're suggesting and I can certainly come up with some such work around (though I don't think this would work, given my example, for sharing the same application.yaml for multiple clusters), but my comment is here to back the concept of having custom environment variables and my example is demonstrating where such a thing would be useful. Thank you for your suggestion :)