andrii-korotkov-verkada
commented
2 days ago ArgoCD versions 2.10 and below have reached EOL. Can you upgrade and let us know if the issue is still present, please?
Open jessebot opened 11 months ago
andrii-korotkov-verkada
commented
2 days ago ArgoCD versions 2.10 and below have reached EOL. Can you upgrade and let us know if the issue is still present, please?
Checklist:
argocd version.Describe the bug
Before we get started, I should mention my main goal is to have proper syntax highlighting on the values override values.
I can't use a
valuesObjectdue to https://github.com/argoproj/argo-cd/issues/14912, and in https://github.com/argoproj/argo-cd/issues/9583#issuecomment-1826828249 it was kindly suggested to try out the multiple sources route.I cannot template use a generator to template anything inside a values.yaml file override done via the multiple sources method.
To Reproduce Here's my ApplicationSet for mastodon:
Above I'm using a generator that just pulls from a secret, but I'm happy to put together an example with a list generator to be more clear too. The reason the generator is important is because I have values I want to override based on an existing Kubernetes secret. The values override file contains this a few parts where
mastodon_hostnameis supposed to be rendered:click me for the full
```yaml image: repository: ghcr.io/mastodon/mastodon pullPolicy: IfNotPresent mastodon: createAdmin: enabled: false existingSecret: mastodon-admin-credentials secretKeys: usernameKey: username passwordKey: password emailKey: email cron: # -- run `tootctl media remove` every week removeMedia: enabled: true schedule: "0 0 * * 0" # -- available locales: https://github.com/mastodon/mastodon/blob/main/config/application.rb#L71 locale: en local_domain: {{ .mastodon_hostname }} # -- Use of WEB_DOMAIN requires careful consideration: https://docs.joinmastodon.org/admin/config/#federation # You must redirect the path LOCAL_DOMAIN/.well-known/ to WEB_DOMAIN/.well-known/ as described # Example: mastodon.example.com web_domain: null # -- If set to true, the frontpage of your Mastodon server will always redirect to the first profile in the database and registrations will be disabled. singleUserMode: false # -- Enables "Secure Mode" for more details see: https://docs.joinmastodon.org/admin/config/#authorized_fetch authorizedFetch: false # -- Enables "Limited Federation Mode" for more detauls see: https://docs.joinmastodon.org/admin/config/#limited_federation_mode limitedFederationMode: false extraVolumes: - name: postgres-ca secret: secretName: mastodon-postgres-server-ca-key-pair defaultMode: 0440 - name: postgres-client-certs secret: secretName: mastodon-postgres-mastodon-cert defaultMode: 0440 extraVolumeMounts: - name: postgres-ca mountPath: /etc/secrets/ca - name: postgres-client-certs mountPath: /etc/secrets/mastodon s3: enabled: true existingSecret: "mastodon-s3-credentials" hostname: {{ .mastodon_s3_endpoint }} secretKeys: s3AccessKeyID: S3_USER s3AccessKey: S3_PASSWORD s3Bucket: BUCKET s3Endpoint: ENDPOINT s3Hostname: HOSTNAME secrets: # these must be set manually; autogenerated keys are rotated on each upgrade existingSecret: "mastodon-server-secrets" sidekiq: workers: - name: all-queues # -- Number of threads / parallel sidekiq jobs that are executed per Pod concurrency: 25 # -- Sidekiq queues for Mastodon that are handled by this worker. See https://docs.joinmastodon.org/admin/scaling/#concurrency # See https://github.com/mperham/sidekiq/wiki/Advanced-Options#queues for how to weight queues as argument queues: - default,8 - push,6 - ingress,4 - mailers,2 - pull # Make sure the scheduler queue only exists once and with a worker that has 1 replica. - scheduler smtp: auth_method: login ca_file: /etc/ssl/certs/ca-certificates.crt delivery_method: smtp domain: {{ .mastodon_hostname }} enable_starttls: 'auto' from_address: toots@{{ .mastodon_hostname }} openssl_verify_mode: peer port: 587 reply_to: no-reply@{{ .mastodon_hostname }} tls: true # keys must be named `server`, `login`, `password` existingSecret: mastodon-smtp-credentials streaming: port: 4000 # -- this should be set manually since os.cpus() returns the number of CPUs on # the node running the pod, which is unrelated to the resources allocated to # the pod by k8s workers: 1 # -- The base url for streaming can be set if the streaming API is deployed to # a different domain/subdomain. base_url: null # -- Number of Streaming Pods running replicas: 1 web: port: 3000 # -- Number of Web Pods running replicas: 1 minThreads: "5" maxThreads: "5" workers: "2" persistentTimeout: "20" metrics: statsd: # -- Enable statsd publishing via STATSD_ADDR environment variable address: "" # Sets the PREPARED_STATEMENTS environment variable: https://docs.joinmastodon.org/admin/config/#prepared_statements preparedStatements: true ingress: enabled: true annotations: kubernetes.io/tls-acme: "true" cert-manager.io/cluster-issuer: {{ .global_cluster_issuer }} # ensure that NGINX's upload size matches Mastodon's nginx.ingress.kubernetes.io/proxy-body-size: 40m ingressClassName: nginx hosts: - host: {{ .mastodon_hostname }} paths: - path: '/' tls: - secretName: mastodon-tls hosts: - {{ .mastodon_hostname }} # https://github.com/bitnami/charts/tree/main/bitnami/elasticsearch#parameters elasticsearch: # `false` will disable full-text search # if you enable ES after the initial install, you will need to manually run # RAILS_ENV=production bundle exec rake chewy:sync # (https://docs.joinmastodon.org/admin/optional/elasticsearch/) enabled: true master: replicaCount: 1 autoscaling: minReplicas: 1 data: replicaCount: 1 coordinating: replicaCount: 1 ingest: replicaCount: 1 externalDatabase: enabled: true hostname: mastodon-postgres-rw.mastodon.svc port: "5432" database: mastodon user: mastodon existingSecret: "mastodon-pgsql-credentials" sslmode: "verify-full" sslcert: "/etc/secrets/mastodon/tls.crt" sslkey: "/etc/secrets/mastodon/tls.key" sslrootcert: "/etc/secrets/ca/ca.crt" # https://github.com/bitnami/charts/tree/main/bitnami/postgresql#parameters postgresql: enabled: false # https://github.com/bitnami/charts/tree/main/bitnami/redis#parameters redis: enabled: false hostname: "mastodon-redis-master" port: 6379 auth: # with a key of redis-password set to the password you want existingSecret: "mastodon-redis-credentials" service: type: ClusterIP port: 80 externalAuth: oidc: enabled: false oauth_global: # -- Automatically redirect to OIDC, CAS or SAML, and don't use local account authentication when clicking on Sign-In omniauth_only: false # -- https://github.com/mastodon/mastodon/blob/main/Dockerfile#L75 # if you manually change the UID/GID environment variables, ensure these values match: podSecurityContext: runAsUser: 991 runAsGroup: 991 fsGroup: 991 securityContext: {} serviceAccount: # -- Specifies whether a service account should be created create: true # -- Annotations to add to the service account annotations: {} ```values-override.yamlexampleThis is the error I get when using the above combination of an ApplicationSet referencing a remote published helm chart and then a values.yaml living in the same directory as the ApplicationSet:
To be clear, ApplicationSet generators of any kind work with inline yaml via
helm.values. Here's an example working ApplicationSet:Click me for ApplicationSet using inline yaml in helm.values
```yaml --- # third sync wave because it has to be up after postgres apiVersion: argoproj.io/v1alpha1 kind: ApplicationSet metadata: name: mastodon-app-set namespace: argocd spec: # enable go templating goTemplate: true # this generator allows us to values from an external k8s secret generators: - plugin: configMapRef: name: secret-var-plugin-generator input: parameters: secret_vars: - mastodon_hostname - mastodon_s3_endpoint - global_cluster_issuer template: metadata: name: mastodon-web-app annotations: argocd.argoproj.io/sync-wave: "3" argocd.argoproj.io/sync-options: ApplyOnly=true spec: project: mastodon destination: server: https://kubernetes.default.svc namespace: mastodon syncPolicy: syncOptions: - ApplyOutOfSyncOnly=true source: # forked from: https://github.com/mastodon/chart/tree/main repoURL: 'https://small-hack.github.io/mastodon-helm-chart' targetRevision: 5.0.0 chart: mastodon helm: values: | image: repository: ghcr.io/mastodon/mastodon pullPolicy: IfNotPresent mastodon: createAdmin: enabled: false existingSecret: mastodon-admin-credentials secretKeys: usernameKey: username passwordKey: password emailKey: email cron: # -- run `tootctl media remove` every week removeMedia: enabled: true schedule: "0 0 * * 0" # -- available locales: https://github.com/mastodon/mastodon/blob/main/config/application.rb#L71 locale: en local_domain: {{ .mastodon_hostname }} # -- Use of WEB_DOMAIN requires careful consideration: https://docs.joinmastodon.org/admin/config/#federation # You must redirect the path LOCAL_DOMAIN/.well-known/ to WEB_DOMAIN/.well-known/ as described # Example: mastodon.example.com web_domain: null # -- If set to true, the frontpage of your Mastodon server will always redirect to the first profile in the database and registrations will be disabled. singleUserMode: false # -- Enables "Secure Mode" for more details see: https://docs.joinmastodon.org/admin/config/#authorized_fetch authorizedFetch: false # -- Enables "Limited Federation Mode" for more detauls see: https://docs.joinmastodon.org/admin/config/#limited_federation_mode limitedFederationMode: false extraVolumes: - name: postgres-ca secret: secretName: mastodon-postgres-server-ca-key-pair defaultMode: 0440 - name: postgres-client-certs secret: secretName: mastodon-postgres-mastodon-cert defaultMode: 0440 extraVolumeMounts: - name: postgres-ca mountPath: /etc/secrets/ca - name: postgres-client-certs mountPath: /etc/secrets/mastodon s3: enabled: true existingSecret: "mastodon-s3-credentials" hostname: {{ .mastodon_s3_endpoint }} secretKeys: s3AccessKeyID: S3_USER s3AccessKey: S3_PASSWORD s3Bucket: BUCKET s3Endpoint: ENDPOINT s3Hostname: HOSTNAME secrets: # these must be set manually; autogenerated keys are rotated on each upgrade existingSecret: "mastodon-server-secrets" sidekiq: workers: - name: all-queues # -- Number of threads / parallel sidekiq jobs that are executed per Pod concurrency: 25 # -- Sidekiq queues for Mastodon that are handled by this worker. See https://docs.joinmastodon.org/admin/scaling/#concurrency # See https://github.com/mperham/sidekiq/wiki/Advanced-Options#queues for how to weight queues as argument queues: - default,8 - push,6 - ingress,4 - mailers,2 - pull # Make sure the scheduler queue only exists once and with a worker that has 1 replica. - scheduler smtp: auth_method: login ca_file: /etc/ssl/certs/ca-certificates.crt delivery_method: smtp domain: {{ .mastodon_hostname }} enable_starttls: 'auto' from_address: toots@{{ .mastodon_hostname }} openssl_verify_mode: peer port: 587 reply_to: no-reply@{{ .mastodon_hostname }} tls: true # keys must be named `server`, `login`, `password` existingSecret: mastodon-smtp-credentials streaming: port: 4000 # -- this should be set manually since os.cpus() returns the number of CPUs on # the node running the pod, which is unrelated to the resources allocated to # the pod by k8s workers: 1 # -- The base url for streaming can be set if the streaming API is deployed to # a different domain/subdomain. base_url: null # -- Number of Streaming Pods running replicas: 1 web: port: 3000 # -- Number of Web Pods running replicas: 1 minThreads: "5" maxThreads: "5" workers: "2" persistentTimeout: "20" metrics: statsd: # -- Enable statsd publishing via STATSD_ADDR environment variable address: "" # Sets the PREPARED_STATEMENTS environment variable: https://docs.joinmastodon.org/admin/config/#prepared_statements preparedStatements: true ingress: enabled: true annotations: kubernetes.io/tls-acme: "true" cert-manager.io/cluster-issuer: {{ .global_cluster_issuer }} # ensure that NGINX's upload size matches Mastodon's nginx.ingress.kubernetes.io/proxy-body-size: 40m ingressClassName: nginx hosts: - host: {{ .mastodon_hostname }} paths: - path: '/' tls: - secretName: mastodon-tls hosts: - {{ .mastodon_hostname }} # https://github.com/bitnami/charts/tree/main/bitnami/elasticsearch#parameters elasticsearch: # `false` will disable full-text search # if you enable ES after the initial install, you will need to manually run # RAILS_ENV=production bundle exec rake chewy:sync # (https://docs.joinmastodon.org/admin/optional/elasticsearch/) enabled: true master: replicaCount: 1 autoscaling: minReplicas: 1 data: replicaCount: 1 coordinating: replicaCount: 1 ingest: replicaCount: 1 externalDatabase: enabled: true hostname: mastodon-postgres-rw.mastodon.svc port: "5432" database: mastodon user: mastodon existingSecret: "mastodon-pgsql-credentials" sslmode: "verify-full" sslcert: "/etc/secrets/mastodon/tls.crt" sslkey: "/etc/secrets/mastodon/tls.key" sslrootcert: "/etc/secrets/ca/ca.crt" # https://github.com/bitnami/charts/tree/main/bitnami/postgresql#parameters postgresql: enabled: false # https://github.com/bitnami/charts/tree/main/bitnami/redis#parameters redis: enabled: false hostname: "mastodon-redis-master" port: 6379 auth: # with a key of redis-password set to the password you want existingSecret: "mastodon-redis-credentials" service: type: ClusterIP port: 80 externalAuth: oidc: enabled: false oauth_global: # -- Automatically redirect to OIDC, CAS or SAML, and don't use local account authentication when clicking on Sign-In omniauth_only: false # -- https://github.com/mastodon/mastodon/blob/main/Dockerfile#L75 # if you manually change the UID/GID environment variables, ensure these values match: podSecurityContext: runAsUser: 991 runAsGroup: 991 fsGroup: 991 securityContext: {} serviceAccount: # -- Specifies whether a service account should be created create: true # -- Annotations to add to the service account annotations: {} ```The only issue with that is that in most IDEs, it will show up as a string, and not yaml, and so the syntax highlighting is off. See Screenshots for example.
I just can't use
helm.valuesObjectorhelm.valuesFiles(with an override values.yaml in a different repo from the helm chart), and wasn't sure how to move forward.Expected behavior
I'd expect the templated values from the generator to be rendered in the values-override.yaml and for it to work the same as in-line yaml overriding.
Screenshots
Example of the error when using multiple sources:
Example of the in-line yaml not rendering properly (this also breaks the LSP's ability to follow where you are, so if your cursor is anywhere under values, it just shows at the top as being one giant values string type, so if you need to get really nested, this can make troubleshooting hard):
The reason I keep pressing this is because I'm dyslexic and syntax highlighting really helps me read.
Version Local CLI version installed via linuxbrew.
If it is helpful, Argo CD is installed via k3s on metal via an Argo CD ApplicationSet: https://github.com/small-hack/argocd-apps/blob/eso-helm-chart-test/argocd/argocd_appset.yaml
Logs
I'm not sure which logs to check, but I am happy to provide them if you let me know. It's not failing in a massive way like the other issue, but again, very happy to assist in any way.