Helm resource generated by helper function not rendered or created

[irishandyb]

I am having a problem in which resources templated by a Helm _helper function are not being rendered or generated when deploying the chart via argo.

I have the following Helm helper function defined in _cert.tpl. It serves to generate a cert-manager Certificate resource based on provided inputs.

{{ define "myapp.cert" }}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: {{ .name }}
  annotations:
    helm.sh/hook: post-install
    helm.sh/hook-weight: "1"
spec:
  secretName: {{ .name }}
  duration: {{ .duration | default "2160h" }}
  renewBefore: {{ .renewBefore | default "360h" }}
  dnsNames:
  {{- range .dnsNames }}
  - {{ . -}}
  {{ end }}
  issuerRef:
{{ toYaml .issuerRef | indent 4 }}
{{ end }}

{{ define "myapp.cert-prefix" }}
{{- default (include "myapp.fullname" .) .prefix -}}
{{ end }}

This function is invoked in the tls.yaml template file as follows:

{{ if .Values.tls.certificates }}
  {{ range .Values.tls.certificates }}

    {{ $prefix := default (include "myapp.fullname" $) .prefix  }}

    {{ $certName := (printf "%s-cert-%s" $prefix .name ) }}
    {{ $issuerRef :=  .issuerRef | default $.Values.tls.defaultIssuerRef }}
    {{ $cert := dict "name" $certName "issuerRef" $issuerRef "dnsNames" .dnsNames "duration" .duration "renewBefore" .renewBefore }}
    {{ include "myapp.cert" $cert }}
  {{ end }}
{{ end }}

This is then combined with the following values.yaml:

tls:
  certificates:
  - dnsNames:
    - this-is-a-test
    issuerRef: {}
    name: tls
    prefix: this-is-a-test

When rendered via helm template the following Certificate resource is generated as expected. It is also created if I deploy the Helm chart via the Helm CLI.

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: this-is-a-test-cert-tls
  annotations:
    helm.sh/hook: post-install
    helm.sh/hook-weight: "1"
spec:
  secretName: this-is-a-test-cert-tls
  duration: 2160h
  renewBefore: 360h
  dnsNames:
  - this-is-a-test
  issuerRef:
    kind: ClusterIssuer
    name: default-local-ca

However when I deploy the same Helm chart via argocd, the Certificate resource is not created or deployed. It's simply not there; neither as a Kubernetes resource or an item listed on the application tree view in argo. The argo logs for the application eventually show as degraded as one of the pods will not start without this particular cert.. but no error or notice of the certificate resource itself.

The Application definition is as follows:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  finalizers:
  - resources-finalizer.argocd.argoproj.io
  labels:
    name: myapp
  name: myapp
  namespace: argocd
spec:
  destination:
    namespace: myapp
    server: https://kubernetes.default.svc
  ignoreDifferences:
  - group: apps
    jsonPointers:
    - /spec/replicas
    kind: Deployment
  - group: '*'
    kind: '*'
    managedFieldsManagers:
    - kube-controller-manager
  project: myapp
  source:
    chart: myapp
    helm:
      fileParameters: []
      releaseName: myapp
      valueFiles: []
      values: |
        tls:
          certificates:
          - dnsNames:
            - this-is-a-test
            issuerRef: {}
            name: tls
            prefix: this-is-a-test
      version: v3
    repoURL: https://harbor.local/chartrepo/servicekit
    targetRevision: 1.0.0
  syncPolicy:
    automated:
      allowEmpty: false
      prune: true
      selfHeal: true
    retry:
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 3m
      limit: 5
    syncOptions:
    - Validate=false
    - CreateNamespace=true
    - PrunePropagationPolicy=foreground
    - PruneLast=true

There are no logs in cert-manager to indicate any issues with admission of a certificate resource. Note that I do also have another hard-coded Certificate (i.e., not generated by a template function) in the Chart and this is created as expected.. so the issue appears to be limited to this helper function.

I have deployed argocd via Helm as below:

$ helm -n argocd ls
NAME    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION
argocd  argocd          1               2022-07-28 19:32:18.314273585 +0000 UTC deployed        argo-cd-4.9.16  v2.4.7

Any input as to whether this may be a bug or user error would be greatly appreciated.

Andy

[irishandyb]

I have made a little progress. The root of the issue is the Helm post-install hook attached to the Certificate resource.

    helm.sh/hook: post-install
    helm.sh/hook-weight: "1"

This is present as cert-manager is included as a dependency/requirement for my application chart. With this, it is required that cert-manager complete its installation and its CRDs be available before Helm creates the Certificate resource.

There seems to be a circular dependency here in which Helm does not execute the post hook to create the Certificate because there is a Pod holding up the install by depending on the Secret created for the Certificate.

What is very confusing is that I do not encounter this issue during a manual helm install of the same chart with the same values.

[smartbit]

Have you tried sync-wave to ensure cert-manager is deployed first? This is a snippet from our deployment of cert-manager:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: cert-manager
  namespace: argocd
  annotations:
    argocd.argoproj.io/sync-wave: "-1"