Helm-hooks/Wrong behavior of argocd to deploy helm chart with hooks

[galsasi1989]

Checklist:

• [x] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
• [x] I've included steps to reproduce the bug.
• [x] I've pasted the output of argocd version.

Describe the bug

I am trying to deploy helm chart which contains a job with helm hook annotation: "helm.sh/hook: post-install,post-upgrade" When helm is used, the job is deployed successfully with the rest of chart's resources. But when the chart is deployed by argocd, the job is ignored. According to argocd documentation(https://argo-cd.readthedocs.io/en/stable/user-guide/helm/#helm-hooks) is seems like this helm annotation is translated into argocd hook annotation: "argocd.argoproj.io/hook: PostSync". According to the documentation about argocd PostSync hook(https://argo-cd.readthedocs.io/en/stable/user-guide/resource_hooks/) it seems like a different behavior: "Using a PostSync hook to run integration and health checks after a deployment.". So the behavior when the chart is deployed by argocd is different from the behavior when it's deployed by helm. The only solution I found is the use argo hook annotations instead those of helm but it's less preferred.

To Reproduce

Expected behavior

The job hook will be triggered by argo with the application workloads as done by helm when using the post-install,post-upgrade annotation.

Screenshots

apiVersion: batch/v1
kind: Job
metadata:
  annotations:
    helm.sh/hook: post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation
  labels:
    app: app-db-migration
  name: app-db-migration
  namespace: staging
spec:
  template:
    metadata:
      labels:
        app: app-db-migration
        cdk8s.io/metadata.addr: appChart-dbMigrationJob-job-c88dfdff
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: role
                    operator: In
                    values:
                      - app
      automountServiceAccountToken: true
      containers:
        - command:
            - python
            - manage.py
            - migrate
          env:
           // list of env vars
          image: <docker image path>
          imagePullPolicy: Always
          name: app
          resources:
            limits:
              cpu: "2"
              memory: 2048Mi
            requests:
              cpu: "2"
              memory: 2048Mi
          securityContext:
            allowPrivilegeEscalation: false
            privileged: false
            readOnlyRootFilesystem: true
            runAsNonRoot: false
          volumeMounts:
            - mountPath: /mnt/secrets-store
              name: app-secret
              readOnly: true
          workingDir: /app
      dnsPolicy: ClusterFirst
      hostNetwork: false
      restartPolicy: Never
      securityContext:
        fsGroupChangePolicy: Always
        runAsNonRoot: false
      serviceAccountName: app-sa
      setHostnameAsFQDN: false
      terminationGracePeriodSeconds: 30
      volumes:
        - csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: app-secret-spc
          name: app-secret
  ttlSecondsAfterFinished: 7200

Version

argocd: v2.8.4+c279299.dirty
  BuildDate: 2023-09-13T22:00:14Z
  GitCommit: c27929928104dc37b937764baf65f38b78930e59
  GitTreeState: dirty
  GoVersion: go1.21.1
  Compiler: gc
  Platform: darwin/amd64
argocd-server: v2.9.5+f943664
  BuildDate: 2024-01-19T17:57:57Z
  GitCommit: f9436641a616d277ab1f98694e5ce4c986d4ea05
  GitTreeState: clean
  GoVersion: go1.21.3
  Compiler: gc
  Platform: linux/amd64
  Kustomize Version: v5.2.1 2023-10-19T20:13:51Z
  Helm Version: v3.13.2+g2a2fb3b
  Kubectl Version: v0.24.17
  Jsonnet Version: v0.20.0

Logs

Paste any relevant application logs here.

[cardoe]

Agreed here. The post-install,post-upgrade Helm hook should really map to the Sync ArgoCD hook. In Helm the post-install,post-upgrade happens after all the resources have been applied but not before they're up and healthy. This maps to the ArgoCD Sync hook instead of the ArgoCD PostSync hook which happens after the resources are applied and healthy.

[andrii-korotkov-verkada]

ArgoCD versions 2.10 and below have reached EOL. Can you upgrade and tell us if the issue is still present, please?