search

argoproj / argo-cd

Declarative Continuous Deployment for Kubernetes
https://argo-cd.readthedocs.io
Apache License 2.0
17.83k stars 5.44k forks source link

Hooks precedence: PreSync namespaces "xxxx" not found #3762

Open slayer opened 4 years ago

slayer commented 4 years ago

Describe the bug

I have few apps (helm charts) combined by one helm chart. Some of child charts have helm hooks for db init/migrations In top level chart I created ns template:

---
apiVersion: v1
kind: Namespace
metadata:
  name: {{ .Release.Namespace }}
  annotations:
    argocd.argoproj.io/sync-wave: "-100"

My problem that child helm hooks executed before namespace creation:

GROUP                  KIND            NAMESPACE    NAME                    STATUS     HEALTH      HOOK     MESSAGE
batch                  Job             review-app1  xx-db-init              Failed     SyncFailed  PreSync  namespaces "review-app1" not found
batch                  Job             review-app1  yy-db-init              Failed     SyncFailed  PreSync  namespaces "review-app1" not found
batch                  Job             review-app1  zz-db-init              Failed     SyncFailed  PreSync  namespaces "review-app1" not found
                       ConfigMap       review-app1  xx-config               OutOfSync  Missing               
                       ConfigMap       review-app1  yy-config               OutOfSync  Missing

even with "-100" sync wave priority

I tried add to ns definition

argocd.argoproj.io/hook: PreSync

It works well only for first deployment, all next fails with:

 GROUP                  KIND            NAMESPACE      NAME                  STATUS     HEALTH   HOOK     MESSAGE
                        Namespace       review-app1    review-app1           Error               PreSync  failed to delete resource: Operation cannot be fulfilled on namespaces "review-app1": The system is ensuring all content is removed from this namespace.  Upon completion, this namespace will automatically be purged by the system.

To Reproduce

Add child helm charts with hooks like:

apiVersion: batch/v1
kind: Job
metadata:
  name: xx-db-init
  annotations:
    "helm.sh/hook": pre-upgrade,pre-install
    "helm.sh/hook-delete-policy": before-hook-creation 
    "helm.sh/hook-weight": "10"

Expected behavior

Namespace should be created before helm hooks

Version

argocd: v1.4.3+2483411
  BuildDate: 2020-04-15T16:48:21Z
  GitCommit: 24834112d66f453de6ae53901d5dad0ad71ec902
  GitTreeState: clean
  GoVersion: go1.12.6
  Compiler: gc
  Platform: linux/amd64
argocd-server: v1.6.0-rc1+b39e93a
  BuildDate: 2020-06-02T21:47:11Z
  GitCommit: b39e93a4d24894248f876d9649636781e21f5cc6
  GitTreeState: clean
  GoVersion: go1.14.1
  Compiler: gc
  Platform: linux/amd64
  Ksonnet Version: v0.13.1
  Kustomize Version: {Version:kustomize/v3.6.1 GitCommit:c97fa946d576eb6ed559f17f2ac43b3b5a8d5dbd BuildDate:2020-05-27T20:47:35Z GoOs:linux GoArch:amd64}
  Helm Version: version.BuildInfo{Version:"v3.2.0", GitCommit:"e11b7ce3b12db2941e90399e874513fbd24bcb71", GitTreeState:"clean", GoVersion:"go1.13.10"}
  Kubectl Version: v1.14.0
slayer commented 4 years ago

As workaround I removed Namespace definition and run

 echo "apiVersion: v1\nkind: Namespace\nmetadata:\n  name: review-app1" | kubectl apply -f -

before argocd deploy

But that looks ugly

k commented 4 years ago

I have this same problem, currently I am creating the namespace manually beforehand but it really needs to be a feature in ArgoCD. There is discussion here: https://github.com/argoproj/argo-cd/issues/1809

dansou901 commented 3 years ago

You can create the namespace the way you want to by adding a preinstall hook annotation to the namespace template, however when the application is synced, the namespace gets auto-deleted. I haven't found a way around this yet. Maybe I'll just create a helm chart that just creates the namespaces and sync that before syncing the application.

dansou901 commented 3 years ago

Ok, if you don't use preinstall hooks at all, it works with at least ArgoCD version 1.8. Then the namespace is created prior to everything else. But, preinstall hooks won't work then.