Disaster Recovery instructions do not work for Google Kubernetes Engine (GKE)

[kahayes]

Checklist:

• [x] I've searched in the docs and FAQ for my answer: http://bit.ly/argocd-faq.
• [x] I've included steps to reproduce the bug.
• [x] I've pasted the output of argocd version.

Describe the bug

Using the disaster recovery instructions here: https://argoproj.github.io/argo-cd/operator-manual/disaster_recovery/

results in an error when using a GKE cluster

echo $VERSION; docker run -v ~/.kube:/home/argocd/.kube --rm argoproj/argocd:$VERSION argocd-util export > backup.yaml
v1.4.2
time="2020-03-02T17:18:31Z" level=fatal msg="Get https://35.196.66.173/api/v1/namespaces/default/configmaps/argocd-cm: error executing access token command \"/usr/lib/google-cloud-sdk/bin/gcloud config config-helper --format=json\": err=fork/exec /usr/lib/google-cloud-sdk/bin/gcloud: no such file or directory output= stderr="

Workaround posted at bottom of page

To Reproduce Create a cluster in GKE and deploy ArgoCD to it. https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-cluster

Then run the disaster recovery instructions against the cluster.

Expected behavior

Screenshots

Version

argocd-linux-amd64 version
argocd: v1.3.0-rc1+8a43840
  BuildDate: 2019-10-16T21:44:49Z
  GitCommit: 8a43840f0baef50946d3eadc1d52d6a2abc162d5
  GitTreeState: clean
  GoVersion: go1.12.6
  Compiler: gc
  Platform: linux/amd64
argocd-server: v1.4.2+48cced9
  BuildDate: 2020-01-24T01:07:03Z
  GitCommit: 48cced9d925b5bc94f6aa9fa4a8a19b2a59e128a
  GitTreeState: clean
  GoVersion: go1.12.6
  Compiler: gc
  Platform: linux/amd64
  Ksonnet Version: v0.13.1
  Kustomize Version: Version: {Version:kustomize/v3.2.1 GitCommit:d89b448c745937f0cf1936162f26a5aac688f840 BuildDate:2019-09-27T00:10:52Z GoOs:linux GoArch:amd64}
  Helm Version: v2.15.2
  Kubectl Version: v1.14.0

Logs

Paste any relevant application logs here.

Workaround run command from argocd-server pod instead of docker

kubectl exec -t -n argocd -c argocd-server $(kubectl get pods -n argocd --selector=app.kubernetes.io/name=argocd-server -o   jsonpath='{.items[*].metadata.name}') bash -- -c "argocd-util export" > backup.yaml

[jasonsattler]

Same issue with AWS since there is external command in the context for auth.

Any workaround for the import. I need to move my argocd to another cluster. I was just going to try the install the same version then try importing the backup using kubectl -n argocd apply -f backup.yaml

[kahayes]

@jasonsattler For the import I installed the same version, then did the following:

kubectl cp -n argocd -c argocd-server backup.yaml $(kubectl get pods -n argocd --selector=app.kubernetes.io/name=argocd-server -o jsonpath='{.items[*].metadata.name}'):/tmp/backup.yaml

kubectl exec -t -n argocd -c argocd-server $(kubectl get pods -n argocd --selector=app.kubernetes.io/name=argocd-server -o jsonpath='{.items[*].metadata.name}') bash -- -c "argocd-util import /tmp/backup.yaml"

kubectl exec -t -n argocd -c argocd-server $(kubectl get pods -n argocd --selector=app.kubernetes.io/name=argocd-server -o jsonpath='{.items[*].metadata.name}') bash -- -c "rm /tmp/backup.yaml"

This worked for me!

[aogier]

@jasonsattler FYI in order to get it working on aws you have to put every piece together:

docker run --rm \
-v ~/.kube:/home/argocd/.kube \
-v ~/.aws:/home/argocd/.aws \
-e KUBECONFIG=/home/argocd/.kube/[redacted] \
-u $(id -u) \
-e HOME=/home/argocd argoproj/argocd:v1.7.9 \
argocd-util export -n argocd

• volumes are configurations/credentials we need in order to have it to work
• KUBECONFIG is because I have a lot of configurations in ~/.kube
• user-related flags -u and -e HOME are because container need to run as my user, and manually set a working HOME env var
• better to specify namespace if your current context isn't pointed to argocd

I think for google cloud there could be something similar, the trick is to put everything together as you would configure another workstation you should access k8s from :+1:

[cheskayang]

with v2.4.11 backup can be exported using argocd -n argocd admin export > backup-2-4-11.yaml