missing client token when using sidecar and k8s auth type

[omriarieli]

Describe the bug Im trying to use this as a side car and get error 400 missing client token

To Reproduce Steps to reproduce the behavior:

1. configure argocd side car as per InitContainer and configuration via sidecar instructions
2. Create a secret for vault configuration

   kind: Secret
   apiVersion: v1
   metadata:
   name: argocd-vault-plugin-credentials
   type: Opaque
   stringData:
   AVP_AUTH_TYPE: "k8s"
   AVP_K8S_MOUNT_PATH: "my-mount-path"
   AVP_K8S_ROLE: "argocd"
   AVP_TYPE: "vault"
   VAULT_ADDR: "https://my-valt-adrress.com"

3. apply a sample application with plugin env

   apiVersion: argoproj.io/v1alpha1
   kind: Application
   metadata:
   name: vault-test-app
   namespace: argocd
   finalizers:
   - resources-finalizer.argocd.argoproj.io
   labels:
   env: test
   spec:
   destination:
   namespace: test
   server: 'https://kubernetes.default.svc'
   source:
   repoURL: 'https://chart-repo.com'
   targetRevision: 0.3.21
   chart: my-chart
   plugin:
     env:
       - name: ARGOCD_ENV_HELM_VALUES
         value: |
           redis:
             auth:
               password: <path:mycompany/dev/data/pws-helm#REDIS_PASS>

4. Also tried from inside the sidecar pod with a sample secret:

   kind: Secret
   apiVersion: v1
   metadata:
   name: test-secret
   type: Opaque
   data:
   password: <path:secret/dev/data/pws-helm#REDIS_PASS>

Expected behavior The REDIS_PASS env var gets the value from vault vault has an argocd role on the my-mount-path access method

Screenshots/Verbose output

$ argocd-vault-plugin generate -s argocd-vault-plugin-credentials --verbose-sensitive-output .
2023/02/13 23:03:37 reading configuration from secret argocd-vault-plugin-credentials
2023/02/13 23:03:37 parsed secret name as argocd-vault-plugin-credentials from namespace argocd
2023/02/13 23:03:37 Setting VAULT_ADDR to https://my-valt-adrress.com for backend SDK
2023/02/13 23:03:37 reading configuration from environment, overriding any previous settings
2023/02/13 23:03:37 AVP configured with the following settings:

2023/02/13 23:03:37 avp_k8s_mount_path: my-vault-auth-mount-path

2023/02/13 23:03:37 avp_k8s_role: argocd

2023/02/13 23:03:37 avp_type: vault

2023/02/13 23:03:37 vault_addr: https://my-valt-adrress.com 

2023/02/13 23:03:37 avp_auth_type: k8s

2023/02/13 23:03:37 avp_kv_version: 2

2023/02/13 23:03:37 Hashicorp Vault authenticating with Vault role argocd using Kubernetes service account token /var/run/secrets/kubernetes.io/serviceaccount/token read from XXXXXXXXXXXXXXXX-KUBERNETES-SERVICE-ACCOUNT-TOKEN-XXXXXXXXXXXXXXXX
Error: Error making API request.

URL: PUT https://my-valt-adrress.com/v1/my-vault-auth-mount-path/login
Code: 400. Errors:

* missing client token

it looks like maybe its trying to authenticate with a vault token even though i set AVP_AUTH_TYPE: "k8s"? or it just cant login to vault and get a token? what about system:auth-delegator cluster role does this come into play here? sidecar image used quay.io/argoproj/argocd:v2.4.0 plugin version used 1.13.1

[omriarieli]

anyone?

[werne2j]

or it just cant login to vault and get a token?

I believe this is the case. I would check your Vault k8s auth setup. Is kubernetes auth enabled at your mount path? https://developer.hashicorp.com/vault/docs/auth/kubernetes#kubernetes-auth-method

Could be a namespace issue, could be something else. This might be helpful https://github.com/hashicorp/vault-plugin-auth-kubernetes/issues/109

[rayhimself]

Hi I have the same issue, i believe it is caused by AVP_K8S_MOUNT_PATH variable, i have exactly the same argo cd installation with default vault auth mount path and it works great, i also tried to set default value to AVP_K8S_MOUNT_PATH var and got the same 400 missing client token error