hashicorp / vault-secrets-operator

The Vault Secrets Operator (VSO) allows Pods to consume Vault secrets natively from Kubernetes Secrets.
https://hashicorp.com
Other
436 stars 89 forks source link

VaultStaticSecrets not resuming secret syncing post a Vault seal event #840

Open MysticalMount opened 1 week ago

MysticalMount commented 1 week ago

Describe the bug VaultSaticSecret doesnt resume syncing post a Vault being sealed, and then unsealed. Vault instance is external to the cluster and was successfully unsealed.

Expected VaultSaticSecret to resume syncing post unseal.

To Reproduce

  1. Set up a VaultConnection and VaultAuth in the operator namespace.
  2. Create a VaultStaticSecrert using default connection and auth
  3. Confirm secret syncing
  4. Terminate a Vault instance
  5. Bring back up a Vault instance
  6. Unseal
  7. VaultStaticSecret will report 503 Vault is sealed permanenty

Application deployment:

    defaultVaultConnection:
      enabled: true
      address: "http://xxx:8200"
    defaultAuthMethod:
      enabled: true
      namespace: ""
      method: kubernetes
      mount: xxx
      kubernetes:
        role: "xxx"
        serviceAccount: vault-auth

Vault operator was restarted. No errors in the controller/operator logs pre or post the restart. Problem persisted for all VaultStaticSecrets.

Expected behavior Expected VaultStaticSecret to resume post Vault becoming re-available, it seemed to stop after roughly 10 minutes but this is a best guess.

Environment

Additional context Add any other context about the problem here.

MysticalMount commented 1 week ago

I think whats happening here is the secret is being refreshed, but no event is generated if the secret already exists. Deletion of the target secret resource, post Vault's status becoming available again (i.e. successful connection and unsealed) - does seem to happen automatically.

However Ive only tested this post a connection error, whereupon I realised this, versus a 503/Vault is sealed but likely the behaviour is the same.