external-secrets / kubernetes-external-secrets

Integrate external secret management systems with Kubernetes
MIT License
2.6k stars 404 forks source link

More debugging information when trying to connect to external engines #852

Closed jonrober closed 2 years ago

jonrober commented 2 years ago

I'm currently having a problem with getting a 404 error when the script tries to poll a vault backend. Fair enough, I probably misconfigured a path. But then I go to vault to check the audit logs to see what's failing, and find no requests that aren't otherwise spoken for. VAULT_ADDR is set, so as far as I can tell, it should be trying to connect to vault but isn't.

It would be useful in this case to have LOG_LEVEL=debug to print out more information about the connection that's failing, not just the response. In the case of vault, that would involve printing out the full URL that it attempted to contact so that I can verify I haven't made a deeper mistake that I'm missing.

{"level":20,"message_time":"2021-10-14T20:45:26.446Z","pid":17,"hostname":"external-secrets-kubernetes-external-secrets-5dc9cd7658-vv8f5","msg":"next poll for default/hello-vault-service in 9976 ms"}
{"level":30,"message_time":"2021-10-14T20:45:36.432Z","pid":17,"hostname":"external-secrets-kubernetes-external-secrets-5dc9cd7658-vv8f5","msg":"running poll on the secret default/hello-vault-service"}
{"level":20,"message_time":"2021-10-14T20:45:36.451Z","pid":17,"hostname":"external-secrets-kubernetes-external-secrets-5dc9cd7658-vv8f5","msg":"fetching new token from vault for role eks-dev-role on eks-dev"}
{"level":50,"message_time":"2021-10-14T20:45:36.473Z","pid":17,"hostname":"external-secrets-kubernetes-external-secrets-5dc9cd7658-vv8f5","payload":{"response":{"statusCode":404,"body":{"errors":[]}}},"msg":"failure while polling the secret default/hello-vault-service"}
jonrober commented 2 years ago

I did just find the actual error -- VAULT_ADDR didn't like the fact that I cut and pasted from my environment where it ended in a /. When I remove the trailing /, external secrets does properly connect to the server and gets just a permission denied error, which I can move on from. The debugging request is still a nice-to-have though.

I'm guessing that the vault server doesn't like the extra / in the URL and gives you a 404 without hitting anything that would trigger the vault audit logs, but the vault client helpfully strips out a trailing / from the vault address when building URLs. So a bit of a gotcha if someone happens to copy their VAULT_ADDR straight from normal environment without realizing the problem.

github-actions[bot] commented 2 years ago

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 30 days.

github-actions[bot] commented 2 years ago

This issue was closed because it has been stalled for 30 days with no activity.