search

hashicorp / consul-template

Template rendering, notifier, and supervisor for @HashiCorp Consul and Vault data.
https://www.hashicorp.com/
Mozilla Public License 2.0
4.76k stars 781 forks source link

Quick Example section in the README does not work as expected: `vault.token: failed to renew:` #1457

Open Dentrax opened 3 years ago

Dentrax commented 3 years ago

Consul Template version

consul-template v0.25.2 (870905d)

Configuration

in.tpl:

{{ key "foo" }}

Command

$ consul agent -dev
$ consul-template -template "in.tpl:out.txt" -once

Debug output

[INFO] consul-template v0.25.2 (870905d)
[INFO] (runner) creating new runner (dry: false, once: true)
[DEBUG] (runner) final config: {"Consul":{"Address":"","Namespace":"","Auth":{"Enabled":false,"Username":"","Password":""},"Retry":{"Attempts":12,"Backoff":250000000,"MaxBackoff":60000000000,"Enabled":true},"SSL":{"CaCert":"","CaPath":"","Cert":"","Enabled":false,"Key":"","ServerName":"","Verify":true},"Token":"","Transport":{"DialKeepAlive":30000000000,"DialTimeout":30000000000,"DisableKeepAlives":false,"IdleConnTimeout":90000000000,"MaxIdleConns":100,"MaxIdleConnsPerHost":13,"TLSHandshakeTimeout":10000000000}},"Dedup":{"Enabled":false,"MaxStale":2000000000,"Prefix":"consul-template/dedup/","TTL":15000000000,"BlockQueryWaitTime":60000000000},"DefaultDelims":{"Left":null,"Right":null},"Exec":{"Command":"","Enabled":false,"Env":{"Denylist":[],"Custom":[],"Pristine":false,"Allowlist":[]},"KillSignal":2,"KillTimeout":30000000000,"ReloadSignal":null,"Splay":0,"Timeout":0},"KillSignal":2,"LogLevel":"trace","MaxStale":2000000000,"PidFile":"","ReloadSignal":1,"Syslog":{"Enabled":false,"Facility":"LOCAL0","Name":"consul-template"},"Templates":[{"Backup":false,"Command":"","CommandTimeout":30000000000,"Contents":"","CreateDestDirs":true,"Destination":"out.txt","ErrMissingKey":false,"Exec":{"Command":"","Enabled":false,"Env":{"Denylist":[],"Custom":[],"Pristine":false,"Allowlist":[]},"KillSignal":2,"KillTimeout":30000000000,"ReloadSignal":null,"Splay":0,"Timeout":30000000000},"Perms":0,"Source":"in.tpl","Wait":{"Enabled":false,"Min":0,"Max":0},"LeftDelim":"","RightDelim":"","FunctionDenylist":[],"SandboxPath":""}],"Vault":{"Address":"","Enabled":false,"Namespace":"","RenewToken":true,"Retry":{"Attempts":12,"Backoff":250000000,"MaxBackoff":60000000000,"Enabled":true},"SSL":{"CaCert":"","CaPath":"","Cert":"","Enabled":true,"Key":"","ServerName":"","Verify":true},"Transport":{"DialKeepAlive":30000000000,"DialTimeout":30000000000,"DisableKeepAlives":false,"IdleConnTimeout":90000000000,"MaxIdleConns":100,"MaxIdleConnsPerHost":13,"TLSHandshakeTimeout":10000000000},"UnwrapToken":false},"Wait":{"Enabled":false,"Min":null,"Max":null},"Once":true,"BlockQueryWaitTime":60000000000}
[INFO] (runner) creating watcher
[DEBUG] (watcher) adding vault.token
[TRACE] (watcher) vault.token starting
[TRACE] (view) vault.token starting fetch
[TRACE] vault.token: starting renewer
[INFO] (runner) starting
[DEBUG] (runner) running initial templates
[DEBUG] (runner) initiating run
[DEBUG] (runner) checking template 9d274ca3ed1c0997e1ccc57f41be5ceb
[DEBUG] (runner) missing data for 1 dependencies
[DEBUG] (runner) missing dependency: kv.block(foo)
[DEBUG] (runner) add used dependency kv.block(foo) to missing since isLeader but do not have a watcher
[DEBUG] (runner) was not watching 1 dependencies
[DEBUG] (watcher) adding kv.block(foo)
[TRACE] (watcher) kv.block(foo) starting
[DEBUG] (runner) diffing and updating dependencies
[DEBUG] (runner) watching 2 dependencies
[TRACE] (view) kv.block(foo) starting fetch
[TRACE] kv.block(foo): GET /v1/kv/foo?stale=true&wait=1m0s
[TRACE] kv.block(foo): returned nil
[TRACE] (view) kv.block(foo) marking successful data response
[TRACE] (view) kv.block(foo) successful contact, resetting retries
[TRACE] (view) kv.block(foo) asked for blocking query
[TRACE] kv.block(foo): GET /v1/kv/foo?index=1&stale=true&wait=1m0s

Expected behavior

Should work as expected?

Actual behavior

It tries to dial Vault on tcp 127.0.0.1:8200: to do renew-self.

[WARN] vault.token: failed to renew: Put "https://127.0.0.1:8200/v1/auth/token/renew-self": dial tcp 127.0.0.1:8200: connect: connection refused
[WARN] vault.token: renewer done (maybe the lease expired)
[WARN] (view) lease expired or is not renewable (retry attempt 1 after "250ms")
eikenb commented 3 years ago

Hey @Dentrax, thanks for the report.

How did you get the "Actual behavior" (the WARN entries about vault) output you show? It doesn't appear in the "Debug output" and I can't reproduce it. Thanks.

eikenb commented 3 years ago

@Dentrax, I had a thought...

Check the home directory of the account you're using to run consul-template for a file, ~/.vault-token. If it sees this file it will enable the vault connection and try to get a token resulting in this warning. Just removing that file will fix it.

Dentrax commented 3 years ago

Thanks! @eikenb Yes, there was a file in ~ dir called vault-token. I just deleted it:

$ cat ~/.vault-token
root
$ rm ~/.vault-token

And now the $ consul-template -template "in.tpl:out.txt" -once command does nothing and being stuck until I send SIGTERM from keyboard. This log shown up in the agent as soon as I run the that command:

2021-03-06T21:41:03.815+0300 [DEBUG] agent.http: Request finished: method=GET url=/v1/kv/foo?stale=&wait=60000ms from=127.0.0.1:50257 latency=42.542µs

Any thoughts? Thanks...

eikenb commented 3 years ago

:fireworks: Glad that helped @Dentrax.

I think your current issue is that the consul server you are using doesn't have the value the template is looking up. Consul-template blocks until all template variables have their values retrieved before rendering the templates and -once doesn't exit until all templates have rendered the one time.

Dentrax commented 3 years ago

I think your current issue is that the consul server you are using doesn't have the value the template is looking up.

I could not understand exactly what it means actually. Do I need to do anything manually before I run consul-template -template "in.tpl:out.txt" -oncet?

tonyganga commented 3 years ago

@Dentrax I had similar issues with the steps in the documentation. I was able to get around this by passing -vault-renew-token=false to consul-template. I think this is because I have vault token being set in one of the three places mentioned here.

This should get you going: consul-template -template "in.tpl:out.txt" -vault-renew-token=false

Reference Issue: https://github.com/hashicorp/consul-template/issues/1297

eikenb commented 3 years ago

Using the original example, does the Consul server you are pulling values from have a value store for the foo key used?

For example, if you have the template with {{ key "foo" }} in it, just started a test consul with consul agent -dev and run consul-template like consul-template -template "in.tmpl:out.txt" -once it will block until that foo entry has a value. You can do that with the command consul kv put foo "hello" and if you run that with consul-template already running (like above) and blocking it will unblock and render the file with "hello" in it.