search

hashicorp / vault

A tool for secrets management, encryption as a service, and privileged access management
https://www.vaultproject.io/
Other
30.04k stars 4.12k forks source link

Vault agent crashes after restart #17166

Closed const-tmp closed 1 year ago

const-tmp commented 1 year ago

Describe the bug Vault agent with templates crashes with panic when restarted:

$ vault agent -config=/etc/vault.d/vault-agent.hcl
==> Vault agent started! Log data will stream in below:

==> Vault agent configuration:

           Api Address 1: http://127.0.0.1:8100
           Api Address 2: http://bufconn
                     Cgo: disabled
               Log Level: info
                 Version: Vault v1.11.3, built 2022-08-26T10:27:10Z
             Version Sha: 17250b25303c6418c283c95b1d5a9c9f16174fe8

2022-09-16T06:50:11.373Z [INFO]  template.server: starting template server
2022-09-16T06:50:11.373Z [INFO] (runner) creating new runner (dry: false, once: false)
2022-09-16T06:50:11.374Z [INFO] (runner) creating watcher
2022-09-16T06:50:11.374Z [INFO]  auth.handler: starting auth handler
2022-09-16T06:50:11.374Z [INFO]  auth.handler: authenticating
2022-09-16T06:50:11.375Z [INFO]  sink.server: starting sink server
2022-09-16T06:50:11.403Z [INFO]  auth.handler: authentication successful, sending token to sinks
2022-09-16T06:50:11.403Z [INFO]  auth.handler: starting renewal process
2022-09-16T06:50:11.403Z [INFO]  template.server: template server received new token
2022-09-16T06:50:11.404Z [INFO] (runner) stopping
2022-09-16T06:50:11.404Z [INFO] (runner) creating new runner (dry: false, once: false)
2022-09-16T06:50:11.404Z [INFO] (runner) creating watcher
2022-09-16T06:50:11.404Z [INFO] (runner) starting
2022-09-16T06:50:11.407Z [INFO]  cache: received request: method=PUT path=/v1/pki-int/issue/consul
2022-09-16T06:50:11.407Z [INFO]  cache.apiproxy: forwarding request: method=PUT path=/v1/pki-int/issue/consul
2022-09-16T06:50:11.407Z [INFO]  auth.handler: renewed auth token
2022-09-16T06:50:11.408Z [INFO]  cache: received request: method=PUT path=/v1/pki-int/issue/consul
2022-09-16T06:50:11.408Z [INFO]  cache: received request: method=PUT path=/v1/pki-int/issue/consul
2022-09-16T06:50:11.410Z [INFO]  cache.apiproxy: forwarding request: method=PUT path=/v1/pki-int/issue/consul
2022-09-16T06:50:11.410Z [INFO]  cache.apiproxy: forwarding request: method=PUT path=/v1/pki-int/issue/consul
2022-09-16T06:50:11.508Z [INFO] (runner) rendered "(dynamic)" => "/etc/consul.d/key.pem"
2022-09-16T06:50:11.512Z [INFO] (runner) rendered "(dynamic)" => "/etc/consul.d/cert.pem"
2022-09-16T06:50:11.524Z [INFO] (runner) rendered "(dynamic)" => "/etc/consul.d/ca.pem"
^C==> Vault agent shutdown triggered
2022-09-16T06:50:18.413Z [INFO]  sink.server: sink server stopped
2022-09-16T06:50:18.413Z [INFO]  sinks finished, exiting
2022-09-16T06:50:18.413Z [INFO] (runner) stopping
2022-09-16T06:50:18.413Z [INFO]  template.server: template server stopped
2022-09-16T06:50:18.413Z [INFO]  auth.handler: shutdown triggered, stopping lifetime watcher
2022-09-16T06:50:18.413Z [INFO]  auth.handler: auth handler stopped
$ vault agent -config=/etc/vault.d/vault-agent.hcl
==> Vault agent started! Log data will stream in below:

==> Vault agent configuration:

           Api Address 1: http://127.0.0.1:8100
           Api Address 2: http://bufconn
                     Cgo: disabled
               Log Level: info
                 Version: Vault v1.11.3, built 2022-08-26T10:27:10Z
             Version Sha: 17250b25303c6418c283c95b1d5a9c9f16174fe8

2022-09-16T06:50:26.826Z [INFO]  template.server: starting template server
2022-09-16T06:50:26.826Z [INFO] (runner) creating new runner (dry: false, once: false)
2022-09-16T06:50:26.826Z [INFO] (runner) creating watcher
2022-09-16T06:50:26.826Z [INFO]  auth.handler: starting auth handler
2022-09-16T06:50:26.827Z [INFO]  auth.handler: authenticating
2022-09-16T06:50:26.827Z [INFO]  sink.server: starting sink server
2022-09-16T06:50:26.851Z [INFO]  auth.handler: authentication successful, sending token to sinks
2022-09-16T06:50:26.852Z [INFO]  auth.handler: starting renewal process
2022-09-16T06:50:26.852Z [INFO]  template.server: template server received new token
2022-09-16T06:50:26.852Z [INFO] (runner) stopping
2022-09-16T06:50:26.852Z [INFO] (runner) creating new runner (dry: false, once: false)
2022-09-16T06:50:26.853Z [INFO] (runner) creating watcher
2022-09-16T06:50:26.853Z [INFO] (runner) starting
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x2b0 pc=0x2c6bd26]

goroutine 41 [running]:
github.com/hashicorp/consul-template/dependency.goodFor(0xc0008f8e00)
    /home/runner/go/pkg/mod/github.com/hashicorp/consul-template@v0.29.2/dependency/vault_pki.go:114 +0x26
github.com/hashicorp/consul-template/dependency.(*VaultPKIQuery).Fetch.func1(0x0)
    /home/runner/go/pkg/mod/github.com/hashicorp/consul-template@v0.29.2/dependency/vault_pki.go:89 +0x125
github.com/hashicorp/consul-template/dependency.(*VaultPKIQuery).Fetch(0xc0005f66c0, 0xc000920090, 0xc000087790)
    /home/runner/go/pkg/mod/github.com/hashicorp/consul-template@v0.29.2/dependency/vault_pki.go:96 +0x115
github.com/hashicorp/consul-template/watch.(*View).fetch(0xc000c77b80, 0xc000071260, 0x6492378, 0xc000c46ac0)
    /home/runner/go/pkg/mod/github.com/hashicorp/consul-template@v0.29.2/watch/view.go:203 +0x131
created by github.com/hashicorp/consul-template/watch.(*View).poll
    /home/runner/go/pkg/mod/github.com/hashicorp/consul-template@v0.29.2/watch/view.go:117 +0x145

Problem solves when rendered files are deleted manually.

To Reproduce

Steps to reproduce the behavior:

  1. vault agent -config=/etc/vault.d/vault-agent.hcl // first run, templates rendered as expected
  2. stop agent
  3. vault agent -config=/etc/vault.d/vault-agent.hcl // crashes
  4. rm /etc/consul.d/ca.pem /etc/consul.d/cert.pem /etc/consul.d/key.pem
  5. vault agent -config=/etc/vault.d/vault-agent.hcl // works

Expected behavior I expected Vault starts and continues working.

Environment:

Vault server configuration file(s):

$ cat /etc/vault.d/vault-agent.hcl

vault {
  address = "https://vault.*****.com:8200"
  retry {
    num_retries = 5
  }
}

auto_auth {
  method {
    type = "approle"

    config = {
      role_id_file_path                   = "roleid"
      secret_id_file_path                 = "secretid"
      remove_secret_id_file_after_reading = false
    }
  }
}

cache {
  use_auto_auth_token = true
}

listener "tcp" {
  address     = "127.0.0.1:8100"
  tls_disable = true
}

template {
  contents    = <<EOT
{{ with pkiCert "pki-int/issue/consul" "common_name=consul.*****.com" }}
{{ .Data.Key }}
{{ end }}
EOT
  destination = "/etc/consul.d/key.pem"
}

template {
  contents    = <<EOT
{{ with pkiCert "pki-int/issue/consul" "common_name=consul.*****.com" }}
{{ .Data.Cert }}
{{ end }}
EOT
  destination = "/etc/consul.d/cert.pem"
}

template {
  contents    = <<EOT
{{ with pkiCert "pki-int/issue/consul" "common_name=consul.*****.com" }}
{{ .Data.CA }}
{{ end }}
EOT
  destination = "/etc/consul.d/ca.pem"
}
const-tmp commented 1 year ago

UPD -log-level=debug

$ vault agent -config=/etc/vault.d/vault-agent.hcl -log-level=debug
==> Vault agent started! Log data will stream in below:

==> Vault agent configuration:

           Api Address 1: http://127.0.0.1:8100
           Api Address 2: http://bufconn
                     Cgo: disabled
               Log Level: debug
                 Version: Vault v1.11.3, built 2022-08-26T10:27:10Z
             Version Sha: 17250b25303c6418c283c95b1d5a9c9f16174fe8

2022-09-16T10:36:59.273Z [DEBUG] cache: auto-auth token is allowed to be used; configuring inmem sink
2022-09-16T10:36:59.276Z [INFO]  template.server: starting template server
2022-09-16T10:36:59.276Z [INFO] (runner) creating new runner (dry: false, once: false)
2022-09-16T10:36:59.277Z [DEBUG] (runner) final config: {"Consul":{"Address":"","Namespace":"","Auth":{"Enabled":false,"Username":""},"Retry":{"Attempts":12,"Backoff":250000000,"MaxBackoff":60000000000,"Enabled":true},"SSL":{"CaCert":"","CaPath":"","Cert":"","Enabled":false,"Key":"","ServerName":"","Verify":true},"Token":"","TokenFile":"","Transport":{"CustomDialer":null,"DialKeepAlive":30000000000,"DialTimeout":30000000000,"DisableKeepAlives":false,"IdleConnTimeout":90000000000,"MaxIdleConns":100,"MaxIdleConnsPerHost":2,"TLSHandshakeTimeout":10000000000}},"Dedup":{"Enabled":false,"MaxStale":2000000000,"Prefix":"consul-template/dedup/","TTL":15000000000,"BlockQueryWaitTime":60000000000},"DefaultDelims":{"Left":null,"Right":null},"Exec":{"Command":[],"Enabled":false,"Env":{"Denylist":[],"Custom":[],"Pristine":false,"Allowlist":[]},"KillSignal":2,"KillTimeout":30000000000,"ReloadSignal":null,"Splay":0,"Timeout":0},"KillSignal":2,"LogLevel":"DEBUG","FileLog":{"LogFilePath":"","LogRotateBytes":0,"LogRotateDuration":86400000000000,"LogRotateMaxFiles":0},"MaxStale":2000000000,"PidFile":"","ReloadSignal":1,"Syslog":{"Enabled":false,"Facility":"LOCAL0","Name":"consul-template"},"Templates":[{"Backup":false,"Command":["sh -c 'date \u0026\u0026 consul reload'"],"CommandTimeout":30000000000,"Contents":"{{ with pkiCert \"pki-int/issue/consul\" \"common_name=consul2.*****.com\" }}\n{{ .Data.Key }}\n{{ end }}\n","CreateDestDirs":true,"Destination":"/etc/consul.d/key.pem","ErrMissingKey":false,"ErrFatal":true,"Exec":{"Command":["sh -c 'date \u0026\u0026 consul reload'"],"Enabled":true,"Env":{"Denylist":[],"Custom":[],"Pristine":false,"Allowlist":[]},"KillSignal":2,"KillTimeout":30000000000,"ReloadSignal":null,"Splay":0,"Timeout":30000000000},"Perms":0,"User":null,"Uid":null,"Group":null,"Gid":null,"Source":"","Wait":{"Enabled":false,"Min":0,"Max":0},"LeftDelim":"","RightDelim":"","FunctionDenylist":[],"SandboxPath":""},{"Backup":false,"Command":["sh -c 'date \u0026\u0026 consul reload'"],"CommandTimeout":30000000000,"Contents":"{{ with pkiCert \"pki-int/issue/consul\" \"common_name=consul2.*****.com\" }}\n{{ .Data.Cert }}\n{{ end }}\n","CreateDestDirs":true,"Destination":"/etc/consul.d/cert.pem","ErrMissingKey":false,"ErrFatal":true,"Exec":{"Command":["sh -c 'date \u0026\u0026 consul reload'"],"Enabled":true,"Env":{"Denylist":[],"Custom":[],"Pristine":false,"Allowlist":[]},"KillSignal":2,"KillTimeout":30000000000,"ReloadSignal":null,"Splay":0,"Timeout":30000000000},"Perms":0,"User":null,"Uid":null,"Group":null,"Gid":null,"Source":"","Wait":{"Enabled":false,"Min":0,"Max":0},"LeftDelim":"","RightDelim":"","FunctionDenylist":[],"SandboxPath":""},{"Backup":false,"Command":["sh -c 'date \u0026\u0026 consul reload'"],"CommandTimeout":30000000000,"Contents":"{{ with pkiCert \"pki-int/issue/consul\" \"common_name=consul2.*****.com\" }}\n{{ .Data.CA }}\n{{ end }}\n","CreateDestDirs":true,"Destination":"/etc/consul.d/ca.pem","ErrMissingKey":false,"ErrFatal":true,"Exec":{"Command":["sh -c 'date \u0026\u0026 consul reload'"],"Enabled":true,"Env":{"Denylist":[],"Custom":[],"Pristine":false,"Allowlist":[]},"KillSignal":2,"KillTimeout":30000000000,"ReloadSignal":null,"Splay":0,"Timeout":30000000000},"Perms":0,"User":null,"Uid":null,"Group":null,"Gid":null,"Source":"","Wait":{"Enabled":false,"Min":0,"Max":0},"LeftDelim":"","RightDelim":"","FunctionDenylist":[],"SandboxPath":""}],"TemplateErrFatal":null,"Vault":{"Address":"http://127.0.0.1:8200","Enabled":true,"Namespace":"","RenewToken":false,"Retry":{"Attempts":0,"Backoff":250000000,"MaxBackoff":60000000000,"Enabled":false},"SSL":{"CaCert":"","CaPath":"","Cert":"","Enabled":false,"Key":"","ServerName":"","Verify":false},"Transport":{"CustomDialer":{},"DialKeepAlive":30000000000,"DialTimeout":30000000000,"DisableKeepAlives":false,"IdleConnTimeout":90000000000,"MaxIdleConns":100,"MaxIdleConnsPerHost":2,"TLSHandshakeTimeout":10000000000},"UnwrapToken":false,"DefaultLeaseDuration":300000000000,"LeaseRenewalThreshold":0.9,"K8SAuthRoleName":"","K8SServiceAccountTokenPath":"/run/secrets/kubernetes.io/serviceaccount/token","K8SServiceAccountToken":"","K8SServiceMountPath":"kubernetes"},"Nomad":{"Address":"","Enabled":false,"Namespace":"","SSL":{"CaCert":"","CaPath":"","Cert":"","Enabled":false,"Key":"","ServerName":"","Verify":true},"AuthUsername":"","AuthPassword":"","Transport":{"CustomDialer":null,"DialKeepAlive":30000000000,"DialTimeout":30000000000,"DisableKeepAlives":false,"IdleConnTimeout":90000000000,"MaxIdleConns":100,"MaxIdleConnsPerHost":2,"TLSHandshakeTimeout":10000000000},"Retry":{"Attempts":12,"Backoff":250000000,"MaxBackoff":60000000000,"Enabled":true}},"Wait":{"Enabled":false,"Min":0,"Max":0},"Once":false,"ParseOnly":false,"BlockQueryWaitTime":60000000000}
2022-09-16T10:36:59.277Z [INFO] (runner) creating watcher
2022-09-16T10:36:59.278Z [INFO]  auth.handler: starting auth handler
2022-09-16T10:36:59.278Z [INFO]  auth.handler: authenticating
2022-09-16T10:36:59.280Z [INFO]  sink.server: starting sink server
2022-09-16T10:36:59.304Z [INFO]  auth.handler: authentication successful, sending token to sinks
2022-09-16T10:36:59.304Z [INFO]  auth.handler: starting renewal process
2022-09-16T10:36:59.304Z [DEBUG] cache.leasecache: storing auto-auth token into the cache
2022-09-16T10:36:59.304Z [INFO]  template.server: template server received new token
2022-09-16T10:36:59.304Z [INFO] (runner) stopping
2022-09-16T10:36:59.304Z [DEBUG] (runner) stopping watcher
2022-09-16T10:36:59.305Z [DEBUG] (watcher) stopping all views
2022-09-16T10:36:59.305Z [INFO] (runner) creating new runner (dry: false, once: false)
2022-09-16T10:36:59.305Z [DEBUG] (runner) final config: {"Consul":{"Address":"","Namespace":"","Auth":{"Enabled":false,"Username":""},"Retry":{"Attempts":12,"Backoff":250000000,"MaxBackoff":60000000000,"Enabled":true},"SSL":{"CaCert":"","CaPath":"","Cert":"","Enabled":false,"Key":"","ServerName":"","Verify":true},"Token":"","TokenFile":"","Transport":{"CustomDialer":null,"DialKeepAlive":30000000000,"DialTimeout":30000000000,"DisableKeepAlives":false,"IdleConnTimeout":90000000000,"MaxIdleConns":100,"MaxIdleConnsPerHost":2,"TLSHandshakeTimeout":10000000000}},"Dedup":{"Enabled":false,"MaxStale":2000000000,"Prefix":"consul-template/dedup/","TTL":15000000000,"BlockQueryWaitTime":60000000000},"DefaultDelims":{"Left":null,"Right":null},"Exec":{"Command":[],"Enabled":false,"Env":{"Denylist":[],"Custom":[],"Pristine":false,"Allowlist":[]},"KillSignal":2,"KillTimeout":30000000000,"ReloadSignal":null,"Splay":0,"Timeout":0},"KillSignal":2,"LogLevel":"DEBUG","FileLog":{"LogFilePath":"","LogRotateBytes":0,"LogRotateDuration":86400000000000,"LogRotateMaxFiles":0},"MaxStale":2000000000,"PidFile":"","ReloadSignal":1,"Syslog":{"Enabled":false,"Facility":"LOCAL0","Name":"consul-template"},"Templates":[{"Backup":false,"Command":["sh -c 'date \u0026\u0026 consul reload'"],"CommandTimeout":30000000000,"Contents":"{{ with pkiCert \"pki-int/issue/consul\" \"common_name=consul2.*****.com\" }}\n{{ .Data.Key }}\n{{ end }}\n","CreateDestDirs":true,"Destination":"/etc/consul.d/key.pem","ErrMissingKey":false,"ErrFatal":true,"Exec":{"Command":["sh -c 'date \u0026\u0026 consul reload'"],"Enabled":true,"Env":{"Denylist":[],"Custom":[],"Pristine":false,"Allowlist":[]},"KillSignal":2,"KillTimeout":30000000000,"ReloadSignal":null,"Splay":0,"Timeout":30000000000},"Perms":0,"User":null,"Uid":null,"Group":null,"Gid":null,"Source":"","Wait":{"Enabled":false,"Min":0,"Max":0},"LeftDelim":"","RightDelim":"","FunctionDenylist":[],"SandboxPath":""},{"Backup":false,"Command":["sh -c 'date \u0026\u0026 consul reload'"],"CommandTimeout":30000000000,"Contents":"{{ with pkiCert \"pki-int/issue/consul\" \"common_name=consul2.*****.com\" }}\n{{ .Data.Cert }}\n{{ end }}\n","CreateDestDirs":true,"Destination":"/etc/consul.d/cert.pem","ErrMissingKey":false,"ErrFatal":true,"Exec":{"Command":["sh -c 'date \u0026\u0026 consul reload'"],"Enabled":true,"Env":{"Denylist":[],"Custom":[],"Pristine":false,"Allowlist":[]},"KillSignal":2,"KillTimeout":30000000000,"ReloadSignal":null,"Splay":0,"Timeout":30000000000},"Perms":0,"User":null,"Uid":null,"Group":null,"Gid":null,"Source":"","Wait":{"Enabled":false,"Min":0,"Max":0},"LeftDelim":"","RightDelim":"","FunctionDenylist":[],"SandboxPath":""},{"Backup":false,"Command":["sh -c 'date \u0026\u0026 consul reload'"],"CommandTimeout":30000000000,"Contents":"{{ with pkiCert \"pki-int/issue/consul\" \"common_name=consul2.*****.com\" }}\n{{ .Data.CA }}\n{{ end }}\n","CreateDestDirs":true,"Destination":"/etc/consul.d/ca.pem","ErrMissingKey":false,"ErrFatal":true,"Exec":{"Command":["sh -c 'date \u0026\u0026 consul reload'"],"Enabled":true,"Env":{"Denylist":[],"Custom":[],"Pristine":false,"Allowlist":[]},"KillSignal":2,"KillTimeout":30000000000,"ReloadSignal":null,"Splay":0,"Timeout":30000000000},"Perms":0,"User":null,"Uid":null,"Group":null,"Gid":null,"Source":"","Wait":{"Enabled":false,"Min":0,"Max":0},"LeftDelim":"","RightDelim":"","FunctionDenylist":[],"SandboxPath":""}],"TemplateErrFatal":null,"Vault":{"Address":"http://127.0.0.1:8200","Enabled":true,"Namespace":"","RenewToken":false,"Retry":{"Attempts":0,"Backoff":250000000,"MaxBackoff":60000000000,"Enabled":false},"SSL":{"CaCert":"","CaPath":"","Cert":"","Enabled":false,"Key":"","ServerName":"","Verify":false},"Transport":{"CustomDialer":{},"DialKeepAlive":30000000000,"DialTimeout":30000000000,"DisableKeepAlives":false,"IdleConnTimeout":90000000000,"MaxIdleConns":100,"MaxIdleConnsPerHost":2,"TLSHandshakeTimeout":10000000000},"UnwrapToken":false,"DefaultLeaseDuration":300000000000,"LeaseRenewalThreshold":0.9,"K8SAuthRoleName":"","K8SServiceAccountTokenPath":"/run/secrets/kubernetes.io/serviceaccount/token","K8SServiceAccountToken":"","K8SServiceMountPath":"kubernetes"},"Nomad":{"Address":"","Enabled":false,"Namespace":"","SSL":{"CaCert":"","CaPath":"","Cert":"","Enabled":false,"Key":"","ServerName":"","Verify":true},"AuthUsername":"","AuthPassword":"","Transport":{"CustomDialer":null,"DialKeepAlive":30000000000,"DialTimeout":30000000000,"DisableKeepAlives":false,"IdleConnTimeout":90000000000,"MaxIdleConns":100,"MaxIdleConnsPerHost":2,"TLSHandshakeTimeout":10000000000},"Retry":{"Attempts":12,"Backoff":250000000,"MaxBackoff":60000000000,"Enabled":true}},"Wait":{"Enabled":false,"Min":0,"Max":0},"Once":false,"ParseOnly":false,"BlockQueryWaitTime":60000000000}
2022-09-16T10:36:59.305Z [INFO] (runner) creating watcher
2022-09-16T10:36:59.305Z [INFO] (runner) starting
2022-09-16T10:36:59.305Z [DEBUG] (runner) running initial templates
2022-09-16T10:36:59.305Z [DEBUG] (runner) initiating run
2022-09-16T10:36:59.306Z [DEBUG] (runner) checking template 7d6ef5b66582f8485abd307375a607b4
2022-09-16T10:36:59.306Z [DEBUG] (runner) missing data for 1 dependencies
2022-09-16T10:36:59.307Z [DEBUG] (runner) missing dependency: vault.pki(pki-int/issue/consul->/etc/consul.d/key.pem)
2022-09-16T10:36:59.307Z [DEBUG] (runner) add used dependency vault.pki(pki-int/issue/consul->/etc/consul.d/key.pem) to missing since isLeader but do not have a watcher
2022-09-16T10:36:59.307Z [DEBUG] (runner) was not watching 1 dependencies
2022-09-16T10:36:59.307Z [DEBUG] (watcher) adding vault.pki(pki-int/issue/consul->/etc/consul.d/key.pem)
2022-09-16T10:36:59.307Z [DEBUG] (runner) checking template dbf4757b63cf3c993ba13352766a3a8d
2022-09-16T10:36:59.308Z [DEBUG] (runner) missing data for 1 dependencies
2022-09-16T10:36:59.308Z [DEBUG] (runner) missing dependency: vault.pki(pki-int/issue/consul->/etc/consul.d/cert.pem)
2022-09-16T10:36:59.308Z [DEBUG] (runner) add used dependency vault.pki(pki-int/issue/consul->/etc/consul.d/cert.pem) to missing since isLeader but do not have a watcher
2022-09-16T10:36:59.308Z [DEBUG] (runner) was not watching 1 dependencies
2022-09-16T10:36:59.308Z [DEBUG] (watcher) adding vault.pki(pki-int/issue/consul->/etc/consul.d/cert.pem)
2022-09-16T10:36:59.308Z [DEBUG] (runner) checking template 4693ee512a16caefe7069d7f774e1615
2022-09-16T10:36:59.309Z [DEBUG] (runner) missing data for 1 dependencies
2022-09-16T10:36:59.309Z [DEBUG] (runner) missing dependency: vault.pki(pki-int/issue/consul->/etc/consul.d/ca.pem)
2022-09-16T10:36:59.309Z [DEBUG] (runner) add used dependency vault.pki(pki-int/issue/consul->/etc/consul.d/ca.pem) to missing since isLeader but do not have a watcher
2022-09-16T10:36:59.309Z [DEBUG] (runner) was not watching 1 dependencies
2022-09-16T10:36:59.309Z [DEBUG] (watcher) adding vault.pki(pki-int/issue/consul->/etc/consul.d/ca.pem)
2022-09-16T10:36:59.309Z [DEBUG] (runner) diffing and updating dependencies
2022-09-16T10:36:59.309Z [DEBUG] (runner) watching 3 dependencies
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x2b0 pc=0x2c6bd26]

goroutine 41 [running]:
github.com/hashicorp/consul-template/dependency.goodFor(0xc000df8e00)
    /home/runner/go/pkg/mod/github.com/hashicorp/consul-template@v0.29.2/dependency/vault_pki.go:114 +0x26
github.com/hashicorp/consul-template/dependency.(*VaultPKIQuery).Fetch.func1(0x0)
    /home/runner/go/pkg/mod/github.com/hashicorp/consul-template@v0.29.2/dependency/vault_pki.go:89 +0x125
github.com/hashicorp/consul-template/dependency.(*VaultPKIQuery).Fetch(0xc000e1eec0, 0xc000e201b0, 0xc000c1bf90)
    /home/runner/go/pkg/mod/github.com/hashicorp/consul-template@v0.29.2/dependency/vault_pki.go:96 +0x115
github.com/hashicorp/consul-template/watch.(*View).fetch(0xc000736280, 0x0, 0x0, 0x0)
    /home/runner/go/pkg/mod/github.com/hashicorp/consul-template@v0.29.2/watch/view.go:203 +0x131
created by github.com/hashicorp/consul-template/watch.(*View).poll
    /home/runner/go/pkg/mod/github.com/hashicorp/consul-template@v0.29.2/watch/view.go:117 +0x145
const-tmp commented 1 year ago

UPD reproduced on macOS Vault v1.11.3 (17250b25303c6418c283c95b1d5a9c9f16174fe8), built 2022-08-26T10:27:10Z

mpalmi commented 1 year ago

This has been addressed in: https://github.com/hashicorp/consul-template/pull/1639. Thanks for the report!

mpalmi commented 1 year ago

My apologies, I got a bit hasty. Opening this back up until the aforementioned fix is tagged and brought into Vault.

Planokur commented 1 year ago

I have the same issue. To mention that in debug log we can see that there were 

(runner) watching 3 dependencies

That means there are 3 different POST to the vault that result in 3 different cert/key. To be sure, inspect the CRT and key file, and you'll see that they didn't match. They result from 2 different issuing call from vault. Everything works well, just if you have been rendering all data in one template like in example below

{{ with pkiCert \"pki/issue/test\" \"common_name=test.test\" \"ttl=1m\" }}
{{ .Data.Key }}
{{ .Data.Cert }}
{{ end }}

In this case, we'll have just one.

 watching 1 dependencie

Using the secret module everything works well even with 2 or more templates in .hcl file and just one watching 1 dependencie appear in logs.