search

hashicorp / consul-template

Template rendering, notifier, and supervisor for @HashiCorp Consul and Vault data.
https://www.hashicorp.com/
Mozilla Public License 2.0
4.76k stars 781 forks source link

Running inside docker container, /usr/local/bin binaries cannot be found. PATH misconfigured? #928

Closed gtrias closed 7 years ago

gtrias commented 7 years ago

Consul Template version

0.18.2

Configuration

consul {
  address = "consul.service.consul:8500"

  retry {
    enabled = true

    attempts = 5

    backoff = "250ms"
  }
}

wait {
  min = "5s"
  max = "10s"
}

vault {
  address = "http://vault.service.consul:8200"

  renew_token = true

  retry {
    # ...
  }
}

exec {
  command = "npm start --prefix /app"

  splay = "5s"

  env {
    pristine = false

    whitelist = ["CONSUL_*"]

    blacklist = ["VAULT_*"]
  }

  reload_signal = ""

  kill_signal = "SIGINT"

  kill_timeout = "2s"
}

template {
  source = "/app/consul/datasources.production.ctmpl.json"

  destination = "/app/server/datasources.production.json"

  backup = true
}
{
  "mongoDB": {
    "host": "mongodb01.service.consul",
    "port": 27017,
    "name": "mongoDB",
    "connector": "mongodb",
    {{with secret "mongodb/creds/holidays_master"}}
    "username": "{{ .Data.username }}",
    "password": "{{ .Data.password }}",
    "database": "{{ .Data.db }}"
    {{end}}
  }
}

Command

#!/bin/bash

/usr/local/bin/consul-template -config '/app/consul/config.hcl'

Debug output

2017/04/27 15:40:21.079629 [INFO] consul-template v0.18.2 (ecb9e46)
2017/04/27 15:40:21.079694 [INFO] (runner) creating new runner (dry: false, once: false)
2017/04/27 15:40:21.079938 [DEBUG] (runner) final config: {"Consul":{"Address":"consul.service.consul:8500","Auth":{"Enabled":false,"Username":"","Password":""},"Retry":{"Attempts":5,"Ba
ckoff":250000000,"Enabled":true},"SSL":{"CaCert":"","CaPath":"","Cert":"","Enabled":false,"Key":"","ServerName":"","Verify":true},"Token":"","Transport":{"DialKeepAlive":30000000000,"Dia
lTimeout":30000000000,"DisableKeepAlives":false,"IdleConnTimeout":90000000000,"MaxIdleConns":100,"MaxIdleConnsPerHost":2,"TLSHandshakeTimeout":10000000000}},"Dedup":{"Enabled":false,"Max
Stale":2000000000,"Prefix":"consul-template/dedup/","TTL":15000000000},"Exec":{"Command":"/app/entrypoint.sh","Enabled":true,"Env":{"Blacklist":["VAULT_*"],"Custom":[],"Pristine":false,"
Whitelist":["CONSUL_*"]},"KillSignal":2,"KillTimeout":2000000000,"ReloadSignal":0,"Splay":5000000000,"Timeout":0},"KillSignal":2,"LogLevel":"trace","MaxStale":2000000000,"PidFile":"","Re
loadSignal":1,"Syslog":{"Enabled":false,"Facility":"LOCAL0"},"Templates":[{"Backup":true,"Command":"","CommandTimeout":30000000000,"Contents":"","Destination":"/app/server/datasources.pr
oduction.json","Exec":{"Command":"","Enabled":false,"Env":{"Blacklist":[],"Custom":[],"Pristine":false,"Whitelist":[]},"KillSignal":2,"KillTimeout":30000000000,"ReloadSignal":null,"Splay
":0,"Timeout":30000000000},"Perms":420,"Source":"/app/consul/datasources.production.ctmpl.json","Wait":{"Enabled":false,"Min":0,"Max":0},"LeftDelim":"","RightDelim":""}],"Vault":{"Addres
s":"http://vault.service.consul:8200","Enabled":true,"RenewToken":true,"Retry":{"Attempts":5,"Backoff":250000000,"Enabled":true},"SSL":{"CaCert":"","CaPath":"","Cert":"","Enabled":true,"
Key":"","ServerName":"","Verify":true},"Transport":{"DialKeepAlive":30000000000,"DialTimeout":30000000000,"DisableKeepAlives":false,"IdleConnTimeout":90000000000,"MaxIdleConns":100,"MaxI
dleConnsPerHost":2,"TLSHandshakeTimeout":10000000000},"UnwrapToken":false},"Wait":{"Enabled":true,"Min":5000000000,"Max":10000000000}}
2017/04/27 15:40:21.080101 [INFO] (runner) creating watcher
2017/04/27 15:40:21.080134 [DEBUG] (watcher) adding vault.token
2017/04/27 15:40:21.080145 [TRACE] (watcher) vault.token starting
2017/04/27 15:40:21.080428 [TRACE] (view) vault.token starting fetch
2017/04/27 15:40:21.080546 [TRACE] vault.token: GET /v1/auth/token/renew-self?stale=true&wait=1m0s
2017/04/27 15:40:21.080606 [INFO] (runner) starting
2017/04/27 15:40:21.080616 [DEBUG] (runner) running initial templates
2017/04/27 15:40:21.080720 [INFO] (runner) initiating run
2017/04/27 15:40:21.080730 [DEBUG] (runner) checking template 951b4fce62e2603b864b12694ddb32e9
2017/04/27 15:40:21.080916 [DEBUG] (runner) was not watching 1 dependencies
2017/04/27 15:40:21.080936 [DEBUG] (watcher) adding vault.read(mongodb/creds/holidays_master)
2017/04/27 15:40:21.080945 [TRACE] (watcher) vault.read(mongodb/creds/holidays_master) starting
2017/04/27 15:40:21.080954 [DEBUG] (runner) diffing and updating dependencies
2017/04/27 15:40:21.080962 [DEBUG] (runner) enabling global quiescence for "951b4fce62e2603b864b12694ddb32e9"
2017/04/27 15:40:21.080979 [DEBUG] (runner) watching 2 dependencies
2017/04/27 15:40:21.081003 [TRACE] (view) vault.read(mongodb/creds/holidays_master) starting fetch
2017/04/27 15:40:21.081019 [TRACE] vault.read(mongodb/creds/holidays_master): GET /v1/mongodb/creds/holidays_master?stale=true&wait=1m0s
2017/04/27 15:40:21.445172 [DEBUG] vault.token: renewed token
2017/04/27 15:40:21.445237 [TRACE] (view) vault.token received data
2017/04/27 15:40:21.445254 [TRACE] (view) vault.token starting fetch
2017/04/27 15:40:21.445290 [TRACE] vault.token: GET /v1/auth/token/renew-self?index=1493307621&stale=true&wait=1m0s
2017/04/27 15:40:21.445300 [TRACE] vault.token: long polling for 382h3m40s
2017/04/27 15:40:21.445312 [INFO] (runner) initiating run
2017/04/27 15:40:21.445319 [DEBUG] (runner) checking template 951b4fce62e2603b864b12694ddb32e9
2017/04/27 15:40:21.445528 [DEBUG] (runner) missing data for 1 dependencies
2017/04/27 15:40:21.445546 [DEBUG] (runner) diffing and updating dependencies
2017/04/27 15:40:21.445558 [DEBUG] (runner) vault.read(mongodb/creds/holidays_master) is still needed
2017/04/27 15:40:21.445569 [DEBUG] (runner) watching 2 dependencies
2017/04/27 15:40:22.138168 [TRACE] (view) vault.read(mongodb/creds/holidays_master) received data
2017/04/27 15:40:22.138222 [TRACE] (view) vault.read(mongodb/creds/holidays_master) starting fetch
2017/04/27 15:40:22.138236 [TRACE] vault.read(mongodb/creds/holidays_master): long polling for 384h0m0s
2017/04/27 15:40:22.138250 [DEBUG] (runner) receiving dependency vault.read(mongodb/creds/holidays_master)
2017/04/27 15:40:22.138259 [INFO] (runner) initiating run
2017/04/27 15:40:22.138279 [DEBUG] (runner) checking template 951b4fce62e2603b864b12694ddb32e9
2017/04/27 15:40:22.138483 [DEBUG] (runner) diffing and updating dependencies
2017/04/27 15:40:22.138514 [DEBUG] (runner) vault.read(mongodb/creds/holidays_master) is still needed
2017/04/27 15:40:22.138523 [DEBUG] (runner) watching 2 dependencies
2017/04/27 15:40:27.138713 [DEBUG] (runner) received template "951b4fce62e2603b864b12694ddb32e9" from quiescence
2017/04/27 15:40:27.138785 [INFO] (runner) initiating run
2017/04/27 15:40:27.138796 [DEBUG] (runner) checking template 951b4fce62e2603b864b12694ddb32e9
2017/04/27 15:40:27.139064 [DEBUG] (runner) rendering "/app/consul/datasources.production.ctmpl.json" => "/app/server/datasources.production.json"
2017/04/27 15:40:27.315325 [INFO] (runner) rendered "/app/consul/datasources.production.ctmpl.json" => "/app/server/datasources.production.json"
2017/04/27 15:40:27.315411 [DEBUG] (runner) diffing and updating dependencies
2017/04/27 15:40:27.315442 [DEBUG] (runner) vault.read(mongodb/creds/holidays_master) is still needed
2017/04/27 15:40:27.315451 [DEBUG] (runner) enabling global quiescence for "951b4fce62e2603b864b12694ddb32e9"
2017/04/27 15:40:27.315459 [DEBUG] (runner) watching 2 dependencies
2017/04/27 15:40:27.315517 [INFO] (child) spawning: /app/entrypoint.sh
/usr/bin/env: node: No such file or directory
2017/04/27 15:40:27.317682 [INFO] (runner) child process died
Consul Template returned errors:
child process died with exit code 127

If I make a link to node as follows ln -s /usr/local/bin/node /usr/bin/node I get the following:

2017/04/27 15:34:28.914051 [INFO] consul-template v0.18.2 (ecb9e46)
2017/04/27 15:34:28.914132 [INFO] (runner) creating new runner (dry: false, once: false)
2017/04/27 15:34:28.914417 [DEBUG] (runner) final config: {"Consul":{"Address":"consul.service.consul:8500","Auth":{"Enabled":false,"Username":"","Password":""},"Retry":{"Attempts":5,"Ba
ckoff":250000000,"Enabled":true},"SSL":{"CaCert":"","CaPath":"","Cert":"","Enabled":false,"Key":"","ServerName":"","Verify":true},"Token":"","Transport":{"DialKeepAlive":30000000000,"Dia
lTimeout":30000000000,"DisableKeepAlives":false,"IdleConnTimeout":90000000000,"MaxIdleConns":100,"MaxIdleConnsPerHost":2,"TLSHandshakeTimeout":10000000000}},"Dedup":{"Enabled":false,"Max
Stale":2000000000,"Prefix":"consul-template/dedup/","TTL":15000000000},"Exec":{"Command":"/app/entrypoint.sh","Enabled":true,"Env":{"Blacklist":["VAULT_*"],"Custom":[],"Pristine":false,"
Whitelist":["CONSUL_*"]},"KillSignal":2,"KillTimeout":2000000000,"ReloadSignal":0,"Splay":5000000000,"Timeout":0},"KillSignal":2,"LogLevel":"trace","MaxStale":2000000000,"PidFile":"","Re
loadSignal":1,"Syslog":{"Enabled":false,"Facility":"LOCAL0"},"Templates":[{"Backup":true,"Command":"","CommandTimeout":30000000000,"Contents":"","Destination":"/app/server/datasources.pr
oduction.json","Exec":{"Command":"","Enabled":false,"Env":{"Blacklist":[],"Custom":[],"Pristine":false,"Whitelist":[]},"KillSignal":2,"KillTimeout":30000000000,"ReloadSignal":null,"Splay
":0,"Timeout":30000000000},"Perms":420,"Source":"/app/consul/datasources.production.ctmpl.json","Wait":{"Enabled":false,"Min":0,"Max":0},"LeftDelim":"","RightDelim":""}],"Vault":{"Addres
s":"http://vault.service.consul:8200","Enabled":true,"RenewToken":true,"Retry":{"Attempts":5,"Backoff":250000000,"Enabled":true},"SSL":{"CaCert":"","CaPath":"","Cert":"","Enabled":true,"
Key":"","ServerName":"","Verify":true},"Transport":{"DialKeepAlive":30000000000,"DialTimeout":30000000000,"DisableKeepAlives":false,"IdleConnTimeout":90000000000,"MaxIdleConns":100,"MaxI
dleConnsPerHost":2,"TLSHandshakeTimeout":10000000000},"UnwrapToken":false},"Wait":{"Enabled":true,"Min":5000000000,"Max":10000000000}}
2017/04/27 15:34:28.914683 [INFO] (runner) creating watcher
2017/04/27 15:34:28.914709 [DEBUG] (watcher) adding vault.token
2017/04/27 15:34:28.914721 [TRACE] (watcher) vault.token starting
2017/04/27 15:34:28.914994 [TRACE] (view) vault.token starting fetch
2017/04/27 15:34:28.920387 [TRACE] vault.token: GET /v1/auth/token/renew-self?stale=true&wait=1m0s
2017/04/27 15:34:28.920570 [INFO] (runner) starting
2017/04/27 15:34:28.920600 [DEBUG] (runner) running initial templates
2017/04/27 15:34:28.920770 [INFO] (runner) initiating run
2017/04/27 15:34:28.920784 [DEBUG] (runner) checking template 951b4fce62e2603b864b12694ddb32e9
2017/04/27 15:34:28.921003 [DEBUG] (runner) was not watching 1 dependencies
2017/04/27 15:34:28.921028 [DEBUG] (watcher) adding vault.read(mongodb/creds/holidays_master)
2017/04/27 15:34:28.921039 [TRACE] (watcher) vault.read(mongodb/creds/holidays_master) starting
2017/04/27 15:34:28.921049 [DEBUG] (runner) diffing and updating dependencies
2017/04/27 15:34:28.921057 [DEBUG] (runner) enabling global quiescence for "951b4fce62e2603b864b12694ddb32e9"
2017/04/27 15:34:28.921065 [DEBUG] (runner) watching 2 dependencies
2017/04/27 15:34:28.921105 [TRACE] (view) vault.read(mongodb/creds/holidays_master) starting fetch
2017/04/27 15:34:28.921122 [TRACE] vault.read(mongodb/creds/holidays_master): GET /v1/mongodb/creds/holidays_master?stale=true&wait=1m0s
2017/04/27 15:34:29.203811 [DEBUG] vault.token: renewed token
2017/04/27 15:34:29.203938 [TRACE] (view) vault.token received data
2017/04/27 15:34:29.203982 [TRACE] (view) vault.token starting fetch
2017/04/27 15:34:29.204009 [TRACE] vault.token: GET /v1/auth/token/renew-self?index=1493307269&stale=true&wait=1m0s
2017/04/27 15:34:29.204020 [TRACE] vault.token: long polling for 382h6m36s
2017/04/27 15:34:29.204044 [INFO] (runner) initiating run
2017/04/27 15:34:29.204052 [DEBUG] (runner) checking template 951b4fce62e2603b864b12694ddb32e9
2017/04/27 15:34:29.204264 [DEBUG] (runner) missing data for 1 dependencies
2017/04/27 15:34:29.204292 [DEBUG] (runner) diffing and updating dependencies
2017/04/27 15:34:29.204301 [DEBUG] (runner) vault.read(mongodb/creds/holidays_master) is still needed
2017/04/27 15:34:29.204310 [DEBUG] (runner) watching 2 dependencies
2017/04/27 15:34:29.438557 [TRACE] (view) vault.read(mongodb/creds/holidays_master) received data
2017/04/27 15:34:29.438579 [TRACE] (view) vault.read(mongodb/creds/holidays_master) starting fetch
2017/04/27 15:34:29.438583 [TRACE] vault.read(mongodb/creds/holidays_master): long polling for 384h0m0s
2017/04/27 15:34:29.438591 [DEBUG] (runner) receiving dependency vault.read(mongodb/creds/holidays_master)
2017/04/27 15:34:29.438593 [INFO] (runner) initiating run
2017/04/27 15:34:29.438597 [DEBUG] (runner) checking template 951b4fce62e2603b864b12694ddb32e9
2017/04/27 15:34:29.438726 [DEBUG] (runner) diffing and updating dependencies
2017/04/27 15:34:29.438732 [DEBUG] (runner) vault.read(mongodb/creds/holidays_master) is still needed
2017/04/27 15:34:29.438735 [DEBUG] (runner) watching 2 dependencies
2017/04/27 15:34:34.438839 [DEBUG] (runner) received template "951b4fce62e2603b864b12694ddb32e9" from quiescence
2017/04/27 15:34:34.438851 [INFO] (runner) initiating run
2017/04/27 15:34:34.438854 [DEBUG] (runner) checking template 951b4fce62e2603b864b12694ddb32e9
2017/04/27 15:34:34.439006 [DEBUG] (runner) rendering "/app/consul/datasources.production.ctmpl.json" => "/app/server/datasources.production.json"
2017/04/27 15:34:34.742102 [INFO] (runner) rendered "/app/consul/datasources.production.ctmpl.json" => "/app/server/datasources.production.json"
2017/04/27 15:34:34.742113 [DEBUG] (runner) diffing and updating dependencies
2017/04/27 15:34:34.742119 [DEBUG] (runner) vault.read(mongodb/creds/holidays_master) is still needed
2017/04/27 15:34:34.742121 [DEBUG] (runner) enabling global quiescence for "951b4fce62e2603b864b12694ddb32e9"
2017/04/27 15:34:34.742125 [DEBUG] (runner) watching 2 dependencies
2017/04/27 15:34:34.742154 [INFO] (child) spawning: /app/entrypoint.sh
npm WARN lifecycle npm is using /usr/local/bin/node but there is no node binary in the current PATH. Use the `--scripts-prepend-node-path` option to include the path for the node binary npm was executed with.

> holidays-api@1.0.0 start /app
> node server/server.js

npm ERR! Linux 3.16.0-4-amd64
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "start"
npm ERR! node v7.7.4
npm ERR! npm  v4.1.2
npm ERR! file sh
npm ERR! path sh
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! syscall spawn sh
npm ERR! holidays-api@1.0.0 start: `node server/server.js`
npm ERR! spawn sh ENOENT
npm ERR!
npm ERR! Failed at the holidays-api@1.0.0 start script 'node server/server.js'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the holidays-api package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node server/server.js
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs holidays-api
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls holidays-api
npm ERR! There is likely additional logging output above.
npm ERR! Linux 3.16.0-4-amd64
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "start"
npm ERR! node v7.7.4
npm ERR! npm  v4.1.2
npm ERR! code ELIFECYCLE

npm ERR! holidays-api@1.0.0 start: `node server/server.js`
npm ERR! Exit status -2
npm ERR!
npm ERR! Failed at the holidays-api@1.0.0 start script 'node server/server.js'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the holidays-api package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node server/server.js
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs holidays-api
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls holidays-api
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /app/npm-debug.log
2017/04/27 15:34:38.060764 [INFO] (runner) child process died
Consul Template returned errors:
child process died with exit code 1

Expected behavior

The nodejs application should run after the template has been generated withouth any problem to find node binary from Docker official node image.

Actual behavior

Steps to reproduce

  1. run a consul agent listening in 0.0.0.0
  2. consul kv put issue/test/value 23
  3. git clone https://github.com/gtrias/consul-template-issue && cd consul-template-issue
  4. docker build . -t issue
  5. docker run issue

Result:

/usr/bin/env: node: No such file or directory
Consul Template returned errors:
child process died with exit code 127

References

I didn't found any reference about this behavior.

sethvargo commented 7 years ago

Hi @gtrias

You are specifying a whitelist and blacklist:

whitelist = ["CONSUL_*"]
blacklist = ["VAULT_*"]

This means only CONSUL_* envvars will pull through to the child process. Things like PATH would not be included, which is likely why you are seeing this behavior.

gtrias commented 7 years ago

OMG you're right! I misunderstood the meaning of whitelist in combination with pristine. I thought it only makes sense with pristine enabled..

Thanks a lot! I close the issue then :)