search

hashicorp / consul-template

Template rendering, notifier, and supervisor for @HashiCorp Consul and Vault data.
https://www.hashicorp.com/
Mozilla Public License 2.0
4.76k stars 781 forks source link

Haproxy configuration is blank #1167

Open rahul9999 opened 5 years ago

rahul9999 commented 5 years ago

Hi,

I have the following template:

 ]# cat template.tmpl

# HTTP Frontends
frontend http
    mode http
    bind 0.0.0.0:80
    option httplog

    {{range ls "marathon"}}
    {{with $root := .}}{{with $app := parseJSON $root.Value}}{{with $tasks := ls (printf "marathon/%s/tasks" $root.Key)}}
    {{if eq (print $app.labels.HAPROXY_HTTP) "true"}}
    # {{$root.Key}} ACLs
    {{range $index, $port := $app.ports}}{{with $localName := index $app.labels (printf "HTTP_PORT_IDX_%d_NAME" $index)}}
    acl host_{{$localName}} hdr_beg(host) -i {{$localName}}
    {{if and (eq (print $app.labels.HTTP_PATH) "true") $app.labels.HTTP_PATH_URL $app.labels.HTTP_PATH_URI}}
    acl host_{{printf $app.labels.HTTP_PATH_URL}} path_beg -i {{printf $app.labels.HTTP_PATH_URI}}
    use_backend {{printf $app.labels.HTTP_PATH_URL}}_backend if host_{{$localName}} host_{{printf $app.labels.HTTP_PATH_URL}}
    {{else}}
    use_backend {{$localName}}_backend if host_{{$localName}}
    {{end}}
    {{else}}
    # no HTTP forwarding information for {{$port}}
    {{end}}{{end}}
    {{end}}
    {{else}}
    # no tasks for {{$root.Key}}, not generating any HTTP ACLs
    {{end}}{{else}}{{end}}{{end}}{{/* end of with $root, $app, and $tasks */}}
    {{end}}{{/* end of range "marathon" */}}

Consul Config Dir

]# cat config/consul.cfg
consul = "xx.xx.xx.xx:8500"
]# cat config/haproxy.cfg
template {
  source = "/temp/consul-template/template.tmpl"
  destination = "/temp/consul-template/haproxy.cfg"
}

Here is the command to run Consul-template

./consul-template -config /temp/consul-template/config -log-level info -wait 2s:10s

And this the log generated ( info and debug level)

INFO

2019/01/17 10:04:14 [INFO] consul-template v0.14.0
2019/01/17 10:04:14 [INFO] (runner) creating new runner (dry: false, once: false)
2019/01/17 10:04:14 [INFO] (runner) creating consul/api client
2019/01/17 10:04:14 [INFO] (runner) creating vault/api client
2019/01/17 10:04:14 [INFO] (runner) creating Watcher
2019/01/17 10:04:14 [INFO] (runner) starting
2019/01/17 10:04:14 [INFO] (runner) running
2019/01/17 10:04:14 [INFO] (runner) was not watching 2 dependencies
2019/01/17 10:04:14 [INFO] (watcher) adding "storeKeyPrefix(marathon)"
2019/01/17 10:04:14 [INFO] (watcher) adding "storeKeyPrefix(marathon)"
2019/01/17 10:04:14 [INFO] (runner) diffing and updating dependencies
2019/01/17 10:04:14 [INFO] (runner) watching 1 dependencies
2019/01/17 10:04:14 [INFO] (view) "storeKeyPrefix(marathon)" received data
2019/01/17 10:04:14 [INFO] (runner) running
2019/01/17 10:04:14 [INFO] (runner) was not watching 8 dependencies
2019/01/17 10:04:14 [INFO] (watcher) adding "storeKeyPrefix(marathon/xxx-test-apps-sampleapptest/tasks)"
2019/01/17 10:04:14 [INFO] (watcher) adding "storeKeyPrefix(marathon/xxx-test-apps-test-cerebro/tasks)"
2019/01/17 10:04:14 [INFO] (watcher) adding "storeKeyPrefix(marathon/mesos-marathon-consul/tasks)"
2019/01/17 10:04:14 [INFO] (watcher) adding "storeKeyPrefix(marathon/xxx-haproxy-https/tasks)"
2019/01/17 10:04:14 [INFO] (watcher) adding "storeKeyPrefix(marathon/xxx-test-apps-test-cerebro/tasks)"
2019/01/17 10:04:14 [INFO] (watcher) adding "storeKeyPrefix(marathon/mesos-marathon-consul/tasks)"
2019/01/17 10:04:14 [INFO] (watcher) adding "storeKeyPrefix(marathon/xxx-haproxy-https/tasks)"
2019/01/17 10:04:14 [INFO] (watcher) adding "storeKeyPrefix(marathon/xxx-test-apps-sampleapptest/tasks)"
2019/01/17 10:04:14 [INFO] (runner) diffing and updating dependencies
2019/01/17 10:04:14 [INFO] (runner) watching 5 dependencies
2019/01/17 10:04:14 [INFO] (view) "storeKeyPrefix(marathon/xxx-haproxy-https/tasks)" received data
2019/01/17 10:04:14 [INFO] (runner) running
2019/01/17 10:04:14 [INFO] (view) "storeKeyPrefix(marathon/mesos-marathon-consul/tasks)" received data
2019/01/17 10:04:14 [INFO] (view) "storeKeyPrefix(marathon/xxx-test-apps-test-cerebro/tasks)" received data
2019/01/17 10:04:14 [INFO] (view) "storeKeyPrefix(marathon/xxx-test-apps-sampleapptest/tasks)" received data
2019/01/17 10:04:14 [INFO] (runner) missing data for 3 dependencies
2019/01/17 10:04:14 [INFO] (runner) diffing and updating dependencies
2019/01/17 10:04:14 [INFO] (runner) watching 5 dependencies
2019/01/17 10:04:14 [INFO] (runner) running
2019/01/17 10:04:14 [INFO] (runner) diffing and updating dependencies
2019/01/17 10:04:14 [INFO] (runner) watching 5 dependencies

DEBUG

2019/01/17 10:05:13 [INFO] (runner) creating consul/api client
2019/01/17 10:05:13 [DEBUG] (runner) setting consul address to xx.xx.xx.xx:8500
2019/01/17 10:05:13 [INFO] (runner) creating vault/api client
2019/01/17 10:05:13 [DEBUG] (runner) enabling vault SSL
2019/01/17 10:05:13 [INFO] (runner) creating Watcher
2019/01/17 10:05:13 [INFO] (runner) starting
2019/01/17 10:05:13 [DEBUG] (runner) running initial templates
2019/01/17 10:05:13 [INFO] (runner) running
2019/01/17 10:05:13 [DEBUG] (runner) checking template /temp/consul-template/template.tmpl
2019/01/17 10:05:13 [INFO] (runner) was not watching 2 dependencies
2019/01/17 10:05:13 [INFO] (watcher) adding "storeKeyPrefix(marathon)"
2019/01/17 10:05:13 [DEBUG] (watcher) "storeKeyPrefix(marathon)" starting
2019/01/17 10:05:13 [INFO] (watcher) adding "storeKeyPrefix(marathon)"
2019/01/17 10:05:13 [DEBUG] (watcher) "storeKeyPrefix(marathon)" already exists, skipping
2019/01/17 10:05:13 [INFO] (runner) diffing and updating dependencies
2019/01/17 10:05:13 [INFO] (runner) watching 1 dependencies
2019/01/17 10:05:13 [DEBUG] (view) "storeKeyPrefix(marathon)" starting fetch
2019/01/17 10:05:13 [DEBUG] ("storeKeyPrefix(marathon)") querying consul with &{Datacenter: AllowStale:true RequireConsistent:false WaitIndex:0 WaitTime:1m0s Token: Near:}
2019/01/17 10:05:13 [DEBUG] ("storeKeyPrefix(marathon)") Consul returned 9 key pairs
2019/01/17 10:05:13 [INFO] (view) "storeKeyPrefix(marathon)" received data
2019/01/17 10:05:13 [DEBUG] (view) "storeKeyPrefix(marathon)" starting fetch
2019/01/17 10:05:13 [DEBUG] ("storeKeyPrefix(marathon)") querying consul with &{Datacenter: AllowStale:true RequireConsistent:false WaitIndex:212907 WaitTime:1m0s Token: Near:}
2019/01/17 10:05:13 [DEBUG] (runner) receiving dependency "storeKeyPrefix(marathon)"
2019/01/17 10:05:13 [INFO] (runner) running
2019/01/17 10:05:13 [DEBUG] (runner) checking template /temp/consul-template/template.tmpl
2019/01/17 10:05:13 [INFO] (runner) was not watching 8 dependencies
2019/01/17 10:05:13 [INFO] (watcher) adding "storeKeyPrefix(marathon/mesos-marathon-consul/tasks)"
2019/01/17 10:05:13 [DEBUG] (watcher) "storeKeyPrefix(marathon/mesos-marathon-consul/tasks)" starting
2019/01/17 10:05:13 [INFO] (watcher) adding "storeKeyPrefix(marathon/xxx-haproxy-https/tasks)"
2019/01/17 10:05:13 [DEBUG] (watcher) "storeKeyPrefix(marathon/xxx-haproxy-https/tasks)" starting
2019/01/17 10:05:13 [INFO] (watcher) adding "storeKeyPrefix(marathon/xxx-test-apps-sampleapptest/tasks)"
2019/01/17 10:05:13 [DEBUG] (watcher) "storeKeyPrefix(marathon/xxx-test-apps-sampleapptest/tasks)" starting
2019/01/17 10:05:13 [INFO] (watcher) adding "storeKeyPrefix(marathon/xxx-test-apps-test-cerebro/tasks)"
2019/01/17 10:05:13 [DEBUG] (watcher) "storeKeyPrefix(marathon/xxxx-test-apps-test-cerebro/tasks)" starting
2019/01/17 10:05:13 [INFO] (watcher) adding "storeKeyPrefix(marathon/xxx-test-apps-sampleapptest/tasks)"
2019/01/17 10:05:13 [DEBUG] (watcher) "storeKeyPrefix(marathon/xxx-test-apps-sampleapptest/tasks)" already exists, skipping
2019/01/17 10:05:13 [DEBUG] (view) "storeKeyPrefix(marathon/mesos-marathon-consul/tasks)" starting fetch
2019/01/17 10:05:13 [DEBUG] (view) "storeKeyPrefix(marathon/xxx-test-apps-test-cerebro/tasks)" starting fetch
2019/01/17 10:05:13 [DEBUG] (view) "storeKeyPrefix(marathon/xxx-haproxy-https/tasks)" starting fetch
2019/01/17 10:05:13 [DEBUG] ("storeKeyPrefix(marathon/mesos-marathon-consul/tasks)") querying consul with &{Datacenter: AllowStale:true RequireConsistent:false WaitIndex:0 WaitTime:1m0s Token: Near:}
2019/01/17 10:05:13 [DEBUG] ("storeKeyPrefix(marathon/xxx-test-apps-test-cerebro/tasks)") querying consul with &{Datacenter: AllowStale:true RequireConsistent:false WaitIndex:0 WaitTime:1m0s Token: Near:}
2019/01/17 10:05:13 [INFO] (watcher) adding "storeKeyPrefix(marathon/xxx-test-apps-test-cerebro/tasks)"
2019/01/17 10:05:13 [DEBUG] ("storeKeyPrefix(marathon/xxx-haproxy-https/tasks)") querying consul with &{Datacenter: AllowStale:true RequireConsistent:false WaitIndex:0 WaitTime:1m0s Token: Near:}
2019/01/17 10:05:13 [DEBUG] (watcher) "storeKeyPrefix(marathon/xxx-test-apps-test-cerebro/tasks)" already exists, skipping
2019/01/17 10:05:13 [INFO] (watcher) adding "storeKeyPrefix(marathon/mesos-marathon-consul/tasks)"
2019/01/17 10:05:13 [DEBUG] (watcher) "storeKeyPrefix(marathon/mesos-marathon-consul/tasks)" already exists, skipping
2019/01/17 10:05:13 [INFO] (watcher) adding "storeKeyPrefix(marathon/xxx-haproxy-https/tasks)"
2019/01/17 10:05:13 [DEBUG] (watcher) "storeKeyPrefix(marathon/xxx-haproxy-https/tasks)" already exists, skipping
2019/01/17 10:05:13 [INFO] (runner) diffing and updating dependencies
2019/01/17 10:05:13 [DEBUG] (runner) "storeKeyPrefix(marathon)" is still needed
2019/01/17 10:05:13 [INFO] (runner) watching 5 dependencies
2019/01/17 10:05:13 [DEBUG] (view) "storeKeyPrefix(marathon/xxx-test-apps-sampleapptest/tasks)" starting fetch
2019/01/17 10:05:13 [DEBUG] ("storeKeyPrefix(marathon/xxx-test-apps-sampleapptest/tasks)") querying consul with &{Datacenter: AllowStale:true RequireConsistent:false WaitIndex:0 WaitTime:1m0s Token: Near:}
2019/01/17 10:05:13 [DEBUG] ("storeKeyPrefix(marathon/xxx-haproxy-https/tasks)") Consul returned 1 key pairs
2019/01/17 10:05:13 [INFO] (view) "storeKeyPrefix(marathon/xxx-haproxy-https/tasks)" received data
2019/01/17 10:05:13 [DEBUG] (view) "storeKeyPrefix(marathon/xxx-haproxy-https/tasks)" starting fetch
2019/01/17 10:05:13 [DEBUG] ("storeKeyPrefix(marathon/mesos-marathon-consul/tasks)") Consul returned 1 key pairs
2019/01/17 10:05:13 [INFO] (view) "storeKeyPrefix(marathon/mesos-marathon-consul/tasks)" received data
2019/01/17 10:05:13 [DEBUG] (view) "storeKeyPrefix(marathon/mesos-marathon-consul/tasks)" starting fetch
2019/01/17 10:05:13 [DEBUG] ("storeKeyPrefix(marathon/xxx-test-apps-test-cerebro/tasks)") Consul returned 1 key pairs
2019/01/17 10:05:13 [INFO] (view) "storeKeyPrefix(marathon/xxx-test-apps-test-cerebro/tasks)" received data
2019/01/17 10:05:13 [DEBUG] (view) "storeKeyPrefix(marathon/xxx-test-apps-test-cerebro/tasks)" starting fetch
2019/01/17 10:05:13 [DEBUG] ("storeKeyPrefix(marathon/mesos-marathon-consul/tasks)") querying consul with &{Datacenter: AllowStale:true RequireConsistent:false WaitIndex:210734 WaitTime:1m0s Token: Near:}
2019/01/17 10:05:13 [DEBUG] ("storeKeyPrefix(marathon/xxx-test-apps-sampleapptest/tasks)") Consul returned 2 key pairs
2019/01/17 10:05:13 [INFO] (view) "storeKeyPrefix(marathon/xxx-test-apps-sampleapptest/tasks)" received data
2019/01/17 10:05:13 [DEBUG] ("storeKeyPrefix(marathon/xxx-test-apps-test-cerebro/tasks)") querying consul with &{Datacenter: AllowStale:true RequireConsistent:false WaitIndex:210730 WaitTime:1m0s Token: Near:}
2019/01/17 10:05:13 [DEBUG] (view) "storeKeyPrefix(marathon/xxx-test-apps-sampleapptest/tasks)" starting fetch
2019/01/17 10:05:13 [DEBUG] ("storeKeyPrefix(marathon/xxx-haproxy-https/tasks)") querying consul with &{Datacenter: AllowStale:true RequireConsistent:false WaitIndex:212907 WaitTime:1m0s Token: Near:}
2019/01/17 10:05:13 [DEBUG] ("storeKeyPrefix(marathon/xxx-test-apps-sampleapptest/tasks)") querying consul with &{Datacenter: AllowStale:true RequireConsistent:false WaitIndex:212797 WaitTime:1m0s Token: Near:}
2019/01/17 10:05:13 [DEBUG] (runner) receiving dependency "storeKeyPrefix(marathon/xxx-haproxy-https/tasks)"
2019/01/17 10:05:13 [DEBUG] (runner) receiving dependency "storeKeyPrefix(marathon/mesos-marathon-consul/tasks)"
2019/01/17 10:05:13 [DEBUG] (runner) receiving dependency "storeKeyPrefix(marathon/xxx-test-apps-test-cerebro/tasks)"
2019/01/17 10:05:13 [DEBUG] (runner) receiving dependency "storeKeyPrefix(marathon/xxx-test-apps-sampleapptest/tasks)"
2019/01/17 10:05:13 [INFO] (runner) running
2019/01/17 10:05:13 [DEBUG] (runner) checking template /temp/consul-template/template.tmpl
2019/01/17 10:05:13 [DEBUG] (runner) checking ctemplate &{Source:/temp/consul-template/template.tmpl Destination:/temp/consul-template/haproxy.cfg Command: CommandTimeout:30s Perms:-rw-r--r-- Backup:false}
2019/01/17 10:05:13 [DEBUG] (runner) wouldRender: true, didRender: false
2019/01/17 10:05:13 [INFO] (runner) diffing and updating dependencies
2019/01/17 10:05:13 [DEBUG] (runner) "storeKeyPrefix(marathon/xxx-test-apps-sampleapptest/tasks)" is still needed
2019/01/17 10:05:13 [DEBUG] (runner) "storeKeyPrefix(marathon/xxx-test-apps-test-cerebro/tasks)" is still needed
2019/01/17 10:05:13 [DEBUG] (runner) "storeKeyPrefix(marathon/mesos-marathon-consul/tasks)" is still needed
2019/01/17 10:05:13 [DEBUG] (runner) "storeKeyPrefix(marathon)" is still needed
2019/01/17 10:05:13 [DEBUG] (runner) "storeKeyPrefix(marathon/xxx-haproxy-https/tasks)" is still needed
2019/01/17 10:05:13 [INFO] (runner) watching 5 dependencies

The haproxy.cfg is coming blank

Consul Template version

consul-template v0.14.0

Consul Version

Consul v0.8.3

Marathon Version

1.7.X

Actual behavior

It should create one ACL and corresponding Backend entry in haproxy.cfg
(One of the app has this Marathon label true HAPROXY_HTTP)

Please assist and let me know if you need any other info from my side.

Thanks Rahul

eikenb commented 5 years ago

Hey @rahul9999, thanks for taking the time to file your issue and sorry for the delayed response.

If you are still having this issue or still have the data can you tell me what's being stored under that marathon key in consul? I can run the test without the marathon part, but I cannot reproduce the issue doing that. Thanks.