Updating loop when using {{ datetime }}

[analytically]

2015-03-17T12:27:01Z lb1 confd[13]: INFO Target config /etc/haproxy-ddos/haproxy.cfg out of sync
2015-03-17T12:27:03Z lb1 confd[13]: INFO Target config /etc/haproxy-ddos/haproxy.cfg has been updated
2015-03-17T12:27:13Z lb1 confd[13]: INFO /etc/haproxy-ddos/haproxy.cfg has md5sum fab9fa9a3fb9bc0e211c90880b1586a7 should be 1e541f15b6b2a8273ca1715aa8fb0a09
2015-03-17T12:27:13Z lb1 confd[13]: INFO Target config /etc/haproxy-ddos/haproxy.cfg out of sync
2015-03-17T12:27:17Z lb1 confd[13]: INFO Target config /etc/haproxy-ddos/haproxy.cfg has been updated
2015-03-17T12:27:27Z lb1 confd[13]: INFO /etc/haproxy-ddos/haproxy.cfg has md5sum 1e541f15b6b2a8273ca1715aa8fb0a09 should be 1d18ce0c261873bead2b789cc8152fc4
2015-03-17T12:27:27Z lb1 confd[13]: INFO Target config /etc/haproxy-ddos/haproxy.cfg out of sync
2015-03-17T12:27:30Z lb1 confd[13]: INFO Target config /etc/haproxy-ddos/haproxy.cfg has been updated
2015-03-17T12:27:40Z lb1 confd[13]: INFO /etc/haproxy-ddos/haproxy.cfg has md5sum 1d18ce0c261873bead2b789cc8152fc4 should be 70b71fc04b4ea7f8f02ae2b400cf6db8
2015-03-17T12:27:40Z lb1 confd[13]: INFO Target config /etc/haproxy-ddos/haproxy.cfg out of sync
2015-03-17T12:27:43Z lb1 confd[13]: INFO Target config /etc/haproxy-ddos/haproxy.cfg has been updated
2015-03-17T12:27:53Z lb1 confd[13]: INFO /etc/haproxy-ddos/haproxy.cfg has md5sum 70b71fc04b4ea7f8f02ae2b400cf6db8 should be 50fbc1e54959bd1334d7fa8d9e3c5d0d
2015-03-17T12:27:53Z lb1 confd[13]: INFO Target config /etc/haproxy-ddos/haproxy.cfg out of sync
2015-03-17T12:27:56Z lb1 confd[13]: INFO Target config /etc/haproxy-ddos/haproxy.cfg has been updated
2015-03-17T12:28:06Z lb1 confd[13]: INFO /etc/haproxy-ddos/haproxy.cfg has md5sum 50fbc1e54959bd1334d7fa8d9e3c5d0d should be d7000eb5df3489430b3e67d3de666c78

We're seeing this. Any idea?

[analytically]

# Generated using confd {{datetime}}

Could this be the culprit?

[analytically]

Yep it seems like it.

[battlecow]

+1

[bacongobbler]

Well, considering that {{ datetime }} just outputs time.Now(), I'm wondering if there's a way we can somehow hook into the time when we boot confd and pipe that to datetime instead. Would that work for you?

[dnelson]

More helpful, although probably more complicated to pull off, would be to update the time whenever there are other changes.

[mishak87]

Would ignoring lines starting with # and so on help? It would require implementing setting ignoreLines=^#.

[kelseyhightower]

This is working as designed. The template is only triggered when there is an updated key in the configuration store. If you call the datetime function in a loop you are going to get a new timestamp for each call.

This would be a bug if templates are being process when there are no changes in the configuration backend.

[icheko]

I ran into this issue as well. When {{datetime}} is part of the source template, it causes the reload_cmd to be triggered (and thus haproxy keeps reloading) without any changes to the keys being monitored.

haproxy.toml

[template]
src = "haproxy.cfg.tmpl"
dest = "/etc/haproxy/haproxy.cfg"
keys = [
        "/skydns/tf/csa/services/sample-app"
]
reload_cmd = "echo restarting && /usr/bin/systemctl reload haproxy"

haproxy.cfg.tmpl

...

backend sample_cluster

    # Generated by confd {{datetime}}
    {{range gets "/skydns/tf/csa/services/sample-app/*"}}
    {{$data := json .Value}}
        server {{ base .Key}} {{$data.host}}:{{$data.port}} check
    {{end}}

/usr/bin/confd[21]: DEBUG "restarting\n"
/usr/bin/confd[21]: INFO Target config /etc/haproxy/haproxy.cfg has been updated
/usr/bin/confd[21]: DEBUG Loading template resources from confdir /etc/confd
/usr/bin/confd[21]: DEBUG Loading template resource from /etc/confd/conf.d/haproxy.toml
/usr/bin/confd[21]: DEBUG Retrieving keys from store
/usr/bin/confd[21]: DEBUG Key prefix set to /
/usr/bin/confd[21]: DEBUG Using source template /etc/confd/templates/haproxy.cfg.tmpl
/usr/bin/confd[21]: DEBUG Compiling source template /etc/confd/templates/haproxy.cfg.tmpl
/usr/bin/confd[21]: DEBUG Comparing candidate config to /etc/haproxy/haproxy.cfg
/usr/bin/confd[21]: INFO /etc/haproxy/haproxy.cfg has md5sum 12edc21a01e8071a55395078bc849c5f should be 7d6b9bb9fc883d04a33d919e022f7254
/usr/bin/confd[21]: INFO Target config /etc/haproxy/haproxy.cfg out of sync
/usr/bin/confd[21]: DEBUG Overwriting target config /etc/haproxy/haproxy.cfg
/usr/bin/confd[21]: DEBUG Running echo restarting && /usr/bin/systemctl reload haproxy
/usr/bin/confd[21]: DEBUG "restarting\n"
/usr/bin/confd[21]: INFO Target config /etc/haproxy/haproxy.cfg has been updated

[bacongobbler]

This would be a bug if templates are being process when there are no changes in the configuration backend.

I think that is what's happening here, judging from @icheko and @analytically's comments. I also tried testing this and verified it's a bug. It's because the md5sum changes any time the template is evaluated because time.Now() is different every interval, causing the md5sum to always be different compared to its predecessor. I'm not sure there's an easy answer on how we fix this behaviour.

[mishak87]

@bacongobbler Setting datetime to write special value and replacing it after computing md5 sum would not work?