search

mrwilson / helixdns

DNS server that serves records from etcd.
MIT License
94 stars 10 forks source link

Panic in setting up watch on /helix. Is etcd running at http://172.30.240.56:4001/ ? #15

Open djannot opened 9 years ago

djannot commented 9 years ago

I always have this error 5 minutes after helixdns is started.

During these 5 first minutes, everything is working fine.

laserpony commented 9 years ago

I'm experiencing this issue when I start HelixDNS using docker and systemd on CoreOS alpha 709.0.0 or CentOS 7.1, pointed to an etcd cluster running on CoreOS (etcd version 2.0.11). HelixDNS starts and runs fine if I execute the command in the service file directly. When started with systemctl, it panics with a complaint about setting up the watch on /helix within a second or so of starting.

Here are the instructions to reproduce from my CentOS 7.1 machine (with etcd running on CoreOS at 10.0.0.72):

Executing docker command directly works fine (it also works fine with the -d option):

$ sudo /usr/bin/docker run -p 53:53/udp --name helixdns probablyfine/helixdns helixdns -etcd-address="http://10.0.0.72:4001/" -forward="8.8.8.8:53" -port=53
2015/06/13 20:27:27 Starting server...
2015/06/13 20:27:27 Setting up watch to validate entries
...
2015/06/13 20:30:27 Could not get record for google.com., forwarding to 8.8.8.8:53

After CTRL-C, create the following hdns.service file in /etc/systemd/system/

Description=HelixDNS
After=docker.service
Requires=docker.service

[Service]
TimeoutStartSec=0
ExecStartPre=-/usr/bin/docker kill helixdns
ExecStartPre=-/usr/bin/docker rm helixdns
ExecStartPre=-/usr/bin/docker pull probablyfine/helixdns
ExecStart=/usr/bin/docker run -p 53:53/udp --name helixdns probablyfine/helixdns helixdns -etcd-address="http://10.0.0.72:4001/" -forward="8.8.8.8:53" -port=53
ExecStop=/usr/bin/docker stop helixdns

[Install]
WantedBy=multi-user.target
$ sudo systemctl start hdns && sudo systemctl status hdns
hdns.service - HelixDNS
   Loaded: loaded (/etc/systemd/system/hdns.service; enabled)
   Active: active (running) since Sat 2015-06-13 16:33:38 EDT; 19ms ago
  Process: 14618 ExecStop=/usr/bin/docker stop helixdns (code=exited, status=0/SUCCESS)
  Process: 14666 ExecStartPre=/usr/bin/docker pull probablyfine/helixdns (code=exited, status=0/SUCCESS)
  Process: 14638 ExecStartPre=/usr/bin/docker rm helixdns (code=exited, status=0/SUCCESS)
  Process: 14632 ExecStartPre=/usr/bin/docker kill helixdns (code=exited, status=0/SUCCESS)
 Main PID: 14673 (docker)
   CGroup: /system.slice/hdns.service
           └─14673 /usr/bin/docker run -p 53:53/udp --name helixdns probablyfine/helixdns helixdns -etcd-address="ht...

Jun 13 16:33:38 redacted docker[14666]: 2a0caa65c6f6: Download complete
Jun 13 16:33:38 redacted docker[14666]: 39da82e14986: Download complete
Jun 13 16:33:38 redacted docker[14666]: 5cceb9646089: Download complete
Jun 13 16:33:38 redacted docker[14666]: 83c7dc80d8f3: Download complete
Jun 13 16:33:38 redacted docker[14666]: 3d55156e7153: Download complete
Jun 13 16:33:38 redacted docker[14666]: afbe3b42ed3b: Download complete
Jun 13 16:33:38 redacted docker[14666]: 420f1e33f1ed: Download complete
Jun 13 16:33:38 redacted docker[14666]: 420f1e33f1ed: Download complete
Jun 13 16:33:38 redacted docker[14666]: Status: Image is up to date for docker.io/probablyfine/helixdns:latest
Jun 13 16:33:38 redacted systemd[1]: Started HelixDNS.

After that, HelixDNS fails.

$ sudo systemctl status hdns -l

hdns.service - HelixDNS
   Loaded: loaded (/etc/systemd/system/hdns.service; enabled)
   Active: failed (Result: exit-code) since Sat 2015-06-13 16:33:45 EDT; 57s ago
  Process: 14787 ExecStop=/usr/bin/docker stop helixdns (code=exited, status=0/SUCCESS)
  Process: 14673 ExecStart=/usr/bin/docker run -p 53:53/udp --name helixdns probablyfine/helixdns helixdns -etcd-address="http://10.0.0.72:4001/" -forward="8.8.8.8:53" -port=53 (code=exited, status=1/FAILURE)
  Process: 14666 ExecStartPre=/usr/bin/docker pull probablyfine/helixdns (code=exited, status=0/SUCCESS)
  Process: 14638 ExecStartPre=/usr/bin/docker rm helixdns (code=exited, status=0/SUCCESS)
  Process: 14632 ExecStartPre=/usr/bin/docker kill helixdns (code=exited, status=0/SUCCESS)
 Main PID: 14673 (code=exited, status=1/FAILURE)

Jun 13 16:33:38 redacted docker[14666]: 420f1e33f1ed: Download complete
Jun 13 16:33:38 redacted docker[14666]: 420f1e33f1ed: Download complete
Jun 13 16:33:38 redacted docker[14666]: Status: Image is up to date for docker.io/probablyfine/helixdns:latest
Jun 13 16:33:38 redacted systemd[1]: Started HelixDNS.
Jun 13 16:33:45 redacted docker[14673]: 2015/06/13 20:33:45 Starting server...
Jun 13 16:33:45 redacted docker[14673]: 2015/06/13 20:33:45 Setting up watch to validate entries
Jun 13 16:33:45 redacted docker[14673]: 2015/06/13 20:33:45 Panic in setting up watch on /helix. Is etcd running at "http://10.0.0.72:4001/" ?
floriandejonckheere commented 9 years ago

The error is caused by etcd 0.4, see coreos/go-etcd#170.