search

etcd-io / etcd

Distributed reliable key-value store for the most critical data of a distributed system
https://etcd.io
Apache License 2.0
47.81k stars 9.76k forks source link

what does it mean for end user in v3? warning - expected IP in URL for binding #7528

Closed raoofm closed 7 years ago

raoofm commented 7 years ago

After upgrading to 3.1.3 I see the below warnings. What does it mean to the end user? Will it not be supported in future and should we remove it from our config.

Wondering as the config to start etcd are coming from a configuration management tool, then what if the ip is unknown at the start and discovery is based on SRV

2017-03-17 15:39:13.211954 W | embed: expected IP in URL for binding (https://node02.qa.example.com:2380) 2017-03-17 15:39:13.212044 W | embed: expected IP in URL for binding (https://node02.qa.example.com:2379)

.

Full logs

[rm@etcd-vm-01 ~]$ tail -1000f /var/log/etcd/etcd.out 2017-03-17 15:39:13.211954 W | embed: expected IP in URL for binding (https://node02.qa.example.com:2380) 2017-03-17 15:39:13.212044 W | embed: expected IP in URL for binding (https://node02.qa.example.com:2379) 2017-03-17 15:39:13.212061 I | etcdmain: etcd Version: 3.1.3 2017-03-17 15:39:13.212070 I | etcdmain: Git SHA: 21fdcc6 2017-03-17 15:39:13.212083 I | etcdmain: Go Version: go1.7.5 2017-03-17 15:39:13.212095 I | etcdmain: Go OS/Arch: linux/amd64 2017-03-17 15:39:13.212102 I | etcdmain: setting maximum number of CPUs to 2, total number of available CPUs is 2 2017-03-17 15:39:13.212146 N | etcdmain: the server is already initialized as member before, starting as etcd member... 2017-03-17 15:39:13.212161 W | embed: expected IP in URL for binding (https://node02.qa.example.com:2380) 2017-03-17 15:39:13.212170 W | embed: expected IP in URL for binding (https://node02.qa.example.com:2379) 2017-03-17 15:39:13.212195 I | embed: peerTLS: cert = /var/local/certs/etcd.crt, key = /var/local/certs/etcd.key, ca = , trusted-ca = /var/local/certs/etcd.trustedca.crt, client-cert-auth = false 2017-03-17 15:39:13.215342 I | embed: listening for peers on https://node02.qa.example.com:2380 2017-03-17 15:39:13.216990 I | embed: listening for client requests on node02.qa.example.com:2379 2017-03-17 15:39:13.220013 I | warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated 2017-03-17 15:39:13.241849 I | etcdserver: recovered store from snapshot at index 96989702 2017-03-17 15:39:13.241875 I | etcdserver: name = node02 2017-03-17 15:39:13.241883 I | etcdserver: data dir = /var/lib/etcd/cluster/datadir 2017-03-17 15:39:13.241892 I | etcdserver: member dir = /var/lib/etcd/cluster/datadir/member 2017-03-17 15:39:13.241899 I | etcdserver: heartbeat = 200ms 2017-03-17 15:39:13.241905 I | etcdserver: election = 2000ms 2017-03-17 15:39:13.241912 I | etcdserver: snapshot count = 10000 2017-03-17 15:39:13.241930 I | etcdserver: advertise client URLs = https://node02.qa.example.com:2379 2017-03-17 15:39:13.893573 I | etcdserver: restarting member de98d7a88277647a in cluster b45f8bf7a306ca3a at commit index 96990605 2017-03-17 15:39:13.893690 I | raft: de98d7a88277647a became follower at term 3298 2017-03-17 15:39:13.893749 I | raft: newRaft de98d7a88277647a [peers: [22f7f7acca1bf43a,318e4ede0816b8ed,91eb8d5ebe44b736,de98d7a88277647a,eaf02d9d1857d9d6], term: 3298, commit: 96990605, applied: 96989702, lastindex: 96990605, lastrm: 3298] 2017-03-17 15:39:13.894021 I | etcdserver/api: enabled capabilities for version 3.0 2017-03-17 15:39:13.894051 I | etcdserver/membership: added member 22f7f7acca1bf43a [https://node03.qa.example.com:2380] to cluster b45f8bf7a306ca3a from store 2017-03-17 15:39:13.894065 I | etcdserver/membership: added member 318e4ede0816b8ed [https://node01.qa.example.com:2380] to cluster b45f8bf7a306ca3a from store 2017-03-17 15:39:13.894076 I | etcdserver/membership: added member 91eb8d5ebe44b736 [https://node05.qa.example.com:2380] to cluster b45f8bf7a306ca3a from store 2017-03-17 15:39:13.894088 I | etcdserver/membership: added member de98d7a88277647a [https://node02.qa.example.com:2380] to cluster b45f8bf7a306ca3a from store 2017-03-17 15:39:13.894109 I | etcdserver/membership: added member eaf02d9d1857d9d6 [https://node04.qa.example.com:2380] to cluster b45f8bf7a306ca3a from store 2017-03-17 15:39:13.894130 I | etcdserver/membership: set the cluster version to 3.0 from store 2017-03-17 15:39:13.968584 I | warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated 2017-03-17 15:39:13.969488 I | warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated 2017-03-17 15:39:13.969539 I | rafthttp: starting peer 22f7f7acca1bf43a... 2017-03-17 15:39:13.969586 I | rafthttp: started HTTP pipelining with peer 22f7f7acca1bf43a 2017-03-17 15:39:13.970242 I | rafthttp: started streaming with peer 22f7f7acca1bf43a (writer) 2017-03-17 15:39:13.970957 I | rafthttp: started streaming with peer 22f7f7acca1bf43a (writer) 2017-03-17 15:39:13.974540 I | rafthttp: started peer 22f7f7acca1bf43a 2017-03-17 15:39:13.974581 I | rafthttp: added peer 22f7f7acca1bf43a 2017-03-17 15:39:13.974606 I | rafthttp: starting peer 318e4ede0816b8ed... 2017-03-17 15:39:13.974666 I | rafthttp: started HTTP pipelining with peer 318e4ede0816b8ed 2017-03-17 15:39:13.975430 I | rafthttp: started streaming with peer 22f7f7acca1bf43a (stream MsgApp v2 reader) 2017-03-17 15:39:13.975837 I | rafthttp: started streaming with peer 22f7f7acca1bf43a (stream Message reader) 2017-03-17 15:39:13.976075 I | rafthttp: started streaming with peer 318e4ede0816b8ed (writer) 2017-03-17 15:39:13.976332 I | rafthttp: started streaming with peer 318e4ede0816b8ed (writer) 2017-03-17 15:39:13.978704 I | rafthttp: started peer 318e4ede0816b8ed 2017-03-17 15:39:13.978738 I | rafthttp: added peer 318e4ede0816b8ed 2017-03-17 15:39:13.978765 I | rafthttp: starting peer 91eb8d5ebe44b736... 2017-03-17 15:39:13.978806 I | rafthttp: started HTTP pipelining with peer 91eb8d5ebe44b736 2017-03-17 15:39:13.983566 I | rafthttp: started streaming with peer 91eb8d5ebe44b736 (writer) 2017-03-17 15:39:13.983670 I | rafthttp: started streaming with peer 318e4ede0816b8ed (stream MsgApp v2 reader) 2017-03-17 15:39:13.984701 I | rafthttp: started peer 91eb8d5ebe44b736 2017-03-17 15:39:13.984726 I | rafthttp: added peer 91eb8d5ebe44b736 2017-03-17 15:39:13.984744 I | rafthttp: starting peer eaf02d9d1857d9d6... 2017-03-17 15:39:13.984771 I | rafthttp: started streaming with peer 318e4ede0816b8ed (stream Message reader) 2017-03-17 15:39:13.984799 I | rafthttp: started streaming with peer 91eb8d5ebe44b736 (stream MsgApp v2 reader) 2017-03-17 15:39:13.984936 I | rafthttp: started streaming with peer 91eb8d5ebe44b736 (writer) 2017-03-17 15:39:13.984958 I | rafthttp: started HTTP pipelining with peer eaf02d9d1857d9d6 2017-03-17 15:39:13.985135 I | rafthttp: started streaming with peer 91eb8d5ebe44b736 (stream Message reader) 2017-03-17 15:39:13.985311 I | rafthttp: started streaming with peer eaf02d9d1857d9d6 (writer) 2017-03-17 15:39:13.985750 I | rafthttp: started streaming with peer eaf02d9d1857d9d6 (writer) 2017-03-17 15:39:13.988982 I | rafthttp: started peer eaf02d9d1857d9d6 2017-03-17 15:39:13.989015 I | rafthttp: added peer eaf02d9d1857d9d6 2017-03-17 15:39:13.989056 I | etcdserver: starting server... [version: 3.1.3, cluster version: 3.0] 2017-03-17 15:39:13.989086 I | embed: ClientTLS: cert = /var/local/certs/etcd.crt, key = /var/local/certs/etcd.key, ca = , trusted-ca = /var/local/certs/etcd.trustedca.crt, client-cert-auth = false 2017-03-17 15:39:13.991284 I | rafthttp: started streaming with peer eaf02d9d1857d9d6 (stream MsgApp v2 reader) 2017-03-17 15:39:13.991527 I | rafthttp: started streaming with peer eaf02d9d1857d9d6 (stream Message reader) 2017-03-17 15:39:14.057550 I | rafthttp: peer 91eb8d5ebe44b736 became active 2017-03-17 15:39:14.059401 I | rafthttp: established a TCP streaming connection with peer 91eb8d5ebe44b736 (stream MsgApp v2 writer) 2017-03-17 15:39:14.060217 I | raft: raft.node: de98d7a88277647a elected leader 22f7f7acca1bf43a at term 3298 2017-03-17 15:39:14.060336 I | rafthttp: peer 22f7f7acca1bf43a became active 2017-03-17 15:39:14.060353 I | rafthttp: established a TCP streaming connection with peer 22f7f7acca1bf43a (stream Message reader) 2017-03-17 15:39:14.060448 I | rafthttp: established a TCP streaming connection with peer 22f7f7acca1bf43a (stream MsgApp v2 reader) 2017-03-17 15:39:14.062079 I | rafthttp: peer eaf02d9d1857d9d6 became active 2017-03-17 15:39:14.062097 I | rafthttp: established a TCP streaming connection with peer eaf02d9d1857d9d6 (stream Message writer) 2017-03-17 15:39:14.062267 I | rafthttp: established a TCP streaming connection with peer 91eb8d5ebe44b736 (stream Message writer) 2017-03-17 15:39:14.062424 I | rafthttp: established a TCP streaming connection with peer eaf02d9d1857d9d6 (stream MsgApp v2 writer) 2017-03-17 15:39:14.096925 I | rafthttp: peer 318e4ede0816b8ed became active 2017-03-17 15:39:14.101323 I | rafthttp: established a TCP streaming connection with peer 318e4ede0816b8ed (stream Message reader) 2017-03-17 15:39:14.101370 I | rafthttp: established a TCP streaming connection with peer 91eb8d5ebe44b736 (stream MsgApp v2 reader) 2017-03-17 15:39:14.101419 I | rafthttp: established a TCP streaming connection with peer 318e4ede0816b8ed (stream MsgApp v2 reader) 2017-03-17 15:39:14.101494 I | rafthttp: established a TCP streaming connection with peer eaf02d9d1857d9d6 (stream Message reader) 2017-03-17 15:39:14.101544 I | rafthttp: established a TCP streaming connection with peer 318e4ede0816b8ed (stream Message writer) 2017-03-17 15:39:14.101566 I | rafthttp: established a TCP streaming connection with peer 318e4ede0816b8ed (stream MsgApp v2 writer) 2017-03-17 15:39:14.101580 I | rafthttp: established a TCP streaming connection with peer eaf02d9d1857d9d6 (stream MsgApp v2 reader) 2017-03-17 15:39:14.101699 I | rafthttp: established a TCP streaming connection with peer 22f7f7acca1bf43a (stream MsgApp v2 writer) 2017-03-17 15:39:14.101734 I | rafthttp: established a TCP streaming connection with peer 91eb8d5ebe44b736 (stream Message reader) 2017-03-17 15:39:14.101763 I | rafthttp: established a TCP streaming connection with peer 22f7f7acca1bf43a (stream Message writer) 2017-03-17 15:39:14.122991 I | etcdserver: published {Name:node02 ClientURLs:[https://node02.qa.example.com:2379]} to cluster b45f8bf7a306ca3a 2017-03-17 15:39:14.123057 I | embed: ready to serve client requests 2017-03-17 15:39:14.123451 I | embed: serving client requests on 10.112.140.34:2379

heyitsanthony commented 7 years ago

@raoofm See #6336 for some context about the security concern for using DNS to select a bind address.

Binding through DNS is slated to be rejected in 3.2. The expectation is the listen binds to a network interface, not a domain name. Is the config tool using DNS to select an interface from a set of several on the same machine?

raoofm commented 7 years ago

config tool has this template and similar for all nodes. etcd_config.txt

If during disaster we want to start a new cluster, we just change the A records and start the cluster which will do the discovery on the basis of same srv records and proxies/gateways would still work on the same domain name based on srv.

Also I'm wondering if cloud providers can guarantee the same ip.

heyitsanthony commented 7 years ago

# _MY_IPADDR=$(/sbin/ifconfig | grep 'inet addr:'| grep -v '127.0.0.1' | cut -d: -f2 | sort | awk '{print $1}') Why was this commented out? This seems closer to what it should do for the listen addresses.

The advertise URLs are fine to use a domain name.

Also I'm wondering if cloud providers can guarantee the same ip.

Why does this matter? The interface could be NATed and etcd would bind based on the interface's NATed IP, not the external IP.

raoofm commented 7 years ago

agree, actually second thought, what you guys did makes perfect sense.

so only --listen-peer-urls, --listen-client-urls should change to ip addresses

Should SRV records be changed from domain names to ip addresses?

raoofm commented 7 years ago

SRV are used by peers, proxy, gateway

heyitsanthony commented 7 years ago

so only --listen-peer-urls, --listen-client-urls should change to ip addresses

Yes.

Should SRV records be changed from domain names to ip addresses?

SRV records must point to A records, so no. The SRV records should be fine as-is; the change only affects bind addresses.

raoofm commented 7 years ago

@heyitsanthony thanks.