hashicorp / consul

Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.
https://www.consul.io
Other
28.45k stars 4.43k forks source link

Consul LAN/WAN Gossip #2292

Closed Kenovo closed 8 years ago

Kenovo commented 8 years ago

Hello, could you please explain to me the difference between Consul LAN goosip protocol and the Consul WAN gossip ? could i connect two datacenters with LAN Gossip Protocol to have a kind of multi-datacenter DNS Loadbalancing provided by Consul DNS Service ? What kind of problemes could a such configuration create ?

Thank you,

sitano commented 8 years ago

LAN and WAN gossip use the same protocol SWIM (https://www.cs.cornell.edu/~asdas/research/dsn02-swim.pdf), implemented in memberlist package with the Serf (with vector clocks blah-blah) on top of it.

Read docs.

https://www.consul.io/docs/internals/gossip.html http://sitano.github.io/2015/10/06/abt-consul-outage/

About connecting cross-dc with lan level serf - you can, but there are certain constraints in the protocol in the deterministic failure detector (timeout based thing) and that may result on the state convergence.

@armon wrote somewhere about thats not recommended and everything. Search if you want. Or just do some tests.

p.s. Actually we do that in one of our clusters cross 3 dcs.

slackpad commented 8 years ago

Here's the thread I think @sitano was referencing:

https://groups.google.com/forum/#!searchin/consul-tool/armon$20latency%7Csort:relevance/consul-tool/y1bFnMntF7M/cpM6Pw9BEAAJ

That has the upper limit of what's considered a LAN. We don't have native multi-DC load balancing, though you can get multi-DC failover via prepared queries - https://www.consul.io/docs/agent/http/query.html. Some folks use something like consul-template to configure a load balancer that feeds traffic across multiple DCs.

This has the best concise summary of LAN vs. WAN gossip - https://www.consul.io/docs/internals/architecture.html.