search

nigoroll / libvmod-dynamic

The Varnish dns/named director continued
BSD 2-Clause "Simplified" License
95 stars 34 forks source link

Feature request: SRV support for port finding #42

Closed danielmotaleite closed 5 years ago

danielmotaleite commented 5 years ago

Right now we can find the backend via DNS, but the port is required to be the same for all the backends.

With docker, we start to get more and more backends with dynamic ports and to keep using static ports we need some "load-balancers". It would be great if we could remove those and allow varnish to find the backends AND ports using DNS SRV entries. SRV DNS entries give us not only the host/IP for the service, but also the correct port for it. Consul service discovery do this automatically and gives great flexibility to deploy and manage services

Example: $ host -t SRV consul.service.consul.internal. consul.service.consul.internal has SRV record 1 2 8300 consul-a01.node.interxion-fra6.consul.internal. consul.service.consul.internal has SRV record 1 3 8301 consul-b01.node.interxion-fra6.consul.internal. consul.service.consul.internal has SRV record 2 1 8302 consul-a02.node.interxion-fra6.consul.internal.

A varnish backend "consul" would setup 3 backends for each DNS:PORT

Bonus: SRV could also later use the priority (1, 1, 2 above) and then weight value (2,3 for priority 1 group, 1 for priority 2 group) to balance the load between nodes

yes, i know that SRV is not as simple as simple A/CNAME DNS, but they are way more flexible and in a long run they can help a lot

nigoroll commented 5 years ago

That's a sensible suggestion, but the main issue here is that we need an own resolver implementation to get our hand on SRV. Last time I looked (because of the equally sensible request to use the DNS TTL as the libvmod-dynamic TTL), I found no resolver implementation which appeared to me to be of sufficient quality.

FWIW, this project may be related to your request:

dridi commented 5 years ago

*cough* getdns *cough*

nigoroll commented 5 years ago

@Dridi that must be a bad cold, get well soon ;)

cough thanks cough I'll check back cough in 2019

danielmotaleite commented 5 years ago

Thanks for the reply! I suspected that the missing dns code would be the major problem! :)

I'm not a programmer, but have you checked unbound? or gdnsd? both should have a "modern" design (again, not a programmer, so it is easy to believe all claims!)

the k8s-ingress is very interesting, we are not using right now Kubernetes, we are actually testing it vs hashicorp nomad (both good, with very different ways of solving mostly the same problems).

It was during this tests i notice that the LB (or ingress in k8s) is not really needed if we had the SRV support, allowing direct access to the backends, allowing better traffic scale up

nigoroll commented 5 years ago

Implemented as of 256b1f01677793f4363318830d7e61ff4de2c0fc