Adding multiple TCP listeners from Consul

[murphymj25]

We have been working on providing a solution for TCP load balancing for a potential large number of applications. Our requirements are that the service can be defined in consul, and the load balancer will automatically start the TCP listener. Also, because we could have different applications requiring the same TCP port, we need to define the route by IP address.

With the current TCP implementation in Fabio, I am only able to start a listener in the properties file, and define a backend route using :port which doesn't match our current use case.

We have built a new proxy option that we are currently calling "tcp-dynamic" that uses the consul urlprefix tagging to start the TCP listeners and defines the backend route using ip:port. Our initial testing has been promising, we were able to define 5000 different TCP ports in consul and start the listeners with pretty minimal CPU increases. We need to do a little more cleanup, but should have something ready for feedback early next week.

[murphymj25]

@craighazen and I have created pull request #626 for this issue. Take a look and let us know if there is any feedback.

[scalp42]

I'm currently trying to do the same thing using consul-template but I'm not getting very far.

Pseudo code would be:

• iterate over every service passing
• check the tags include "proto=tcp"
• update the proxy.addr in fabio.properties to include the urlprefix- tag and do some cleanup

Something like this (pseudo template):

{{ range services }}
{{ if .Tags | join "," | regexMatch "proto=tcp" }}
<< something here to extract `fabio-:10000 proto=tcp` >> 
proxy.addr = :9999,<<:10000;proto=tcp>>
{{ end }}
{{ end }}

It'd be much better to just have this dynamic for sure.

UPDATE: because my Go fu is limited I went another way with Ruby directly

fabio_addr.rb:

#! /usr/bin/env ruby

if ARGV.empty?
  Kernel.exit(0)
end

if ARGV.first.include?('proto=tcp') && ARGV.first.start_with?('urlprefix-:')
  puts ARGV.first.split('tmpl_').last.split(' ')[0..1].join(';')
  exit 0
end

fabio.ctmpl (easy to read):

{{ range services }}
{{ range .Tags }}
{{ if . | contains "proto=tcp" }}
{{ . | plugin "/root/fabio_addr.rb" }}
{{ end }}
{{ end }}
{{ end }}

fabio.ctmpl (real template):

proxy.addr = :9999,{{ range services }}{{ range .Tags }}{{if . | contains "proto=tcp" }}{{ . | plugin "/root/fabio_addr.rb" }}{{ end }}{{ end }}{{ end }}

Exec:

root@consul-i-0d383c794127f6ede [dev-usw2-core1] ~ # consul-template -once -dry -template "fabio.ctmpl:fabio.properties"

proxy.addr = :9999,:1000;proto=tcp

One issue is that you can't specify Fabio metrics.names & metrics.prefix as it contains Go language, so you'll need to escape it:

metrics.names = {{clean .Service}}.{{clean .Host}}.{{clean .Path}}.{{clean .TargetURL.Host}}
metrics.prefix = {{clean .Hostname}}.{{clean .Exec}}