Load extra options from a file

[markvr]

I'm using the EXTRA_DEFAULT_SETTINGS environment variable to capture headers and configure the log format, and it's all starting to get a bit hairy on one very long line, with various levels of escaping for quotes and commas needed.

How about having a mechanism where these can be specified in a file that is added to the image, and merged with the config? e.g. EXTRA_DEFAULT_SETTINGS_FILE="/extra-defaults.cfg", similar to how I think CA_CERT_FILE works?

It's then down to the user how they get that file into the container, whether using a bind mount, or better, by building a custom image FROM this one and copying them in.

This would apply to the following settings: EXTRA_BIND_SETTINGS EXTRA_DEFAULT_SETTINGS EXTRA_FRONTENDSETTINGS EXTRA_GLOBAL_SETTINGS

Thoughts?!

[markvr]

I've thought about this a bit more. Rather than have an increasingly complex set of things that can be configured through environment variables, and need to document exactly how these are all composed together into the final config - instead for "advanced" users, enable them to supply their own custom haproxy.cfg template. This would then be rendered using the discovered backend/front ends inserted where the template placeholders are.

This would also make it easier to ultimately understand how the config will end up looking, and separate the config layout from the logic, similar to how is commonly done with generating HTML.

I might have a go at doing this. The way I imagine this working is for the default to remain as currently, but there is an extra "configTemplateFile" env var. If supplied, a new method "templateUpdate" in haproxycfg.py is called instead of "update". This uses a templating framework (jinja seems to be a popular python one?) with a user supplied template file to generate the output. There would also be a default config file for reference that returned an output identical to the normal output.

This gives the user a lot more flexibility in how haproxy is configured, and they can also remove default values that are unsuitable.

Thoughts again?!

[tifayuki]

@markvr

We had a plan that let user provide a template and the script is mainly responsible for manipulating the dynamic stuff like frontend, backend routes, etc.

There is a proposal about configs (https://github.com/docker/swarmkit/pull/2036) in swarm that can help inject the template to the service. I would like wait until it is merged and published and then do something on this repo to support templates.

Thank you

[markvr]

Sure, I started playing around with this idea using Jinja2 and then realised quite how intricate the templating would need to be to support all the edge cases. Instead I implemented my original idea of simply having EXTRA_DEFAULT_SETTINGS_FILE etc which gets added in along with the EXTRA_DEFAULT_SETTINGS from the env variable and was pretty trivial.

My reason for this is because I'm shipping the haproxy logs to Logstash as JSON, which is quite a complex and long string to put in as an env variable, especially as all the quotes and commas need escaping. Along with various "header capture" settings, being able to specify all this with no escaping in a file has made things simpler.

I'll tidy it up and send a PR if you think it would be useful for others?

[tifayuki]

@markvr

Yes, sure. It will be very appreciated :)

[markvr]

https://github.com/docker/dockercloud-haproxy/pull/184