Abandoned - go-rancher-gen

rancher-gen is a file generator that renders templates using Rancher Metadata.

Core features:

Powerful template syntax that embraces Rancher services, containers and hosts as first-class objects
Ability to run arbitrary commands when a file has been updated (e.g. to reload an application's configuration)
Ability to run check commands on staged files before updating the destination files
Ability to specify multiple template sets using a TOML config file

Usage

Command Line

rancher-gen [options] source [dest]

options

Flag	Description
`config`	Path to an optional config file. Options specified on the CLI always take precedence.
`metadata-version`	Metadata version string used when querying the Rancher Metadata API. Default: `latest`.
`include-inactive`	Not yet implemented
`interval`	Interval (in seconds) for polling the Metadata API for changes. Default: `5`.
`onetime`	Process all templates once and exit. Default: `false`.
`log-level`	Verbosity of log output. Default: `info`.
`check-cmd`	Command to check the content before updating the destination. Use the `{{staging}}` placeholder to reference the staging file.
`notify-cmd`	Command to run after the destination file has been updated.
`notify-lbl`	Run command once for each container with this label.
`notify-output`	Print the result of the notify command to STDOUT.
`version`	Show application version and exit.

`source`

Path to the template.

`dest`

Path to the destination file. If omitted, then the generated content is printed to STDOUT.

Examples

rancher-gen --onetime --notify-cmd="/usr/sbin/service nginx reload" \
/etc/rancher-gen/nginx.tmpl /etc/nginx/nginx.conf

rancher-gen --interval 2 --check-cmd="/usr/sbin/nginx -t -c {{staging}}" \
--notify-cmd="/usr/sbin/service nginx reload" /etc/rancher-gen/nginx.tmpl /etc/nginx/nginx.conf

Configuration file

You can optionally pass a configuration file to rancher-gen. The configuration file is a TOML file. It allows you to specify multiple template sets grouped by template sections. You can specify the same options as on the command line. Options specified on the command line or via environment variables take precedence over the corresponding values in the configuration file. An example file is available here.

How to dynamically configure your applications with Rancher Metadata

You can bundle rancher-gen with the application image or run it as a service sidekick, that exposes the generated configuration file in a shared volume.

Bundled with application image

Download the binary from the release page. Add the binary to your Docker image and provide a mechanism that runs rancher-gen on container start and then executes the main application. This functionality could be provided by a Bash script executed as image ENTRYPOINT. If you want to reload the application whenever the Metadata referenced in the template changes, you can use a container process supervisor (e.g. S6-overlay) to keep rancher-gen running in the background and notify the application when it needs to reload the configuration (by sending it a SIGHUP for example).

Sidekick Container

Create a new Docker image using janeczku/rancher-gen:latest as base. Add the template(s) and configuration file(s) to the image. Expose the configuration folder as VOLUME. Run rancher-gen on container start, specifying relevant options as command line parameters.

Example acme/nginx-config sidekick image

FROM janeczku/rancher-gen:latest
COPY config.toml /etc/rancher-gen/
COPY nginx.tmpl /etc/rancher-gen/
VOLUME /etc/nginx
CMD ["--config", "/etc/rancher-gen/config.toml"]

Example Rancher Compose file

nginx:
  image: nginx:latest
  volumes_from:
  - config-sidekick
  labels:
    io.rancher.sidekicks: template-sidekick
config-sidekick:
  image: acme/nginx-config

Notify Labels

Specifying a notify-lbl config parameter will run the check-cmd and notify-cmd hook once for each container matching the notify-lbl value. This parameter can be a single label name (ie. notify-lbl = "mylabel") or a specific label:value pair (ie. notify-lbl = "mylabel:myvalue"). Label names and values are colon-separated.

Container fields like {{.Name}} and {{.Address}} (see the Container model for reference) values can be used as template variables in the notify-cmd string. This allows the notify-cmd to be tailored to specific containers. Labels can be accessed by period separation (ie. {{.Labels.my.label}}. See the following example which appends text to a file:

Example config.toml file

[[template]]
source = "/etc/rancher-gen/nginx.tmpl"
dest = "/etc/rancher-gen/nginx.conf"
check-cmd = "echo $(date)\t check >> /etc/rancher-gen/check.log"
notify-cmd = "echo $(date)\t notify N:{{Name}} A:{{Address}} >> /etc/rancher-gen/notify.log"
notify-lbl = "testlabel:1"
notify-output = true

This example will run notify-cmd once for each unique container with a label name of testlabel and a label value of 1, and the container Name and Address will substitue for values during runtime.

Example Output

Mon Feb 13 22:58:39 UTC 2017 notify N:whoami-whoami-1 A:10.42.85.195
Mon Feb 13 22:58:39 UTC 2017 notify N:whoami2-whoami-1 A:10.42.130.246
Mon Feb 13 22:58:39 UTC 2017 notify N:whoami3-whoami-1 A:10.42.234.149

Template Language

Templates are Go text templates. In addition to the built-in functions, rancher-gen exposes functions and methods to easily discover Rancher services, containers and hosts.

Service Discovery Objects

type Service struct {
    Name        string
    Stack       string
    Kind        string
    Vip         string
    Fqdn        string
    Ports       []Port
    Labels      LabelMap
    Metadata    MetadataMap
    Containers  []Container
}

type Port struct {
    PublicPort   string
    InternalPort string
    Protocol     string
}

type Container struct {
    Name        string
    Address     string
    Stack       string
    Service     string
    Health      string
    State       string
    Labels      LabelMap
    Host        Host
}

type Host struct {
    UUID        string
    Name        string
    Address     string
    Hostname    string
    Labels      LabelMap
}

The LabelMap and MetadataMap types implement methods for easily checking the existence of specific keys and accessing their values:

Labels.Exists(key string) bool
Returns true if the given label key exists in the map.

Labels.GetValue(key, default string) string
Returns the value of the given label key. The function accepts an optional default value that is returned when the key doesn't exist or is set to an empty string.

Metadata.Exists(key string) bool
Returns true if the given metadata key exists in the map.

Metadata.GetValue(key, default interface{}) interface{}
Returns the value of the given label key. The function accepts an optional default value that is returned when the key doesn't exist.

Examples:

Check if the label exists:

{{range services}}
{{if .Labels.Exists "foo"}}
{{do something}}
{{end}}
{{end}}

Get the value of a Metadata key:

{{range services}}
Metadata foo: {{.Metadata.GetValue "foo"}}
{{end}}

Using a default value:

{{range services}}
Label foo: {{.Labels.GetValue "foo" "default value"}}
{{end}}

Service Discovery Functions

`host`

Lookup a specific host

Optional argument
UUID string
Return Type
Host

If the argument is omitted the local host is returned:

{{host}}

`hosts`

Lookup hosts

Optional parameters
labelSelector string
Returned Type
[]Host

The function returns a slice of Host which can be used for ranging in a template:

{{range hosts}}
host {{.Name}} {{.Address}}
{{end}}

which would produce something like:

host aws-sm-01 148.210.10.10
host aws-sm-02 148.210.10.11

One or multiple label selectors can be passed as arguments to limit the result to hosts with matching labels. The syntax of the label selector is @label-key=label-value.

The following function returns only hosts that have a label "foo" with the value "bar":

{{hosts "@foo=bar"}}

The label selector syntax supports a regex pattern on it's right side. E.g. to lookup hosts that have a specific label regardless of the value:

{{hosts "@foo=.*"}}

If the argument is omitted all hosts are returned:

{{hosts}}

`service`

Lookup a specific service

Optional parameter
serviceIdentifier string
Returned Type
Service

The function returns a Service struct. You can use the Containers field for ranging over all containers belonging to the service:

{{with service "web.production"}}
{{range .Containers}}
http://{{.Address}}:9090
{{end}}
{{end}}

which would produce something like:

http://10.12.20.111:9090
http://10.12.20.122:9090

The syntax of the serviceIdentifier parameter is service-name[.stack-name]:

{{service "web.production"}}

If the stack name is omitted the service is looked up in the local stack:

{{service "web"}}

If no argument is given the local service is returned:

{{service}}

`services`

Lookup services matching the given stack and label selectors

Optional parameters
stackSelector string
labelSelector string
Return Type
[]Service

Just like with the hosts function multiple label selectors can be passed to select services with matching labels:

{{services "@foo=bar"}}

The stack selector parameter uses the syntax .stack-name:

{{services ".production"}}

Stack and label selectors can be combined like this:

{{services ".production" "@foo=bar"}}

If arguments are omitted then all services are returned:

{{services}}

Helper Functions and Pipes

`whereLabelExists`

Filter a slice of hosts, services or containers returning the items that have the given label key.

Parameters
labelKey string
input []Host, []Service or []Container
Return Type
same as input

`whereLabelEquals`

Filter a slice of hosts, services or containers returning the items that have the given label key and value.

Arguments
labelKey string
labelValue string
input []Host, []Service or []Container
Return Type
same as input

{{$ervice := service "web.production"}}
{{range $container := whereLabelEquals "foo" "bar" $service.Containers}}
{{do something with $container}}
{{end}}

`whereLabelMatches`

Filter a slice of hosts, services or containers returning the items that have the given label and a value matching the regex pattern.

Arguments
labelKey string
regexPattern string
input []Host, []Service or []Container
Return Type
same as input

`groupByLabel`

This function takes a slice of hosts, services or containers and groups the items by their value of the given label. It returns a map with label values as key and a slice of corresponding elements items as value.

Arguments
label-key string
input []Host,[]Service,[]Container
Return Type
map[string][]Host/[]Service/[]Container

{{range $labelValue, $hosts := hosts | groupByLabel "foo"}}
{{$labelValue}}
{{range $hosts}}
IP: {{.Address}}
{{end}}
{{end}}

`base`

Alias for the path.Base function

filename: {{$service.Metadata.GetValue "targetPath" | base}}

See Go's path.Base() for more information.

`dir`

Alias for the path.Dir function

See Go's path.Dir() for more information.

`exists`

Test for existence of path or file

{{ exists (/path/to/file) }}

`env`

Returns the value of the given environment variable or an empty string if the variable isn't set

{{env "FOO_VAR"}}

`timestamp`

Alias for time.Now

# Generated by rancher-gen {{timestamp}}

The timestamp can be formatted as required by invoking the Format method:

# Generated by rancher-gen {{timestamp.Format "Jan 2, 2006 15:04"}}

See Go's time.Format() for more information about formatting the date according to the layout of the reference time.

`split`

Alias for strings.Split

{{$items := split $someString ":"}}

See Go's strings.Split() for more information.

`join`

Alias for strings.Join
Takes the given slice of strings as a pipe and joins them on the provided string:

{{$items | join ","}}

See Go's strings.Join() for more information.

`toLower`

Alias for strings.ToLower
Takes the argument as a string and converts it to lowercase.

{{$svc.Metadata.GetValue "foo" | toLower}}

See Go's strings.ToLower() for more information.

`toUpper`

Alias for strings.ToUpper
Takes the argument as a string and converts it to uppercase.

{{$svc.Metadata.GetValue "foo" | toUpper}}

See Go's strings.ToUpper() for more information.

`contains`

Alias for strings.Contains

See Go's strings.Contains() for more information.

`replace`

Alias for strings.Replace

{{$foo := $svc.Labels.GetValue "foo"}}
foo: {{replace $foo "-" "_" -1}}

See Go's strings.Replace() for more information.

Examples

TODO

Development

Prerequisites:

Go
Running Rancher instance
[rancher-compose(https://docs.rancher.com/rancher/v1.4/en/cattle/rancher-compose/) (and associated environment variables)

Setup:

Ensure Go is installed, cd into your go-work (or equivalent) directory within $GOPATH
git clone this repo, cd into repo
Ensure test directory exists: mkdir -p test

Development workflow can happen as follows:

Build project: make build
Create Docker image: make dev-image
Run in Rancher: rancher-compose up (add -d if needed)
Inspect, and rancher-compose rm when finished. Repeat.

See make for more commands.

CausticLab / go-rancher-gen

readme

Abandoned - go-rancher-gen

Usage

Command Line

options

source

dest

Examples

Configuration file

How to dynamically configure your applications with Rancher Metadata

Bundled with application image

Sidekick Container

Example acme/nginx-config sidekick image

Example Rancher Compose file

Notify Labels

Example config.toml file

Example Output

Template Language

Service Discovery Objects

Service Discovery Functions

host

hosts

service

services

Helper Functions and Pipes

whereLabelExists

whereLabelEquals

whereLabelMatches

groupByLabel

base

dir

exists

env

timestamp

split

join

toLower

toUpper

contains

replace

Examples

Development

`source`

`dest`

`host`

`hosts`

`service`

`services`

`whereLabelExists`

`whereLabelEquals`

`whereLabelMatches`

`groupByLabel`

`base`

`dir`

`exists`

`env`

`timestamp`

`split`

`join`

`toLower`

`toUpper`

`contains`

`replace`