FR: Render orchestrator specific templating.

[ChristianKniep]

Description In order to resume operations for a given tasks in case the tasks is rescheduled, I like to use {{.Tasks.Slot}}. Here KAFKA_BROKER_ID={{.Task.Slot}} and hostname: "{{.Service.Name}}.{{.Task.Slot}}.{{.Task.ID}}".

Otherwise each kafka-broker would pick up a new broker-ID, which does not make sense. The hostname makes it easy to spot which broker-id is used by which tasks.

version: '3'
services:
  zookeeper:
    image: qnib/${zk_img_name}:${zk_img_tag}
  zkui:
    image: qnib/plain-zkui:${zkui_img_tag}
    ports:
      - "${zkui_port}:9090"
  broker:
    image: qnib/${kafka_img_name}:${kafka_img_tag}
    hostname: "{{.Service.Name}}.{{.Task.Slot}}.{{.Task.ID}}"
    deploy:
      replicas: 3
    environment:
      - KAFKA_BROKER_ID={{.Task.Slot}}
      - LOG_MESSAGE_FORMAT_VERSION=${kafka_log_msg_format}
  manager:
    image: qnib/plain-kafka-manager:${kmanager_img_tag}
    ports:
      - "${kmanager_port}:9000"
    environment:
      ZOOKEEPER_HOSTS: "tasks.zookeeper:2181"

Using the experimental templating flag, {{.Task.Slot}} is interpreted as configuration.

Describe the results you received: When rendering, I get:

$ docker-app render
Error: failed to execute go template: template: compose:12:25: executing "compose" at <.Service.Name>: map has no entry for key "Service"

As the renderer fails when interpreting the hostname.

Describe the results you expected:

I would like to use those templates - maybe even agnostic to the orchestrator, so that I use a generic template that will be filled out depending on the orchestrator. Alternatively a condition can be made, which let's me drop it in case I do not use SWARM.

Output of docker version:

Client:
 Version:      18.06.0-ce-rc3
 API version:  1.38
 Go version:   go1.10.3
 Git commit:   cbfa3d9
 Built:        Thu Jul 12 05:15:46 2018
 OS/Arch:      darwin/amd64
 Experimental: true

Server:
 Engine:
  Version:      18.06.0-ce-rc3
  API version:  1.38 (minimum version 1.12)
  Go version:   go1.10.3
  Git commit:   cbfa3d9
  Built:        Thu Jul 12 05:26:13 2018
  OS/Arch:      linux/amd64
  Experimental: true

Output of docker-app version:

docker-app version
Version:      v0.3.0-83-gc05143e
Git commit:   c05143e
Built:        Thu Jul 26 05:06:45 2018
OS/Arch:      darwin/amd64
Experimental: on
Renderers:    gotemplate, mustache, none, yatee

Output of docker info:

Containers: 45
 Running: 19
 Paused: 0
 Stopped: 26
Images: 142
Server Version: 18.06.0-ce-rc3
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: active
 NodeID: ntmgx18xgesozgsy2dsgzb4b9
 Is Manager: true
 ClusterID: yolyvxvy2bjceh662p8bt17av
 Managers: 1
 Nodes: 1
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 10
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
  Force Rotate: 0
 Autolock Managers: false
 Root Rotation In Progress: false
 Node Address: 192.168.65.3
 Manager Addresses:
  192.168.65.3:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: d64c661f1d51c48782c9cec8fda7604785f93587
runc version: 69663f0bd4b60df09991c08812a60108003fa340
init version: fec3683
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.93-linuxkit-aufs
Operating System: Docker for Mac
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 9.744GiB
Name: linuxkit-025000000001
ID: BXDU:L4YL:7GPX:DOI3:JWXD:DSUG:SXVD:DKVL:EAO7:VKNO:HNO6:DQWL
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 150
 Goroutines: 269
 System Time: 2018-07-26T07:55:13.8281597Z
 EventsListeners: 3
HTTP Proxy: gateway.docker.internal:3128
HTTPS Proxy: gateway.docker.internal:3129
Registry: https://index.docker.io/v1/
Labels:
Experimental: true
Insecure Registries:
 ec2-34-242-200-232.eu-west-1.compute.amazonaws.com
 127.0.0.0/8
Live Restore Enabled: false

[ChristianKniep]

The workaround is to deactivate the render altogether:

export DOCKERAPP_RENDERERS="none"

Which leads to not being able to use a render...

[ChristianKniep]

An example using both: https://github.com/qnib/service-orchestration/blob/master/Analytics/data-pipeline/kafka/kafka.dockerapp

[ijc]

Behaviour today is a little different from reported, but still not working:

$ curl -LOJ "https://github.com/qnib/service-orchestration/raw/master/Analytics/data-pipeline/kafka/kafka.dockerapp"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   179  100   179    0     0    648      0 --:--:-- --:--:-- --:--:--   648
100  1186  100  1186    0     0   3478      0 --:--:-- --:--:-- --:--:--  3478
$ cat kafka.dockerapp
---
version: 0.1.1
name: kafka
description: "Kafka stack for development purposes (downscaling impossible)"
maintainers:
- name: Christian Kniep
  email: christian@qnib.org
targets:
  swarm: true
  kubernetes: false

---

version: '3.6'
services:
  zookeeper:
    image: qnib/${zookeeper.name}:${zookeeper.tag}
  zkui:
    image: qnib/plain-zkui:${zkui.tag}
    ports:
      - "${zkui.port}:9090"
  broker:
    image: qnib/${kafka.name}:${kafka.tag}
    hostname: "{{.Service.Name}}.{{.Task.Slot}}.{{.Task.ID}}"
    deploy:
      mode: ${kafka.deploy.mode}
      {{if eq .kafka.deploy.mode "replicated"}}
      replicas: ${kafka.deploy.replicas}
      {{end}}
    environment:
      KAFKA_BROKER_ID: {{.Task.Slot}}
      LOG_MESSAGE_FORMAT_VERSION: ${kafka.log_msg_format}
  manager:
    image: qnib/plain-kafka-manager:${kmanager.tag}
    ports:
      - "${kmanager.port}:9000"
    environment:
      ZOOKEEPER_HOSTS: "tasks.zookeeper:2181"

---
zookeeper:
  name: plain-zookeeper
  tag: 2018-04-25
kafka:
  deploy:
    mode: replicated
    replicas: 3
  name: plain-kafka
  tag: 1.1.1
  log_msg_format: 1.0-IV0
zkui:
  tag: 8d3441d
  port: 9090
kmanager:
  tag: 1.3.3.18
  port: 9000
$ ./bin/docker-app render kafka.dockerapp 
Error: failed to load composefiles: failed to parse Compose file 
version: '3.6'
services:
  zookeeper:
    image: qnib/${zookeeper.name}:${zookeeper.tag}
  zkui:
    image: qnib/plain-zkui:${zkui.tag}
    ports:
      - "${zkui.port}:9090"
  broker:
    image: qnib/${kafka.name}:${kafka.tag}
    hostname: "{{.Service.Name}}.{{.Task.Slot}}.{{.Task.ID}}"
    deploy:
      mode: ${kafka.deploy.mode}
      {{if eq .kafka.deploy.mode "replicated"}}
      replicas: ${kafka.deploy.replicas}
      {{end}}
    environment:
      KAFKA_BROKER_ID: {{.Task.Slot}}
      LOG_MESSAGE_FORMAT_VERSION: ${kafka.log_msg_format}
  manager:
    image: qnib/plain-kafka-manager:${kmanager.tag}
    ports:
      - "${kmanager.port}:9000"
    environment:
      ZOOKEEPER_HOSTS: "tasks.zookeeper:2181"
: yaml: line 16: could not find expected ':'

Do I understand correctly that the ask here is to escape (and not barf on) the {{...}} stuff such that it comes out of the docker-app render unscathed (to be processed by some tool further down the line)?

[ChristianKniep]

Yes, indeed. Services like Kafka or other clustered DB fancy to keep the same hostname so that they can identify which partition of data belongs to them and is already local to the hosts storage. Thus, the hostname should be evaluated at schedule time and not while rendering the dockerapp "{{.Service.Name}}.{{.Task.Slot}}.{{.Task.ID}}".

[ijc]

Thanks.

Your kafka.dockerapp seems to go one further than just deferring the evaluation of some placeholders like the hostname since it uses e.g. {{if eq .kafka.deploy.mode "replicated"}} to make entire bits of yaml optional, this add to the complexity since it means the document is not valid yaml while it is in docker-app's hands, it's not until the final step further down the line that it gets fully realised.

[ChristianKniep]

I reckon I want to have the cake (templating) and eat it (skip evaluation for certain parts of the template). :) Not sure if that is a solvable problem; can I somehow quote some template code to not have it evaluated?