Reusability of Templates to simplify setup and maintenance for multiple Docker-Hosts

[Blogshot]

Hello!

This is a feature-request.

Currently, templates are defined inside each Docker host individually like this:

VM1
  |--> Template 1
  |--> Template 2
  L--> Template 3
VM2
  |--> Template 1
  |--> Template 2
  L--> Template 3
VM3
  |--> Template 2
  |--> Template 3
  L--> Template 4

This behaviour is highly redundant and error-prone. It means that a changed option in "Template 2" has to be edited three times in total. In combination with buttons like "Container settings", which can only be expanded and not collapsed, it is difficult to keep an overview of the configuration page and enter all changes correctly.

It would be far easier to configure the Templates independently of the Docker-Hosts and just reference them similar to this:

a) Template 1
b) Template 2
c) Template 3
d) Template 4

VM1
  |--> a
  |--> b
  L--> c
VM2
  |--> a
  |--> b
  L--> b
VM3
  |--> b
  |--> c
  L--> d

In short: Moving the configuration of "Docker Agent templates" to an independent place and just referencing them in "Configure Clouds" improves user experience and reduces human error.

[pjdarton]

While I approve of your thinking (I have a similar situation where I work, only it's more than 3 VMs), it's not as simple as you suggest.

e.g. we can't "move" the templates; dropping support for host-specific templates would break the world for too many people.

We could, I guess, have a set of not-cloud-specific templates, but that would have to be in addition to host-specific templates, not "instead of". ...but that would be tricky to make the configuration easily understandable, let alone pretty. I'd suggest you take a look at the OpenStack plugin as an example - that has a two-layer configuration setup (there's a default template per OpenStack cloud, and then that's customised in one or more templates allowing most of each template to be empty) - but it's far from pretty (the "just use the default" bits aren't any less visible so it's still hard to "see the wood for the trees").

Also, IME one aspect of a template I find very useful is the ability to specify a name that's based on the name of the docker host it's running on. e.g. VM1's template 1 would have a name of "VM1template1", so that way I can see "at a glance" what builds are running where and on what template. If templates were "up a level" then that distinction would be lost and that would be a big step backwards IMO.

I think that a lot of folks who are faced with the problem you're describing turn to using configuration-as-code as the answer. e.g. either running some hand-written groovy code on the Jenkins admin console that adds/removes templates, or using JCasC.

Others go down the "swarm" route and run a standalone swarm (so that, from Jenkins' point of view, it only has one docker host to talk to - the swarm code distributes the workload over the different VMs). Note that's not docker's "swarm mode" stuff as this plugin (sadly) does not support that (I'd welcome a PR that added it tho'!) (and others use both, e.g. I have some Jenkins servers that can talk to multiple swarms, and I configure those using code)

...and Kubernetes seems to be taking over the world which further reduces the focus on this plugin.

TL;DR: It's non-trivial and there's workarounds that mitigate demand for such a feature enough that nobody's been keen enough to implement it yet.

...but if you are keen enough to do it, I'd be happy to discuss design issues up-front and do code-reviews etc. I have no intention of stopping folks making things better, but it does need to be "better for everyone".

[Daryes]

Alternate possibility : the config as code plugin for jenkins; https://plugins.jenkins.io/configuration-as-code/ And making use of the yaml templating possbilities in the yaml file.

The UI is nice to start with, but when you need to manage 8 templates for different languages, it's becoming very tedious each time you need to make a change. Using a yaml file make it very simple.

When installed => manage jenkins => config as code => view configuration. Something to know: Jenkins shows a complete configuration set, but you can give it multiple files, each with a specific part of the configuration. So you can have a file dedicated to the docker plugin configuration.

Something to keep in mind : the last loaded file must be accessible by jenkins when restarted, otherwise it'll crash.

Here's a sample of what I'm using.

# docker_cloud.yml

# Agent template - general
x-docker-node-general: &docker_node_general
  connector:
    jnlp:
      # always end the url with a /
      jenkinsUrl: "https://jenkins.yourdomain/"
      jnlpLauncher:
        webSocket: true
        workDirSettings:
          disabled: false
          failIfWorkDirIsMissing: false
          internalDir: "remoting"
          workDirPath: "/workspace"
      user: "root"
  mode: EXCLUSIVE
  pullStrategy: PULL_NEVER
  pullTimeout: 300
  remoteFs: "/home/jenkins"
  removeVolumes: true

# Agent template - docker settings
x-docker-node-templatebase: &docker_node_templatebase
  # the webSocker setting isn't given to the agent in the container, an environment var is required
  environmentsString: "JENKINS_WEB_SOCKET=true"
  # Required for Docker in Docker
  privileged: true

jenkins:
  clouds:
  - docker:
      containerCap: 10
      dockerApi:
        connectTimeout: 60
        dockerHost:
          uri: "unix:///var/run/docker.sock"
        readTimeout: 60
      name: "host1-docker-cloud"
      templates:
        - name: "docker-agent-base"
          labelString: "docker-agent base java11"
          <<: *docker_node_general
          dockerTemplateBase:
            image: "jks-agent-inbound/base"
            <<: *docker_node_templatebase

        - name: "docker-agent-nodejs10"
          labelString: "docker-agent nodejs10"
          <<: *docker_node_general
          dockerTemplateBase:
            image: "jks-agent-inbound/nodejs10"
            <<: *docker_node_templatebase

        - name: "docker-agent-nodejs12"
          labelString: "docker-agent nodejs12"
          <<: *docker_node_general
          dockerTemplateBase:
            image: "jks-agent-inbound/nodejs12"
            <<: *docker_node_templatebase
 (...)

Worth mentioning :

• each docker host has a jenkins agent on it, there's no external access to docker. Hence the unix socket
• those are custom images, using the jenkins docker agent as the base image.
• Another pipeline does the heavy lifting of building and propagating the image to the host, hence the pullStrategy set to no
• I'm using Docker in Docker, this explains some special configuration
• the connector uses the websocket, but it won't work without the JENKINS_WEB_SOCKET env variable. That makes also a more complete example

[pjdarton]

Agreed - configuration-as-code is the most practical answer for this.