bad Volumes Option UX, require multiline input

[dtothefp]

Potentially similar to https://github.com/jenkinsci/docker-plugin/issues/206 but wanted to isolate this issue and see if I'm missing something with how the "Volumes" options works under the "Container Settings" menu.

I'm running a Jenkins Master using the official Jenkins Docker image. I'm bind mounting the docker socket into the Jenkins docker container as suggested here https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/:

version: "2.1"
services:
  jenkinsdata:
    build: ${DATA_CONTAINER}
  jenkinsmaster:
    build: ${MASTER_CONTAINER}
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ${DOCKER_BIN:-/usr/bin/docker}:/usr/bin/docker
    volumes_from:
      - jenkinsdata
    ports:
      - "50000:50000"
  jenkinsnginx:
    build: ${NGINX_CONTAINER}
    ports:
      - "80:80"
    links:
      - jenkinsmaster:${MASTER_CONTAINER}

This works fine for using docker commands within builds running on master.

Now, I've switched to running jobs on a slave server using the Jenkins docker-plugin using the evarga-jenkins-slave image.

A simple freestyle build works fine on the slave, but bind mounting the docker socket into the slave container using the "Volumes" option under "Container Settings" does not seem to work if I do which docker in the build it fails. Maybe there is something I don't understand about mounting volumes in the docker slave or am passing config incorrectly?

Jenkins Version

2.32.2

Docker Version

Client:
 Version:      1.13.1
 API version:  1.26
 Go version:   go1.7.5
 Git commit:   092cba3
 Built:        Wed Feb  8 06:50:14 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.13.1
 API version:  1.26 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   092cba3
 Built:        Wed Feb  8 06:50:14 2017
 OS/Arch:      linux/amd64
 Experimental: false

Plugins

docker-workflow:1.10
mailer:1.20
workflow-job:2.10
git:3.2.0
ant:1.4
pam-auth:1.3
pipeline-graph-analysis:1.3
authentication-tokens:1.3
ssh-credentials:1.13
build-timeout:1.18
matrix-auth:1.4
ws-cleanup:0.32
pipeline-stage-tags-metadata:1.1.2
credentials-binding:1.11
ssh-agent:1.14
structs:1.6
github:1.26.2
handlebars:1.1.1
pipeline-milestone-step:1.3.1
pipeline-rest-api:2.6
workflow-support:2.14
rebuild:1.25
credentials:2.1.13
workflow-aggregator:2.5
bouncycastle-api:2.16.1
github-api:1.85
ace-editor:1.1
external-monitor-job:1.7
github-branch-source:2.0.5
icon-shim:2.0.3
gradle:1.26
windows-slaves:1.3.1
resource-disposer:0.6
junit:1.20
antisamy-markup-formatter:1.5
mapdb-api:1.0.9.0
github-organization-folder:1.6
ldap:1.14
docker-commons:1.6
jquery-detached:1.2.1
workflow-step-api:2.9
workflow-cps:2.29
pipeline-model-api:1.1.2
workflow-durable-task-step:2.10
git-client:2.4.1
pipeline-input-step:2.5
matrix-project:1.9
pipeline-stage-view:2.6
email-ext:2.57.1
workflow-basic-steps:2.4
script-security:1.27
token-macro:2.1
pipeline-stage-step:2.2
scm-api:2.1.1
durable-task:1.13
timestamper:1.8.8
pipeline-model-extensions:1.1.2
pipeline-build-step:2.4
workflow-cps-global-lib:2.7
pipeline-model-definition:1.1.2
cloudbees-folder:6.0.3
workflow-api:2.12
ssh-slaves:1.16
branch-api:2.0.8
pipeline-github-lib:1.0
pipeline-model-declarative-agent:1.1.1
momentjs:1.1.1
workflow-multibranch:2.14
plain-credentials:1.4
workflow-scm-step:2.4
git-server:1.7
ghprb:1.35.0
subversion:2.7.2
docker-plugin:0.16.2
display-url-api:1.1.1

config.xml

  <clouds>
    <com.nirima.jenkins.plugins.docker.DockerCloud plugin="docker-plugin@0.16.2">
      <name>docker</name>
      <templates>
        <com.nirima.jenkins.plugins.docker.DockerTemplate>
          <configVersion>2</configVersion>
          <labelString>docker-slave</labelString>
          <launcher class="com.nirima.jenkins.plugins.docker.launcher.DockerComputerSSHLauncher">
            <sshConnector plugin="ssh-slaves@1.16">
              <port>22</port>
              <credentialsId>JenkinsDockerSlave</credentialsId>
              <jvmOptions></jvmOptions>
              <javaPath></javaPath>
              <maxNumRetries>0</maxNumRetries>
              <retryWaitTime>0</retryWaitTime>
              <sshHostKeyVerificationStrategy class="hudson.plugins.sshslaves.verifiers.KnownHostsFileKeyVerificationStrategy"/>
            </sshConnector>
          </launcher>
          <remoteFsMapping></remoteFsMapping>
          <remoteFs>/home/jenkins</remoteFs>
          <instanceCap>1</instanceCap>
          <mode>EXCLUSIVE</mode>
          <retentionStrategy class="com.nirima.jenkins.plugins.docker.strategy.DockerOnceRetentionStrategy">
            <idleMinutes>10</idleMinutes>
            <idleMinutes defined-in="com.nirima.jenkins.plugins.docker.strategy.DockerOnceRetentionStrategy">10</idleMinutes>
          </retentionStrategy>
          <numExecutors>1</numExecutors>
          <dockerTemplateBase>
            <image>dfp/jenkins-slave</image>
            <dockerCommand></dockerCommand>
            <lxcConfString></lxcConfString>
            <hostname></hostname>
            <dnsHosts/>
            <network></network>
            <volumes>
              <string>/var/run/docker.sock:/var/run/docker.sock /usr/bin/docker:/usr/bin/docker</string>
            </volumes>
            <volumesFrom2/>
            <environment/>
            <bindPorts></bindPorts>
            <bindAllPorts>false</bindAllPorts>
            <privileged>false</privileged>
            <tty>false</tty>
            <extraHosts class="java.util.Collections$UnmodifiableRandomAccessList" resolves-to="java.util.Collections$UnmodifiableList">
              <c class="list"/>
              <list reference="../c"/>
            </extraHosts>
          </dockerTemplateBase>
          <removeVolumes>false</removeVolumes>
          <pullStrategy>PULL_LATEST</pullStrategy>
        </com.nirima.jenkins.plugins.docker.DockerTemplate>
        <com.nirima.jenkins.plugins.docker.DockerTemplate>
          <configVersion>2</configVersion>
          <labelString>docker-slave-evarga</labelString>
          <launcher class="com.nirima.jenkins.plugins.docker.launcher.DockerComputerSSHLauncher">
            <sshConnector plugin="ssh-slaves@1.16">
              <port>22</port>
              <credentialsId>JenkinsDockerSlave</credentialsId>
              <jvmOptions></jvmOptions>
              <javaPath></javaPath>
              <maxNumRetries>0</maxNumRetries>
              <retryWaitTime>0</retryWaitTime>
              <sshHostKeyVerificationStrategy class="hudson.plugins.sshslaves.verifiers.KnownHostsFileKeyVerificationStrategy"/>
            </sshConnector>
          </launcher>
          <remoteFsMapping></remoteFsMapping>
          <remoteFs>/home/jenkins</remoteFs>
          <instanceCap>1</instanceCap>
          <mode>EXCLUSIVE</mode>
          <retentionStrategy class="com.nirima.jenkins.plugins.docker.strategy.DockerOnceRetentionStrategy">
            <idleMinutes>10</idleMinutes>
            <idleMinutes defined-in="com.nirima.jenkins.plugins.docker.strategy.DockerOnceRetentionStrategy">10</idleMinutes>
          </retentionStrategy>
          <numExecutors>1</numExecutors>
          <dockerTemplateBase>
            <image>evarga/jenkins-slave</image>
            <dockerCommand></dockerCommand>
            <lxcConfString></lxcConfString>
            <hostname></hostname>
            <dnsHosts/>
            <network></network>
            <volumes>
              <string>/var/run/docker.sock:/var/run/docker.sock /usr/bin/docker:/usr/bin/docker</string>
            </volumes>
            <volumesFrom2/>
            <environment/>
            <bindPorts></bindPorts>
            <bindAllPorts>false</bindAllPorts>
            <privileged>false</privileged>
            <tty>false</tty>
            <extraHosts class="java.util.Collections$UnmodifiableRandomAccessList" resolves-to="java.util.Collections$UnmodifiableList">
              <c class="list"/>
              <list reference="../c"/>
            </extraHosts>
          </dockerTemplateBase>
          <removeVolumes>false</removeVolumes>
          <pullStrategy>PULL_LATEST</pullStrategy>
        </com.nirima.jenkins.plugins.docker.DockerTemplate>
      </templates>
      <serverUrl>tcp://<my_ip>:4243</serverUrl>
      <connectTimeout>5</connectTimeout>
      <readTimeout>15</readTimeout>
      <version></version>
      <credentialsId></credentialsId>
      <containerCap>5</containerCap>
    </com.nirima.jenkins.plugins.docker.DockerCloud>
  </clouds>

[dtothefp]

Update

So it looks like from minimal investigation into the code that volumes are split on a newline rather than a space as it suggests in the docs https://github.com/jenkinsci/docker-plugin/blob/master/docker-plugin/src/main/java/com/nirima/jenkins/plugins/docker/DockerTemplateBase.java#L144

Using docker inspect I found the string was not being split properly

//master
"HostConfig": {
            "Binds": [
                "/var/run/docker.sock:/var/run/docker.sock:rw",
                "/usr/bin/docker:/usr/bin/docker:rw"
            ],

//slave
"HostConfig": {
            "Binds": [
                "/var/run/docker.sock:/var/run/docker.sock /usr/bin/docker:rw"
            ],

Adding a newline in the Jenkins UI input allows for command like which docker to work in a build but when I run sudo docker run hello-world I'm getting a strange error

+ docker run hello-world
docker: error while loading shared libraries: libltdl.so.7: cannot open shared object file: No such file or directory
Build step 'Execute shell' marked build as failure
Finished: FAILURE

[ndeloof]

Thanks for investigating this Would probably be better for plugin to rely on new Mounts API to define bindings, and at least to offer a cleaner UI to define them. About issue running docker command it sounds like your docker image is missing minimal system libraries required by go runtime

[erenatas]

Are there no updates about this? Slave isn't created when I enter volumes correctly on my side.

[bguerin]

@pjdarton https://github.com/jenkinsci/docker-plugin/blob/master/src/main/java/com/nirima/jenkins/plugins/docker/DockerTemplateBase.java#L395 should be splitAndFilterEmpty(mountsString, " ") instead of "\n" ?

[pjdarton]

I don't think it's that simple - sometimes file paths have spaces in them 😢

Spaces in paths are a perpetual pain the backside and I wish there was a law against it but, unfortunately, at least one large multinational company decided that names like "Documents and Settings" or "Program Files (x86)" was a good idea, so a lot of folks tend to assume that it's OK to have spaces (and other weird characters) in file paths, so we can't really make the assumption that they never will have spaces.

I'm not opposed to the idea of splitting on spaces (or maybe "whitespace" in general) but if we were to choose to do that, we would also need to introduce a means of having spaces in pathnames (that doesn't rely on the space character itself). e.g. we could use gitattributes [::space::] notation, or  , %20 etc ... but all of these ideas would mean that the code would have to encode and decode the strings, and deal with auto-encoding existing configuration data if folks upgrade the plugin etc. ...which is why splitting on a linefeed character seemed to be an acceptable option - folks don't tend to put those into filenames with the expectation that everything will cope.

PS. "it was like that before I got here". The use of "\n" goes back to (well) before I got involved with this plugin and it didn't annoy me enough for me to change it ... but if it annoys you then feel free to do a PR ... as long as the PR still copes with spaces in filepaths 😉