Environment Variables are not passed when building the container (JENKINS example)

[Jgerardopine]

ISSUE TYPE

• Bug Report

Hi,

I am struggling with a similar situation exposed here but it is a bit different as he has a workaround when running it, whereas I need it when being build. I followed the example contained here WHAT I AM TRYING TO DO: Create a jenkins container. My Dockerfile works fine but when translating it (manually, not using the import option of ansible-container) I have problems because the environment variables needed for the build are not taken in consideration, thus it fails.

container.yml (uncomment the environment variables when testing)

version: "2"
settings:
  conductor:
    base: debian:stretch
#    environment:
#      JAVA_OPTS: "-Xmx4096m -Djenkins.install.runSetupWizard=false"
      #JENKINS_UC_DOWNLOAD: "http://ftp-chi.osuosl.org/pub/jenkins"
#      JENKINS_UC_DOWNLOAD: "http://mirrors.jenkins-ci.org"
#      JENKINS_OPTS: "--logfile=/var/log/jenkins/jenkins.log --webroot=/var/cache/jenkins/war --argumentsRealm.passwd.jenkins={{ JENKINS_PWD }} --argumentsRealm.roles.jenkins=admin --httpPort={{ HTTP_PORT }}"

  project_name: jenkins

services:
  jenkins-master:
    from: jenkins/jenkins:2.73.1
    roles:
      - master
    user: jenkins
#    environment:
#      JAVA_OPTS: "-Xmx4096m -Djenkins.install.runSetupWizard=false"  
      #JENKINS_UC_DOWNLOAD: "http://ftp-chi.osuosl.org/pub/jenkins"
#      JENKINS_UC_DOWNLOAD: "http://mirrors.jenkins-ci.org"
#      JENKINS_OPTS: "--logfile=/var/log/jenkins/jenkins.log --webroot=/var/cache/jenkins/war --argumentsRealm.passwd.jenkins={{ JENKINS_PWD }} --argumentsRealm.roles.jenkins=admin --httpPort={{ HTTP_PORT }}"
    ports:
      - '{{ EXPOSED_PORT }}'
    volumes_from:
      - jenkins_jenkins-data_1

tasks/main.yml

---

- name: Make log directory
  #become: yes
  file:
    name: /var/log/jenkins
    state: directory
    owner: jenkins
    group: jenkins

- name: Make cache directory
  #become: yes
  file:
    name: /var/cache/jenkins
    state: directory
    owner: jenkins
    group: jenkins

- name: update
  #become: yes
  apt:
    update_cache: yes

#- name: Export variable to change download center
#  shell: export JENKINS_UC_DOWNLOAD=http://mirrors.jenkins-ci.org
#  args:
#    executable: /bin/bash
- name: Copy plugins.txt
  copy:
    src: "plugins.txt"
    dest: "/usr/share/jenkins/ref/plugins.txt"
    owner: jenkins

- name: Install plugins
  shell: xargs /usr/local/bin/install-plugins.sh < /usr/share/jenkins/ref/plugins.txt
  #become: yes
  #become_user: jenkins
  #async: 1000
  #poll:  100
  register: task_result
  until: task_result.rc == 0
  retries: 3
  delay: 2
  ignore_errors: yes
  args:
    executable: /bin/bash

- name:
  shell: echo 2.0 > /usr/share/jenkins/ref/jenkins.install.UpgradeWizard.state
  #become: yes
  #become_user: jenkins
  args:
    executable: /bin/bash
- name: copy basic security script
  #become: yes
  #become_user: jenkins
  copy:
    src: "basic-security.groovy"
    dest: "/usr/share/jenkins/init.groovy.d/basic-security.groovy"

Dockerfile

FROM jenkins/jenkins:2.73.1
MAINTAINER Jose Pineda

ARG HTTP_PORT=8082
ARG EXPOSED_PORT=8080
ARG JENKINS_PWD= yourpassword

USER root
RUN mkdir /var/log/jenkins
RUN mkdir /var/cache/jenkins
RUN chown -R jenkins:jenkins /var/log/jenkins
RUN chown -R jenkins:jenkins /var/cache/jenkins
RUN apt-get update
USER jenkins

ENV JAVA_OPTS="-Xmx4096m -Djenkins.install.runSetupWizard=false"

#Install plugins
ENV JENKINS_UC_DOWNLOAD="http://mirrors.jenkins-ci.org"

COPY conf/plugins.txt /usr/share/jenkins/ref/plugins.txt
RUN xargs /usr/local/bin/install-plugins.sh < /usr/share/jenkins/ref/plugins.txt

# "For 2.x-derived images, you may also want to" - we're 2.0 dervied, so we want this
RUN echo 2.0 > /usr/share/jenkins/ref/jenkins.install.UpgradeWizard.state
COPY conf/basic-security.groovy /usr/share/jenkins/init.groovy.d/basic-security.groovy

ENV JENKINS_OPTS="--logfile=/var/log/jenkins/jenkins.log --webroot=/var/cache/jenkins/war --argumentsRealm.passwd.jenkins=${JENKINS_PWD} --argumentsRealm.roles.jenkins=admin --httpPort=${HTTP_PORT}"
#ENV JENKINS_OPTS="--handlerCountStartup=100 --handlerCountMax=300 --logfile=/var/log/jenkins/jenkins.log --webroot=/var/cache/jenkins/war"

EXPOSE ${EXPOSED_PORT}

DEBUG ERROR:

TASK [master : Install plugins] ************************************************
task path: /src/roles/master/tasks/main.yml:36
Using module file /usr/local/lib/python2.7/dist-packages/ansible/modules/commands/command.py
<c1ddc6b6ebf84016bd93dcc64ecbfe515775ab50dd49a07e98f4de5bd4a96514> ESTABLISH DOCKER CONNECTION FOR USER: root
<c1ddc6b6ebf84016bd93dcc64ecbfe515775ab50dd49a07e98f4de5bd4a96514> EXEC ['/usr/local/bin/docker', 'exec', '-i', u'c1ddc6b6ebf84016bd93dcc64ecbfe515775ab50dd49a07e98f4de5bd4a96514', u'/bin/sh', '-c', u"/bin/sh -c 'echo ~ && sleep 0'"]
<c1ddc6b6ebf84016bd93dcc64ecbfe515775ab50dd49a07e98f4de5bd4a96514> EXEC ['/usr/local/bin/docker', 'exec', '-i', u'c1ddc6b6ebf84016bd93dcc64ecbfe515775ab50dd49a07e98f4de5bd4a96514', u'/bin/sh', '-c', u'/bin/sh -c \'( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1509372427.7-260997527029860 `" && echo ansible-tmp-1509372427.7-260997527029860="` echo /root/.ansible/tmp/ansible-tmp-1509372427.7-260997527029860 `" ) && sleep 0\'']
<c1ddc6b6ebf84016bd93dcc64ecbfe515775ab50dd49a07e98f4de5bd4a96514> PUT /tmp/tmpEcoa7L TO /root/.ansible/tmp/ansible-tmp-1509372427.7-260997527029860/command.py
.
.
.
FAILED - RETRYING: Install plugins (3 retries left).Result was: {
    "attempts": 1,
    "changed": true,
    "cmd": "xargs /usr/local/bin/install-plugins.sh < /usr/share/jenkins/ref/plugins.txt",
    "delta": "0:04:28.413555",
    "end": "2017-10-30 14:11:36.438883",
    "failed": true,
    "invocation": {
        "module_args": {
            "_raw_params": "xargs /usr/local/bin/install-plugins.sh < /usr/share/jenkins/ref/plugins.txt",
            "_uses_shell": true,
            "chdir": null,
            "creates": null,
            "executable": "/bin/bash",
            "removes": null,
            "stdin": null,
            "warn": true
        }
    },
    "msg": "non-zero return code",
    "rc": 123,
    "retries": 4,
    "start": "2017-10-30 14:07:08.025328",
    "stderr": "Failed to download plugin: credentials or credentials-plugin\nFailed to download plugin: authentication-tokens or authentication-tokens-plugin\nFailed to download plugin: blueocean-commons or blueocean-commons-plugin\nFailed
.
.
.
    "stdout": "Creating initial locks...\nAnalyzing war...\nDownloading plugins...\nDownloading plugin: credentials from https://updates.jenkins.io/download/plugins/credentials/latest/credentials.hpi\nDownloading plugin: blueocean-commons from https://updates.jenkins.io/download/plugins/blueocean-commons/latest/blueocean-commons.hpi\nDownloading plugin: authentication-tokens from https://updates.jenkins.io/download/plugins/authentication-tokens/latest/authentication-tokens.hpi ...
    "stdout_lines": [
        "Creating initial locks...",
        "Analyzing war...",
        "Downloading plugins...",
        "Downloading plugin: credentials from https://updates.jenkins.io/download/plugins/credentials/latest/credentials.hpi",
        "Downloading plugin: blueocean-commons from **https://updates.jenkins.io/**download/plugins/blueocean-commons/latest/blueocean-commons.hpi",

SUMMARY

WHAT I HAVE DONE:

• I added the environment variables in the conductor -- FAILS
• I added the environment variables in the service itself --FAILS
• I added the environment variables in the task itself -- FAILS
• Pass thevariables via env.yml as suggested by @shalomb -- FAILS
• I added the env variables as a task with shell (export VAR=variable) -- FAILS

As you can see the ENV JENKINS_UC_DOWNLOAD is "http://mirrors.jenkins-ci.org" but when it builds the container it still uses the default URL. As you can see it is important to make those variables available for the building process as there are some of them that needs to be passed to the container so it can work properly when being build.

Do you have a workaround or maybe getting closer to get this feature working anytime soon?

STEPS TO REPRODUCE

sudo ansible-container --debug build

or

sudo ansible-container --vars-files env.yml --debug build

EXPECTED RESULTS

Cleaning up locks
 ---> feda9e269187
Removing intermediate container 9dcc58d41f47
Step 16/19 : RUN echo 2.0 > /usr/share/jenkins/ref/jenkins.install.UpgradeWizard.state
 ---> Running in 7a57afd93fd3
 ---> 7a8cc10552ab
Removing intermediate container 7a57afd93fd3
Step 17/19 : COPY conf/basic-security.groovy /usr/share/jenkins/init.groovy.d/basic-security.groovy
 ---> 523f048a8ebf
Removing intermediate container eaa60d3eca6e
Step 18/19 : ENV JENKINS_OPTS "--logfile=/var/log/jenkins/jenkins.log --webroot=/var/cache/jenkins/war --argumentsRealm.passwd.jenkins=l0vet3ch --argumentsRealm.roles.jenkins=admin --httpPort=${HTTP_PORT}"
 ---> Running in 106411b778db
 ---> 734096049606
Removing intermediate container 106411b778db
Step 19/19 : EXPOSE ${exposed_port}
 ---> Running in 414bf9b28960
 ---> 86475e1ae37c
Removing intermediate container 414bf9b28960
Successfully built 86475e1ae37c

ACTUAL RESULTS

It fails when trying any different way there si to include ENV VARIABLES

...
        "Failed to download plugin: pipeline-graph-analysis or pipeline-graph-analysis-plugin",
        "Failed to download plugin: subversion or subversion-plugin",
        "Failed to download plugin: pipeline-milestone-step or pipeline-milestone-step-plugin",
        "Failed to download plugin: variant or variant-plugin",
        "Failed to download plugin: docker-plugin or docker-plugin-plugin",
        "Failed to download plugin: matrix-project or matrix-project-plugin",
        "Failed to download plugin: sse-gateway or sse-gateway-plugin",
        "Some plugins failed to download! Not downloaded: credentials",
        "Not downloaded: authentication-tokens",
        "Not downloaded: blueocean-commons",
        "Not downloaded: structs",
        "Not downloaded: script-security",
        "Not downloaded: display-url-api",

...

Thanks for your help and support in advance.

[bmcgair]

No response? Having a very similar problem with --with-variables option. I thought it would work similar to --extra-vars in ansible-playbook but no dice. Has no real documentation and AFAICT it does nothing.

[Jgerardopine]

Hi @bmcgair , They didn't answer but I found the way to do it. You need to use the environment variables under your role:

services:
  jenkins-master:
    from: "image"
    roles:
      - role: master
        environment:
          JENKINS_UC_DOWNLOAD: http://mirrors.jenkins-ci.org
          JAVA_OPTS: "-Xmx4096m -Djenkins.install.runSetupWizard=false"
          JENKINS_OPTS: "--logfile=/var/log/jenkins/jenkins.log --webroot=/var/cache/jenkins/war --argumentsRealm.passwd.jenkins={{ JENKINS_PWD }} --argumentsRealm.roles.jenkins=admin --httpPort={{ HTTP_PORT }} --prefix=/jenkins"

After doing it this way it works perfectly. Hope it helps.