Could not create parent directory for lock file when a non-root user

[markieww]

How to run the image using a non-root user offline?

The previously downloaded already locally available gradle distribution (downloaded at the moment of building the https://github.com/docToolchain/docker-image/blob/master/alpine/Dockerfile) should be used.

Problem

I integrated this docker-image in my Jenkins-CI pipeline like this.

pipeline {

    agent {
        docker {
            image 'rdmueller/doctoolchain:rc-1.2.0'
        }
    }

    stages {
        stage ("Generate PDFs") {
            steps {
                sh "doctoolchain . generatePDF"
            }
        }
}

Which is equivalent to

#!/usr/bin/env bash
docker run --user jenkinbuilduser --rm -it --entrypoint /bin/bash -v ${PWD}:/project rdmueller/doctoolchain:rc-1.2.0 \
-c "doctoolchain . $1 $2 $3 $4 $5 $6 $7 $8 $9 -PinputPath=src/main/asciidoc -PmainConfigFile=config/docToolchain.groovy && exit"

or

#!/usr/bin/env bash
docker run --user 501 --rm -it --entrypoint /bin/bash -v ${PWD}:/project rdmueller/doctoolchain:rc-1.2.0 \
-c "doctoolchain . $1 $2 $3 $4 $5 $6 $7 $8 $9 -PinputPath=src/main/asciidoc -PmainConfigFile=config/docToolchain.groovy && exit"

from your documentation. Note the added option --user jenkinbuilduser and --user 501.

But this leads to exceptions like

Exception in thread "main" java.lang.RuntimeException: Could not create parent directory for lock file /docToolchain/?/.gradle/wrapper/dists/gradle-6.0.1-bin/1lxlpkiy24sb18odw96cp4ojv/gradle-6.0.1-bin.zip.lck
    at org.gradle.wrapper.ExclusiveFileAccessManager.access(ExclusiveFileAccessManager.java:43)
    at org.gradle.wrapper.Install.createDist(Install.java:48)
    at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:107)
    at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:63)

My investigations showed that /root/.gradle (which holds the downloaded gradle distribution created by https://github.com/docToolchain/docker-image/blob/master/alpine/Dockerfile#L41) is not accessible by a non-root user like jenkinbuilduser. gradlew interprets it that no distribution is locally available and starts the download of the gradle distribution into /docToolchain/?/.gradle. This directory is read-only, which leads to the exception.

First workaround (non-working)

My first workaround was to run the image as user root.

pipeline {

    agent {
        docker {
            image 'rdmueller/doctoolchain:rc-1.2.0'
            args """-u root """
        }
    }

    stages {
        stage ("Generate PDFs") {
            steps {
                sh "doctoolchain . generatePDF"
            }
        }
}

Which is equivalent to

#!/usr/bin/env bash
docker run --rm -it --user root --entrypoint /bin/bash -v ${PWD}:/project rdmueller/doctoolchain:rc-1.2.0 \
-c "doctoolchain . $1 $2 $3 $4 $5 $6 $7 $8 $9 -PinputPath=src/main/asciidoc -PmainConfigFile=config/docToolchain.groovy && exit"

(Note the option --user root)

This way there is no exception, the generation works, but the generated file are owned by user root on the host file-system. This is not acceptable, because these files cannot be removed anymore (at least not by a next build steps within the build job. It can only be deleted manually by a fellow administrator with root-user rights)

Second workaround (partially working)

Set a non-root user and a rw-directory for the gradle distribution.

#!/usr/bin/env bash
docker run --user 501 -e GRADLE_USER_HOME='/project/.gradlecustom' --rm -it ---entrypoint /bin/bash -v ${PWD}:/project rdmueller/doctoolchain:rc-1.2.0 \
-c "doctoolchain . $1 $2 $3 $4 $5 $6 $7 $8 $9 -PinputPath=src/main/asciidoc -PmainConfigFile=config/docToolchain.groovy && exit"

gradlew will download a distribution to /project/.gradlecustom , the generation succeeds and the generated files are owned by a non-root user.

The problem: A buildjob should not have access to the internet.

The remaining problem

The issue is still remaining: How to run the image using a non-root user offline. The previously downloaded already locally available gradle distribution should be used.

[rdmueller]

thanx for your problem report. I do understand the problem. I guess the best solution is to modify the docker file to fit your needs: https://github.com/docToolchain/docker-image/blob/master/alpine/Dockerfile

I am currently trying to re-write docToolchain in a way that it is more usable - this will include a new docker file. But at the moment, the docker file is more experimental. It seems to work for some use cases but anfortunately not for yours.

I guess the solution needed is to switch the user in the docker file in order to install sdkman and gradle as another user and run the docker file with this user. Unfortunately, I lack the time to try this out but I would be happy to accept a PR.

[markieww]

Hi @rdmueller,

here is our temporary solution.

In our case we know that the container will started using UID 300 GID 300. So we create such a user (jenkinsuser with 300:300) and its home directory in the image itself. Gradle in the Dockerfile is initially invoked within /home/jenkinsuser/doctoolchain by jenkinsuser and thus all the .gradle-caches and -downloads are accessible for the user 300, when running the container later.

FROM openjdk:8u191-jdk-alpine

# see https://github.com/docker-library/openjdk/issues/73
ENV LC_CTYPE en_US.UTF-8

# Workaround für https://github.com/docToolchain/docker-image/issues/10
#
# Jenkins startet Docker-Images aus Jenkins-Pipelines wie
#
#    agent {
#        docker {
#            image 'rdmueller/doctoolchain:rc-1.2.0'
#        }
#    }
#
# heraus mit
#
#    docker run -t -d -u 300:300 -w /home/jenkins-slave/workspace/... -e ******** rdmueller/doctoolchain:rc-1.2.0
#
# Also dem Jenkins-Nutzer (UID 300:300) - extern definiert durch unsere CI-Infrastruktur
#
# Dieser Nutzer (und sein Homeverzeichnis) sind nicht im Image vorhanden, also erstellen wir einen solchen im Docker-Image.
USER root
RUN echo "jenkinsuser:x:300:300:300:/home/jenkinsuser:/bin/ash">>/etc/passwd
RUN mkdir /home/jenkinsuser
RUN chown jenkinsuser /home/jenkinsuser
RUN chgrp abuild /home/jenkinsuser

USER root
RUN echo "add needed tools" && \
    apk add --no-cache curl wget zip unzip git bash \
    git \
    graphviz \
    python \
    ruby \
    py-pygments \
    libc6-compat \
    ttf-dejavu

SHELL ["/bin/bash", "-c"]

# Ausfuehrung von Gradle mit dem gemappten Nutzer.
# Dadurch sind die gecachten Dateien von /home/jenkinsuser/.gradle beim Start des Images mit -u 300:300 zugreifbar
USER jenkinsuser

# Hier ggf. den Unternehmensproxy eintragen
# ENV GRADLE_OPTS="-Dhttp.proxyHost=server.org -Dhttp.proxyPort=8080 -Dhttps.proxyHost=server.org -Dhttps.proxyPort=8080"

RUN     cd /home/jenkinsuser && \
        pwd && \
        env | sort && \
        git clone https://github.com/docToolchain/docToolchain.git && \
        cd docToolchain && \
        git checkout v1.2.0 && \
        git submodule update -i && \
        ./gradlew tasks && \
        ./gradlew

ENV PATH="/home/jenkinsuser/docToolchain/bin:${PATH}"

USER root
RUN mkdir /project

WORKDIR /project

VOLUME /project

ENTRYPOINT /bin/bash

[rdmueller]

great to see that you've found a solution which works for you. But I also have the feeling that this is a solution from which others could benefit. Should be publish this as an additional dockerfile?

[rdmueller]

today I tried to get it to work and here is what I came up with:

FROM openjdk:13-jdk-alpine

# see https://github.com/docker-library/openjdk/issues/73
ENV LC_CTYPE en_US.UTF-8

RUN addgroup -S dtcgroup && adduser -S dtcuser -G dtcgroup

RUN echo "add needed tools" && \
    apk add --no-cache curl wget zip unzip git bash \
    git \
    graphviz \
    python \
    ruby \
    py-pygments \
    libc6-compat \
    ttf-dejavu && \
    gem install rdoc --no-document && \
    gem install pygments.rb

# Add pandoc
# https://github.com/advancedtelematic/dockerfiles/blob/master/doctools/Dockerfile
#RUN apk add --no-cache cmark --repository http://nl.alpinelinux.org/alpine/edge/testing && \
#    apk add --no-cache --allow-untrusted pandoc --repository https://conoria.gitlab.io/alpine-pandoc/

SHELL ["/bin/bash", "-c"]

USER dtcuser
WORKDIR /home/dtcuser

RUN echo "Install sdkman" &&\
    curl -s "https://get.sdkman.io" | bash && \
    chmod +x $HOME/.sdkman/bin/sdkman-init.sh && \
    $HOME/.sdkman/bin/sdkman-init.sh

RUN echo "Install java, groovy" && \
    source $HOME/.sdkman/bin/sdkman-init.sh
#    sdk install groovy 2.5.5

ENV GRADLE_USER_HOME=/home/dtcuser/.gradle

RUN     git clone --depth 1 --branch ng https://github.com/docToolchain/docToolchain.git  && \
        cd docToolchain && \
        git fetch --tags && \
        git checkout v2.0.0-rc4 && \
        git submodule update -i && \
        # remove .git folders
        rm -rf `find -type d -name .git` && \
        umask g+w && \
        ./gradlew tasks && \
        ./gradlew dependencies && \
        chmod g+s $GRADLE_USER_HOME && \
        chmod -R 770 $GRADLE_USER_HOME && \
        rm -r $GRADLE_USER_HOME/daemon && \
        PATH="/home/dtcuser/docToolchain/bin:${PATH}"

ENV PATH="/home/dtcuser/docToolchain/bin:${PATH}"

USER dtcuser

WORKDIR /project

VOLUME /project

ENTRYPOINT /bin/bash

you can now specify a user as --user $(id -u):dtcgroup

A new image will be soon available as rdmueller/doctoolchain:v2.0.0-rc10 (build is running)

[rdmueller]

ok, the newest build will allow you to run the container with any user. For running it locally, your own user makes most sense: --user $(id -u):$(id -g)

[arcusbude]

ok, the newest build will allow you to run the container with any user. For running it locally, your own user makes most sense: --user $(id -u):$(id -g)

which newest build ? iam still getting the same error: Could not create parent directory for lock file /docToolchain/?/.gradle/wrapper/dists/gradle-6.0.1-bin/1lxlpkiy24sb18odw96cp4ojv/gradle-6.0.1-bin.zip.lck

is there a chance to get a docker version of doctoolchain running as non root user ?

[rdmueller]

this is the currently used dockerfile to create the rdmueller/doctoolchain:v2.0.3 image: https://github.com/docToolchain/docker-image/blob/ng-beta/alpine/Dockerfile

And this is how it is used by the wrapper: https://github.com/docToolchain/doctoolchain.github.io/blob/master/dtcw#L209

does this help?