Cannot run inside of docker due to user permissions

[jinglejengel]

Semi related to https://github.com/alexcojocaru/elasticsearch-maven-plugin/issues/68

When attempting to run inside of a docker container, the following is returned from a maven verify:

[2018-09-19T23:52:52,197][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root

While elasticsearch surely shouldn't run as root in a typical environment, is it not possible to override the user attribute for this to account for things like containers where the typical POSIX model is not necessarily at play, at the very least for unit/integration tests?

[alexcojocaru]

As stated in #68 , I believe implementing this is not going to address the root cause. In the case of using Docker, could you set the user in the Dockerfile to a non root user (see https://docs.docker.com/engine/reference/builder/#user), so that the process uses it to execute?

[jinglejengel]

It would take a little bit of re-work if you're looking at things that use community based images (e.g. https://hub.docker.com/_/maven/). While there are options to run as non-root, it makes needing to maintain extra utility images that much more of a pain, rather than just being able to plug an play.

I definitely understand the choice made, but I'm wondering if there's ways the plugin can work around it, or at least introduce flags to work around it.

[alexcojocaru]

Given the common use of the use case you described, I thought of taking another stab at this problem.

Could you provide details on the docker based build process you tried to use? If running as root is only needed to support the docker use case, then I could disregard some of the concerns (ie. restoring the ES directory ownership, dealing with errors in that case, etc).

[jinglejengel]

@alexcojocaru sure thing, and appreciate picking this up :D

We have an integration test that is run inside of our CI pipeline which runs as a docker container. In our POM.xml we define a test cluster to use during integration with the following:

            <plugin>
                <groupId>com.github.alexcojocaru</groupId>
                <artifactId>elasticsearch-maven-plugin</artifactId>
                <version>5.6</version>
                <configuration>
                    <clusterName>testCluster</clusterName>
                    <transportPort>9500</transportPort>
                    <httpPort>9400</httpPort>
                    <version>5.6.9</version>
                    <pathInitScript>src/test/resources/elasticsearch/Init.script</pathInitScript>
                </configuration>
                <executions>
                    <execution>
                        <id>start-elasticsearch</id>
                        <phase>process-test-classes</phase>
                        <goals>
                            <goal>runforked</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>stop-elasticsearch</id>
                        <phase>prepare-package</phase>
                        <goals>
                            <goal>stop</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

This docker container has maven installed and simply invokes the integration tests via:

mvn --batch-mode integration-test failsafe:verify -Dunit.skip=true -Dinteg.skip=false"

Since this runs as the root user inside the container, the cluster fails to start, resulting in our tests failing.

[alexcojocaru]

Can you link me to the public base image for your CI pipeline image?

[jinglejengel]

Sure: https://hub.docker.com/_/maven/ (specifically 3.5.2-alpine)

[alexcojocaru]

I set up a very simple maven project which run ES during the integration-test phase and I could build it with: docker run -it --rm --name es-test -v ~/.m2:/var/maven/.m2 -v "$(pwd)":/usr/src/mymaven -u 1000 -w /usr/src/mymaven -e MAVEN_CONFIG=/var/maven/.m2 maven:3.5.2-alpine mvn clean install

Note the -u 1000 option in the command above. The uid of the local user on the host is 1000, and that option makes the maven process run as the local user. I found that on https://hub.docker.com/_/maven/

[jinglejengel]

Ah, sorry, missed out on that detail from my end -- We're using Gitlab CI runners which are executing in Kubernetes. I'd need to do some more heavy digging, but I don't think we can easily specify the user this runs as without baking a user into the image which is less than ideal.

[alexcojocaru]

A quick search shows that it would be doable though config only, without modifying the image.

I also did some research on how the plugin could run ES as root: it would have to create a user in the container, then use the runuser utility to run the ES server as that user. That would only work if runuser is available in the container (which is not for the vanilla maven image); if not provided by the base image, it would have to be installed while building the image.

[AndreaGiardini]

We are hitting the same problem with Jenkins/Kubernetes and I do not manage to get the plugin working. In particular I struggle to evade the bootstrap checks:

      <plugin>
        <groupId>com.github.alexcojocaru</groupId>
        <artifactId>elasticsearch-maven-plugin</artifactId>
        <configuration>
          <version>${version.elasticsearch}</version>
          <logLevel>DEBUG</logLevel>
          <instanceSettings>
            <properties>
              <discovery.type>single-node</discovery.type>
            </properties>
          </instanceSettings>
        </configuration>
      </plugin>

Even with this configuration (with discovery.type=single-node) the bootstrap checks are enforced

[INFO] Elasticsearch[0]: Executing command '[bin/elasticsearch, -p, pid, -Ecluster.name=test, -Ehttp.port=9200, -Etransport.tcp.port=9300, -Ehttp.cors.enabled=true, -Ediscovery.type=single-node]' in directory '/home/agiardini/git/camunda/zeebe/zeebe/exporters/elasticsearch-exporter/target/elasticsearch0'
[DEBUG] Waiting  up to 30s for the Elasticsearch instance to start ...
[2018-12-18T15:50:26,862][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [unknown] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:140) ~[elasticsearch-6.5.3.jar:6.5.3]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:127) ~[elasticsearch-6.5.3.jar:6.5.3]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.5.3.jar:6.5.3]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.5.3.jar:6.5.3]
        at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.5.3.jar:6.5.3]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.5.3.jar:6.5.3]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:86) ~[elasticsearch-6.5.3.jar:6.5.3]
Caused by: java.lang.RuntimeException: can not run elasticsearch as root
        at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:103) ~[elasticsearch-6.5.3.jar:6.5.3]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170) ~[elasticsearch-6.5.3.jar:6.5.3]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.5.3.jar:6.5.3]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-6.5.3.jar:6.5.3]
        ... 6 more

[alexcojocaru]

@AndreaGiardini Elasticsearch just cannot be run as root, no matter how you configure it. It's a hard check and the ES developers are not willing to remove the restriction. Have you read the previous comments on this issue? Depending on your docker image, you may be able to override the user you execute the plugin / elasticsearch as.

[AndreaGiardini]

@alexcojocaru I actually managed to workaround the problem. I don't know the details of GitLab CI since I have never used it but maybe it might be useful to @Joeskyyy too

In my case we are running Jenkins on Kubernetes, using the kubernetes-plugin to spawn Jenkins slaves. In order to solve the problem I had to specify the following podSpec:

---
apiVersion: v1
kind: Pod
metadata:
  labels:
    agent: test
spec:
  nodeSelector:
    cloud.google.com/gke-nodepool: test
  containers:
    - name: maven
      image: maven:3.5-jdk-8
      command: ["cat"]
      securityContext:
        runAsUser: 10000
      tty: true
      env:
        - name: LIMITS_CPU
          valueFrom:
            resourceFieldRef:
              resource: limits.cpu
        - name: JAVA_TOOL_OPTIONS
          value: |
            -XX:+UnlockExperimentalVMOptions
            -XX:+UseCGroupMemoryLimitForHeap
      resources:
        limits:
          cpu: 4
          memory: 16Gi
        requests:
          cpu: 4
          memory: 16Gi

The magic is done by the securityContext section.

The number 10000 is not casual and it is not related to the maven docker image, but it's the uid of the jenkins user (running in its separate jnlp container).

➜ docker run -it jenkins/jnlp-slave:alpine  id            
uid=10000(jenkins) gid=10000(jenkins) groups=10000(jenkins)

[alexcojocaru]

Thanks for that, @AndreaGiardini . I hope it will help others which run into similar issues.

[treilhes]

Hello, I was also running the plugin as part of gitlab ci on kubernetes/openshift. To make it run, you only need to use a maven image built for non root usage like jtim/maven-non-root

specifying the image in your gitlab-ci file as follow image: jtim/maven-non-root:3.5.4-jdk-8-alpine

[alexcojocaru]

Thanks for the details, @treilhes . FWIW I am going to close this issue soon and add a short note to the README with a link to this thread.

[alexcojocaru]

added a note to this in the README