How to use dcompose plugin inside a multistage-docker build

[alexdinescu]

Hi,

Awesome job with this! I am trying to use the plugin in a multistage dockerfile. Something like:

FROM gradle:jdk11 as builder
COPY --chown=gradle:gradle . /home/gradle/src
RUN gradle build

FROM openjdk:11-jre-slim
COPY --from=builder /home/gradle/src/build/libs/app.jar app.jar
.........

The gradle build requires a database, so I'm trying to use the plugin to spawn it inside a container:

dcompose {
    database {
        image = 'myImage'
    }
}

However, the plugin isn't able to create the container, as it doesn't have access to the docker from within the container doing the build. I have tried to add a volume to the docker.sock but the result remained the same.

* What went wrong:
Execution failed for task ':createDefaultNetwork'.
> Docker command failed: org.newsclub.net.unix.AFUNIXSocketException: No such file or directory (socket: /run/docker.sock)

Do you have any sample or hint on how to achieve this ? Note I am trying this on windows using docker-for-windows, but I expect the solution to work cross-platform. Thanks!

[chrisgahlert]

When running the Gradle build inside a Docker Container itself, the plugin still needs access to a Docker Daemon.

There are multiple options, on how to achieve this:

1. On Linux, you could start a container e.g. by mounting the Unix socket into the Container: docker run -v /var/run/docker.sock:/var/run/docker.sock. However this option is not available here, as you are using a multistage build and you cannot mount host files during the build phase. (As far as I know of...) Therefore you would need to go away from using a multistage Docker build...
2. When communicating with the Docker Daemon via TCP (on Docker for Windows you can enable access to the port 2375), you can run the build passing in the required environment variables like docker build --build-arg DOCKER_HOST=tcp://<ip of host>:2375 --build-arg DOCKER_TLS_VERIFY=0. (The <ip of host> is the IP address, where the Docker Daemon is running.)
3. One other options would be to launch multiple containers like this:
   • You run an instance of Docker in Docker, e.g. docker run -d --name dind docker:18-dind
   • You run your Docker build in a Container having access to the dind container: docker run -it --name build -v "$(pwd):/home/gradle/src" --link dind -e DOCKER_HOST=tcp://dind:2375 -e DOCKER_TLS_VERIFY=0 gradle:jdk11 gradle build
   • You copy the produced file from the container e.g. by using docker cp build:/home/gradle/src/build/libs/app.jar . and the use the produced file to run your actual build.

However "the correct way" to use this plugin would be: You run the build itself on the host and have to make sure that it can access the Docker Daemon somehow. (As it really depends on your solution like Docker for Windows/Mac, Docker Toolbox for Windows/Mac, Linux, ... this has to be done by each developer individually.) You then use this plugin to produce the actual Docker image (in your example the 2nd stage of the build). So your resulting build.gradle should look something like this:

plugins {
    id "com.chrisgahlert.gradle-dcompose-plugin" version "0.14.1"
}
apply plugin: 'java'

dcompose {
  app {
    buildFiles = project.copySpec {
      from jar
    }
    repository = 'myregistry.com/where/this/image/should/be/deployed:to'
  }
  buildDb {
    image = 'myregistry.com/where/this/image/should/be/pulled:from'
    deploy = false
    portBindings = ['1234'] // the db port, that needs to be accessed...
  }
}

createBuildDbContainer.upToDateWhen { false } // to always restart the db

test {
  dependsOn startBuildDbContainer
  finalizedBy removeBuildDbContainer

  doFirst {
    systemProperty 'db.host', dcompose.buildDb.dockerHost
    systemProperty 'db.port', dcompose.buildDb.findHostPort(1234)
    }
  doLast {
    systemProperties.remove 'db.host'
    systemProperties.remove 'db.port'
  }
}

[alexdinescu]

Thanks a lot for the detailed answer. I think that as much as I would like the multi-stage docker build solution I would move away from it as of now, as it creates more problems than it resolves... considering my current needs