Cannot run program "docker-compose" - No such file or directory

[georgebax]

Hi, we have been using the docker-compose-plugin for a long time in our project. Recently, after migrating to java 17, I've noticed a strange behaviour.

Now, I can still run ./gradlew :startUpDevelopment to deploy our wildfly, but only if I run it with JAVA_HOME set to Java 11. It only takes exporting the Java Home to java 17 to break the build, and the output is the following. Is java 17 not supported somehow ?

* What went wrong:
Execution failed for task ':integration-tests:teamcityComposeBuild'.
> A problem occurred starting process 'command 'docker-compose''

* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':teamcityComposeBuild'.
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.lambda$executeIfValid$1(ExecuteActionsTaskExecuter.java:188)
        at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:282)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:186)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:174)
        at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:109)
        at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
        at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:51)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:200)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:195)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$3.execute(DefaultBuildOperationRunner.java:75)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$3.execute(DefaultBuildOperationRunner.java:68)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:153)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:68)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:62)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.lambda$call$2(DefaultBuildOperationExecutor.java:79)
        at org.gradle.internal.operations.UnmanagedBuildOperationWrapper.callWithUnmanagedSupport(UnmanagedBuildOperationWrapper.java:54)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:79)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
        at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:74)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:402)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:389)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:382)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:368)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.lambda$run$0(DefaultPlanExecutor.java:127)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:191)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:182)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:124)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:61)
Caused by: org.gradle.process.internal.ExecException: A problem occurred starting process 'command 'docker-compose''
        at org.gradle.process.internal.DefaultExecHandle.execExceptionFor(DefaultExecHandle.java:241)
        at org.gradle.process.internal.DefaultExecHandle.setEndStateInfo(DefaultExecHandle.java:218)
        at org.gradle.process.internal.DefaultExecHandle.failed(DefaultExecHandle.java:369)
        at org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:87)
        at org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable.run(CurrentBuildOperationPreservingRunnable.java:42)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:61)
Caused by: net.rubygrapefruit.platform.NativeException: Could not start 'docker-compose'
        at net.rubygrapefruit.platform.internal.DefaultProcessLauncher.start(DefaultProcessLauncher.java:27)
        at net.rubygrapefruit.platform.internal.WrapperProcessLauncher.start(WrapperProcessLauncher.java:36)
        at org.gradle.process.internal.ExecHandleRunner.startProcess(ExecHandleRunner.java:98)
        at org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:71)
        ... 4 more
Caused by: java.io.IOException: Cannot run program "docker-compose" (in directory "/Users/......"): error=2, No such file or directory
        at net.rubygrapefruit.platform.internal.DefaultProcessLauncher.start(DefaultProcessLauncher.java:25)
        ... 7 more
Caused by: java.io.IOException: error=2, No such file or directory
        ... 8 more

This is what part of the build.gradle looks like:

plugins {
    id 'com.avast.gradle.docker-compose' version '0.16.12'
}

apply plugin: 'docker-compose'

dockerCompose {
    useComposeFiles = ['docker/docker-compose.yml']
    waitForTcpPorts = false
    teamcity {
        useComposeFiles = ['docker/docker-compose.yml', 'docker/docker-compose-development.yml', 'docker/docker-compose-oidc.yml']
        forceRecreate = true
        waitForHealthyStateTimeout  = java.time.Duration.ofMinutes(5)
        waitAfterHealthyStateProbeFailure = java.time.Duration.ofSeconds(15) 
    }
...
}

// copies the war into the docker/deployments dir - then a docker image with the war file copied into it is built and run
task fastDeploy(dependsOn: ['fastDeployComposeDown', 'copyWarToDeployments', 'fastDeployComposeUp']) { 
}
// copies the war into the docker/deployments dir - then a docker image with the war file copied into it is built and run
tasks.register('startUpDevelopment') {
    dependsOn = [ ':ourproject.web:clean', 'copyWarToDeployments', 'teamcityComposeUp'] 
    group = 'Launch'
    description = 'Runs the app in docker containers with the development database and debugging support.'
}

[dbwiddis]

I can somewhat reproduce this with a different environment, but I suspect a similar root cause. In my case it's the docker command that doesn't execute, and using debug logging it's because the working directory for the command (truncated in the report above to (in directory "/Users/......")) did not exist.

It's reproducible on JDK 17.0.5 and 19.0.1, but works fine on JDK 11.0.17.

[augi]

Hi, the root cause is Cannot run program "docker-compose" (in directory "/Users/......"): error=2, No such file or directory.

So it is not able to find the docker-compose program. This execution is delegated to the OS, and I guess it should also look in PATH. So are you sure docker-compose is available in your path?

Relevant plugin code: https://github.com/avast/gradle-docker-compose-plugin/blob/main/src/main/groovy/com/avast/gradle/dockercompose/ComposeExecutor.groovy#L80-L84

[georgebax]

Hi @augi, yes, docker-compose is in my path (/usr/local/bin/), and it looks like it works now also with java 17. The only difference is that I have the export statement in my .rc file and haven't manually executed the export command from the shell. I don't know how this would affect it, but I guess it's fine for now.

[jillesvangurp]

Running into this after switching to Java 21. Something is wonky with path resolution. I have docker for mac and homebrew.

I added this workaround, which seems to work. I would suggest integrating that into the code. My docker executable lives in /usr/local/bin.

    listOf("/usr/bin/docker","/usr/local/bin/docker").firstOrNull { 
        File(it).exists()
    }?.let { docker ->
        // works around an issue where the docker 
        // command is not found
        // falls back to the default, which may work on
        // some platforms
        dockerExecutable.set(docker)
    }