search

bmuschko / gradle-docker-plugin

Gradle plugin for managing Docker images and containers.
https://bmuschko.github.io/gradle-docker-plugin/current/user-guide/
Apache License 2.0
1.23k stars 362 forks source link

docker for mac 1.12 cannot connect #235

Closed bitsofinfo closed 7 years ago

bitsofinfo commented 8 years ago

Getting this.

Tried setting DOCKER_HOST to the docker socket but this gradle task still doesn't work

[dockerjava-jaxrs-async-0] ERROR com.github.dockerjava.core.async.ResultCallbackTemplate - Error during callback
org.apache.http.conn.HttpHostConnectException: Connect to localhost:2375 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1, localhost/fe80:0:0:0:0:0:0:1%1] failed: Connection refused
        at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
        at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:71)
        at com.github.dockerjava.jaxrs.connector.ApacheConnector.apply(ApacheConnector.java:437)

If i try

docker {
    url = "unix:///var/run/docker.sock"
}

I get 

java.lang.UnsatisfiedLinkError: Could not find library in classpath, tried: [libjunixsocket-macosx-1.8-x86_64.dylib, libjunixsocket-macosx-1.5-x86_64.dylib]
        at org.newsclub.net.unix.NativeUnixSocket.load(NativeUnixSocket.java:81)
        at org.newsclub.net.unix.NativeUnixSocket.<clinit>(NativeUnixSocket.java:112)
        at org.newsclub.net.unix.AFUNIXSocket.<init>(AFUNIXSocket.java:36)
        at org.newsclub.net.unix.AFUNIXSocket.newInstance(AFUNIXSocket.java:50)
        at com.github.dockerjava.jaxrs.ApacheUnixSocket.<init>(ApacheUnixSocket.java:51)
        at com.github.dockerjava.jaxrs.UnixConnectionSocketFactory.createSocket(UnixConnectionSocketFactory.java:65)
        at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:118)
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)

? https://forums.docker.com/t/worked-around-docker-http-s-api-no-longer-available-in-beta9/10474/6

https://github.com/docker-java/docker-java/issues/537

bitsofinfo commented 8 years ago

This definitely aids the issue

brew install socat
nohup socat -4 TCP-LISTEN:2375,fork UNIX-CONNECT:/var/run/docker.sock &

or /var/tmp/docker.sock (depends on your local setup)

cdancy commented 8 years ago

@bitsofinfo this is definitely a known issue and the link to the docker-forum, and subsequently the docker-devs themselves, suggest that a "tcp fix" may not be forthcoming at all. Not sure what to make of that. The fix, as you note, does work as expected! Being a mac user myself I greatly appreciate you sharing this!

However, I've stumbled upon a few issues related to docker-on-mac being on v1.24 of their API and docker-java not yet supporting this. Trying to set the apiVersion to 1.23, 1.22, 1.21 still was throwing exceptions. I've commented on an open issue they have already which is similar. Lets see what they have to say.

bitsofinfo commented 8 years ago

related #231 ?

cdancy commented 8 years ago

Somewhat related but that is more ties to using Unix sockets than going over tcp. If we can get tcp working OOTB that would be the preferred solution and one we could move forward with instead of waiting on all the work that needs to be done to get Unix sockets on Mac working as expected.

double16 commented 8 years ago

I had Docker for Mac 1.11 working using this recipe:

dependencies {
    dockerJava 'com.github.docker-java:docker-java:3.0.0'
    dockerJava 'com.kohlschutter.junixsocket:junixsocket-common:2.0.4' // for gradle-docker-plugin using Unix sockets
    dockerJava 'com.kohlschutter.junixsocket:junixsocket-native-common:2.0.4' // for gradle-docker-plugin using Unix sockets
    dockerJava 'org.slf4j:slf4j-simple:1.7.5'
    dockerJava 'cglib:cglib:3.2.0'
}

docker {
  url = "unix:///var/run/docker.sock"
}

Docker for Mac 1.12 broke with the server API compatibility. At which point I uninstalled and am using docker-machine exclusively.

tclift commented 8 years ago

Force upgrading Docker Java's unix-socket-factory dependency worked for me. Using Docker for Mac 1.12.0-a (stable) and docker-java-application 3.0.2.

configurations {
    dockerJava {
        resolutionStrategy {
            force 'de.gesellix:unix-socket-factory:2016-04-06T22-21-19'
        }
    }
}

docker {
  url = 'unix:///var/run/docker.sock'
}
childnode commented 8 years ago

@tclift great find ... but without digging into it: shure it's @gesellix's de.gesellix:unix-socket-factory (gesellix/unix-socket-factory) but not de.gesellix:docker-socket-factory (gesellix/docker-socket-factory)?

gesellix commented 8 years ago

The docker-java lib depends on unix-socket-factory, which is actually the one depending on the relevant com.kohlschutter.junixsocket:junixsocket* libs. Beware of Netty, though, as mentioned here: https://github.com/docker-java/docker-java/issues/537#issuecomment-206712757

priyadarsh commented 8 years ago

@tclift I tried your suggestion and it led me to another exception. I tried running as sudo as well.

org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':buildImage'.
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:69)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)
        at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35)
        at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64)
        at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
        at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:52)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
        at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:203)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:185)
        at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:66)
        at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:50)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:25)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:110)
        at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:37)
        at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:37)
        at org.gradle.execution.DefaultBuildExecuter.access$000(DefaultBuildExecuter.java:23)
        at org.gradle.execution.DefaultBuildExecuter$1.proceed(DefaultBuildExecuter.java:43)
        at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
        at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:37)
        at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:30)
        at org.gradle.initialization.DefaultGradleLauncher$4.run(DefaultGradleLauncher.java:154)
        at org.gradle.internal.Factories$1.create(Factories.java:22)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:52)
        at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:151)
        at org.gradle.initialization.DefaultGradleLauncher.access$200(DefaultGradleLauncher.java:32)
        at org.gradle.initialization.DefaultGradleLauncher$1.create(DefaultGradleLauncher.java:99)
        at org.gradle.initialization.DefaultGradleLauncher$1.create(DefaultGradleLauncher.java:93)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:62)
        at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:93)
        at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:82)
        at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:94)
        at org.gradle.tooling.internal.provider.ExecuteBuildActionRunner.run(ExecuteBuildActionRunner.java:28)
        at org.gradle.launcher.exec.ChainingBuildActionRunner.run(ChainingBuildActionRunner.java:35)
        at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:43)
        at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:28)
        at org.gradle.launcher.exec.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:75)
        at org.gradle.launcher.exec.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:45)
        at org.gradle.launcher.daemon.server.exec.ExecuteBuild.doBuild(ExecuteBuild.java:52)
        at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.WatchForDisconnection.execute(WatchForDisconnection.java:37)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.ResetDeprecationLogger.execute(ResetDeprecationLogger.java:26)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.RequestStopIfSingleUsedDaemon.execute(RequestStopIfSingleUsedDaemon.java:34)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:74)
        at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:72)
        at org.gradle.util.Swapper.swap(Swapper.java:38)
        at org.gradle.launcher.daemon.server.exec.ForwardClientInput.execute(ForwardClientInput.java:72)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.health.DaemonHealthTracker.execute(DaemonHealthTracker.java:47)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.LogToClient.doBuild(LogToClient.java:66)
        at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment.doBuild(EstablishBuildEnvironment.java:72)
        at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.health.HintGCAfterBuild.execute(HintGCAfterBuild.java:41)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy$1.run(StartBuildOrRespondWithBusy.java:50)
        at org.gradle.launcher.daemon.server.DaemonStateCoordinator$1.run(DaemonStateCoordinator.java:246)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54)
        at org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40)
Caused by: net.sf.cglib.proxy.UndeclaredThrowableException: java.lang.reflect.InvocationTargetException-->null
        at com.github.dockerjava.core.command.BuildImageResultCallback$$EnhancerByCGLIB$$3c8bede6.awaitImageId(<generated>)
        at com.github.dockerjava.core.command.BuildImageResultCallback$$EnhancerByCGLIB$$3c8bede6$awaitImageId.call(Unknown Source)
        at com.github.dockerjava.core.command.BuildImageResultCallback$$EnhancerByCGLIB$$20a8d4c6$awaitImageId.call(Unknown Source)
        at com.bmuschko.gradle.docker.tasks.image.DockerBuildImage.runRemoteCommand(DockerBuildImage.groovy:119)
        at com.bmuschko.gradle.docker.tasks.AbstractDockerRemoteApiTask$_start_closure1.doCall(AbstractDockerRemoteApiTask.groovy:48)
        at com.bmuschko.gradle.docker.utils.DockerThreadContextClassLoader.withClasspath(DockerThreadContextClassLoader.groovy:43)
        at com.bmuschko.gradle.docker.utils.ThreadContextClassLoader$withClasspath.call(Unknown Source)
        at com.bmuschko.gradle.docker.tasks.AbstractDockerRemoteApiTask.runInDockerClassPath(AbstractDockerRemoteApiTask.groovy:53)
        at com.bmuschko.gradle.docker.tasks.AbstractDockerRemoteApiTask$runInDockerClassPath.callCurrent(Unknown Source)
        at com.bmuschko.gradle.docker.tasks.AbstractDockerRemoteApiTask.start(AbstractDockerRemoteApiTask.groovy:47)
        at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:75)
        at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.doExecute(AnnotationProcessingTaskFactory.java:227)
        at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:220)
        at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:209)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:585)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:568)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61)
        ... 68 more
Caused by: java.lang.reflect.InvocationTargetException
        at java_lang_reflect_Method$invoke.call(Unknown Source)
        at java_lang_reflect_Method$invoke.call(Unknown Source)
        at com.bmuschko.gradle.docker.utils.DockerThreadContextClassLoader$_createPrintStreamProxyCallback_closure8.doCall(DockerThreadContextClassLoader.groovy:354)
        at com.sun.proxy.$Proxy373.invoke(Unknown Source)
        ... 86 more
Caused by: com.github.dockerjava.api.DockerClientException: Could not build image: unauthorized: authentication required
        at com.github.dockerjava.core.command.BuildImageResultCallback.getImageId(BuildImageResultCallback.java:71)
        at com.github.dockerjava.core.command.BuildImageResultCallback.awaitImageId(BuildImageResultCallback.java:48)
        ... 90 more
gesellix commented 8 years ago

the relevant bit is probably this one:

Caused by: com.github.dockerjava.api.DockerClientException: Could not build image: unauthorized: authentication required

You need to configure the registry credentials.

priyadarsh commented 8 years ago

Yes I did.

docker {
  url = 'unix:///var/run/docker.sock'
  registryCredentials {
    url = dockerRegistryUrl
    username = dockerRegistryUser
    password = dockerRegistryPass
    email = dockerRegistryEmail
  }
}

and also executed docker login before running build.

rafaroca commented 8 years ago

To me, docker-java-3.0.6 fixed this issue. I see that the master branch of gradle-docker-plugin already contains 3.0.6 Is there a new release possible/planned even though this would just contain the version bump of docker-java?

cdancy commented 8 years ago

@rafaroca some bits do indeed work but not all. The owner/maintainer of docker-java has been working on proper osx support but is not yet ready.

I can certainly release a new version if it will help with your workload.

rafaroca commented 8 years ago

@cdancy I just tested my use case again. Some parts are indeed not working. My gradle build starts but has issues. Appreciate your offer but I realize that a new version would not help here.

hsyed commented 7 years ago

Had to do this to get it working.

docker run -d -v /var/run/docker.sock:/var/run/docker.sock -p 2375:2375 bobrik/socat TCP4-LISTEN:2375,fork,reuseaddr UNIX-CONNECT:/var/run/docker.sock

Alxandr commented 7 years ago

With docker-machine I have the following working:

docker {
      url = System.getenv('DOCKER_HOST')
      certPath = new File(System.getenv('DOCKER_CERT_PATH'))
}

It seems to me that this should maybe be default behaviour?

cdancy commented 7 years ago

@Alxandr that is more for a general use-case and not really relevant to the topic at hand.

cdancy commented 7 years ago

Closing issue for now as I use docker on mac on a daily basis and cannot reproduce the OP's ISSUE (at least not with the latest plugin and latest docker-on-mac). Setting docker.url to unix:///var/run/docker.sock is working as expected.