arquillian / arquillian-cube

Control (docker, kubernetes, openshift) containers in your tests with ease!
http://arquillian.org/arquillian-cube/
120 stars 98 forks source link

Docker For Mac Beta Unix Socket Not Supported #372

Closed margic closed 6 years ago

margic commented 8 years ago

The newly released Docker for mac https://docs.docker.com/docker-for-mac replaces boot2docker supported by arquillian. The biggest difference is the native use of the hypervisor which uses a unix socket instead of tcp in the docker host.

At installation time, Docker for Mac provisions an xhyve VM based on Alpine Linux, running Docker Engine. It exposes the docker API on a socket in /var/tmp/docker.sock. Since this is the default location where docker will look if no environment variables are set, you can start using docker and docker-compose without setting any environment variables.

This is actually desirable however arquillian cube is configured to use boot2docker on mac by default using the default boot2docker url. If the serverUri is overridden see below:

    <extension qualifier="docker">
        <property name="serverVersion">1.12</property>
        <property name="serverUri">unix:///var/run/docker.sock</property>
        <property name="tlsVerify">false</property>
        <property name="clean">true</property>
        <property name="dockerContainers">
            transaction:
                buildImage:
                    dockerfileLocation: src/test/resources/docker
                    noCache: true
                    remove: true
                await:
                    strategy: polling
                portBindings: [8080/tcp]
        </property>
    </extension>

The docker-java client version used attempt to NativeUnixSocket causing the exception below.

java.lang.UnsatisfiedLinkError: Could not find library in classpath, tried: [libjunixsocket-macosx-1.8-x86_64.dylib, libjunixsocket-macosx-1.5-x86_64.dylib]
    at org.newsclub.net.unix.NativeUnixSocket.load(NativeUnixSocket.java:81)
    at org.newsclub.net.unix.NativeUnixSocket.<clinit>(NativeUnixSocket.java:112)
    at org.newsclub.net.unix.AFUNIXSocket.<init>(AFUNIXSocket.java:36)
    at org.newsclub.net.unix.AFUNIXSocket.newInstance(AFUNIXSocket.java:50)
    at com.github.dockerjava.jaxrs.ApacheUnixSocket.<init>(ApacheUnixSocket.java:51)
    at com.github.dockerjava.jaxrs.UnixConnectionSocketFactory.createSocket(UnixConnectionSocketFactory.java:65)
    at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:118)
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
    at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:71)
    at com.github.dockerjava.jaxrs.connector.ApacheConnector.apply(ApacheConnector.java:437)
    at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:246)
    at org.glassfish.jersey.client.JerseyInvocation$2.call(JerseyInvocation.java:683)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:228)
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:424)
    at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:679)
    at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:435)
    at org.glassfish.jersey.client.JerseyInvocation$Builder.post(JerseyInvocation.java:338)
    at com.github.dockerjava.jaxrs.async.POSTCallbackNotifier.response(POSTCallbackNotifier.java:29)
    at com.github.dockerjava.jaxrs.async.AbstractCallbackNotifier.call(AbstractCallbackNotifier.java:50)
    at com.github.dockerjava.jaxrs.async.AbstractCallbackNotifier.call(AbstractCallbackNotifier.java:24)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

This exception is thrown by dependency of docker-java de.gesellix:unix-socket-factory:2015-01-27T15-02-14. This dependency wraps a version of kohlschutter/junixsocket that uses native libraries on unix and mac osx. This version of junixsocket uses a class loader to load the library from the jar classloader and to get around this the unix-socket-factory jar is build with the libs for unix packaged in the jar. However the libs for mac osx are not supplied and attempts to provide them do not work do to the classloader.

A newer version of gesellix uses the newer kohlschutter/junixsocket that changes the way the class loading operates. If overriding the import in my pom as follows

        <dependency>
            <groupId>org.arquillian.cube</groupId>
            <artifactId>arquillian-cube-docker</artifactId>
            <version>${arquillian.cube.version}</version>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <artifactId>unix-socket-factory</artifactId>
                    <groupId>de.gesellix</groupId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>de.gesellix</groupId>
            <artifactId>unix-socket-factory</artifactId>
            <version>2016-04-06T22-21-19</version>
        </dependency>

With the 2016 version it does connect to the docker for mac using the unix socket and creates the container. However it fails to start with the following exception.

org.arquillian.cube.spi.CubeControlException: Could not start transaction

    at org.arquillian.cube.spi.CubeControlException.failedStart(CubeControlException.java:23)
    at org.arquillian.cube.docker.impl.model.DockerCube.start(DockerCube.java:124)
    at org.arquillian.cube.impl.client.CubeLifecycleController.start(CubeLifecycleController.java:19)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
    at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)
    at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:81)
    at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:145)
    at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:116)
    at org.jboss.arquillian.core.impl.EventImpl.fire(EventImpl.java:67)
    at org.arquillian.cube.impl.containerless.ContainerlessDockerDeployableContainer.deploy(ContainerlessDockerDeployableContainer.java:92)
    at org.jboss.arquillian.container.impl.client.container.ContainerDeployController$3.call(ContainerDeployController.java:161)
    at org.jboss.arquillian.container.impl.client.container.ContainerDeployController$3.call(ContainerDeployController.java:128)
    at org.jboss.arquillian.container.impl.client.container.ContainerDeployController.executeOperation(ContainerDeployController.java:271)
    at org.jboss.arquillian.container.impl.client.container.ContainerDeployController.deploy(ContainerDeployController.java:127)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
    at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)
    at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:81)
    at org.jboss.arquillian.container.impl.client.container.DeploymentExceptionHandler.verifyExpectedExceptionDuringDeploy(DeploymentExceptionHandler.java:50)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
    at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)
    at org.jboss.arquillian.container.impl.client.ContainerDeploymentContextHandler.createDeploymentContext(ContainerDeploymentContextHandler.java:78)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
    at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)
    at org.jboss.arquillian.container.impl.client.ContainerDeploymentContextHandler.createContainerContext(ContainerDeploymentContextHandler.java:57)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
    at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)
    at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:145)
    at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:116)
    at org.jboss.arquillian.core.impl.EventImpl.fire(EventImpl.java:67)
    at org.jboss.arquillian.container.impl.client.container.ContainerDeployController$1.perform(ContainerDeployController.java:95)
    at org.jboss.arquillian.container.impl.client.container.ContainerDeployController$1.perform(ContainerDeployController.java:80)
    at org.jboss.arquillian.container.impl.client.container.ContainerDeployController.forEachDeployment(ContainerDeployController.java:263)
    at org.jboss.arquillian.container.impl.client.container.ContainerDeployController.forEachManagedDeployment(ContainerDeployController.java:239)
    at org.jboss.arquillian.container.impl.client.container.ContainerDeployController.deployManaged(ContainerDeployController.java:79)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
    at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)
    at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:81)
    at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:145)
    at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:116)
    at org.jboss.arquillian.core.impl.EventImpl.fire(EventImpl.java:67)
    at org.jboss.arquillian.container.test.impl.client.ContainerEventController.execute(ContainerEventController.java:101)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
    at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)
    at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:81)
    at org.jboss.arquillian.test.impl.TestContextHandler.createClassContext(TestContextHandler.java:92)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
    at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)
    at org.jboss.arquillian.test.impl.TestContextHandler.createSuiteContext(TestContextHandler.java:73)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
    at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:88)
    at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:145)
    at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:116)
    at org.jboss.arquillian.test.impl.EventTestRunnerAdaptor.beforeClass(EventTestRunnerAdaptor.java:87)
    at org.jboss.arquillian.junit.Arquillian$2.evaluate(Arquillian.java:201)
    at org.jboss.arquillian.junit.Arquillian.multiExecute(Arquillian.java:426)
    at org.jboss.arquillian.junit.Arquillian.access$200(Arquillian.java:54)
    at org.jboss.arquillian.junit.Arquillian$3.evaluate(Arquillian.java:218)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.jboss.arquillian.junit.Arquillian.run(Arquillian.java:166)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:119)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: com.github.dockerjava.api.exception.InternalServerErrorException: EOF

    at com.github.dockerjava.jaxrs.filter.ResponseStatusExceptionFilter.filter(ResponseStatusExceptionFilter.java:53)
    at org.glassfish.jersey.client.ClientFilteringStages$ResponseFilterStage.apply(ClientFilteringStages.java:134)
    at org.glassfish.jersey.client.ClientFilteringStages$ResponseFilterStage.apply(ClientFilteringStages.java:123)
    at org.glassfish.jersey.process.internal.Stages.process(Stages.java:171)
    at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:251)
    at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:667)
    at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:664)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:228)
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:424)
    at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:664)
    at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:424)
    at org.glassfish.jersey.client.JerseyInvocation$Builder.post(JerseyInvocation.java:333)
    at com.github.dockerjava.jaxrs.StartContainerCmdExec.execute(StartContainerCmdExec.java:29)
    at com.github.dockerjava.jaxrs.StartContainerCmdExec.execute(StartContainerCmdExec.java:14)
    at com.github.dockerjava.jaxrs.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:23)
    at com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:35)
    at com.github.dockerjava.core.command.StartContainerCmdImpl.exec(StartContainerCmdImpl.java:46)
    at org.arquillian.cube.docker.impl.docker.DockerClientExecutor.startContainer(DockerClientExecutor.java:450)
    at org.arquillian.cube.docker.impl.model.DockerCube.start(DockerCube.java:115)
    ... 103 more
lordofthejars commented 8 years ago

For what you mention, it seems that the problem could be in docker-java library. The first thing that should be done is create a simple example with only docker-java and see if it can connect and create a docker instance using native docker in Mac, just to discard where is the problem, since in theory and from the point of view of Cube, we are only calling docker-java methods and it works with native Linux.

But thank you so much for initial investigation.

lordofthejars commented 8 years ago

https://github.com/docker-java/docker-java/issues/537

margic commented 8 years ago

I saw that ticket. I started trying to update the docker-java client to use the newer dependency. There will be downstream changes necessary in arquillian cube for both docker-java api changes and a need to support the new default mac option in addition to boot2docker currently the mac default.

There are changes to the test suite for the docker-java that are unrelated (they are due to changes in docker 1.12) that I need to fix before I can submit the docker-java pull request. Once that is done I'll look again at this. There is definitely some need for change on arquillian cube. Should we keep this issue open?

lordofthejars commented 8 years ago

yes let's leave it opened until we know how to manage this.

Thanks for your research and of course any PR when everything is fixed is welcomed as well.

sfcoy commented 6 years ago

I think you can close this now. Arquillian Cube (1.7.0 and 1.9.0) works great with Docker for Mac now.

<extension qualifier="docker">
    <property name="serverVersion">1.32</property>
    <property name="serverUri">unix:///var/run/docker.sock</property>
    <property name="definitionFormat">CUBE</property>
    <property name="tlsVerify">false</property>
    <property name="dockerContainers">
        wildfly-remote:
            image: local/wildfly-it:latest
            exposedPorts: [9990/tcp, 8080/tcp]
            links: [peopledb]
            await:
                strategy: polling
            portBindings: [9990/tcp, 8080/tcp]
    </property>
</extension>