OutOfMemoryError when copying logs from docker container

foxlegend commented 7 years ago

Issue Overview

Using the "log" directive into a "beforeStop" section results in an OutOfMemoryError exception.

beforeStop:
    - log:
        to: target/logs/jboss.log
        stderr: true

Expected Behaviour

Logs should be copied into the specified target directory.

Current Behaviour

An OutOfMemoryError is raised (I'm sorry, I didn't used the details balise as it doesn't take into account indents, etc.) :

(E) AfterSuite
    (I) TestContextHandler.createSuiteContext
    (O) ServerKillerExtension.throwErrpr
    (O) ContainerEventController.execute
    (E) StopSuiteContainer
        (O) ContainerLifecycleController.stopSuiteContainers
        (E) StopContainer
            (I) ContainerDeploymentContextHandler.createContainerContext
            (O) ContainerLifecycleController.stopContainer
            (E) BeforeStop
                (O) ArquillianServiceDeployer.undeploy
            (E) AfterStop
                (O) CubeContainerLifecycleController.stopCubeMappedContainer
                (E) StopCube
                    (O) CubeLifecycleController.stop
                    (E) BeforeStop
                        (O) BeforeStopContainerObserver.processCommands
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at org.arquillian.cube.impl.docker.DockerClientExecutor.readDockerRawStream(DockerClientExecutor.java:604)
    at org.arquillian.cube.impl.docker.DockerClientExecutor.copyLog(DockerClientExecutor.java:586)
    at org.arquillian.cube.impl.client.BeforeStopContainerObserver.processCommands(BeforeStopContainerObserver.java:37)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
    at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)
    at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:81)
    at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:145)
    at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:116)
    at org.jboss.arquillian.core.impl.EventImpl.fire(EventImpl.java:67)
    at org.arquillian.cube.impl.model.DockerCube.stop(DockerCube.java:98)
    at org.arquillian.cube.impl.client.CubeLifecycleController.stop(CubeLifecycleController.java:23)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
    at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)
    at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:81)
    at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:145)
    at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:116)
    at org.jboss.arquillian.core.impl.EventImpl.fire(EventImpl.java:67)
    at org.arquillian.cube.impl.client.container.CubeContainerLifecycleController.stopCubeMappedContainer(CubeContainerLifecycleController.java:77)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:94)
    at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:99)
                        (E) OutOfMemoryError
                    (E) OutOfMemoryError
                (E) OutOfMemoryError
            (E) OutOfMemoryError
            (E) OutOfMemoryError
        (E) OutOfMemoryError
    (E) OutOfMemoryError
    (E) OutOfMemoryError

Steps To Reproduce

When specifying dockerContainers, use the beforeStop directive to copy logs from containers like follows:
```
beforeStop:
- log:
    to: target/logs/jboss.log
    stderr: true
```
Launch tests with Arquillian Cube.

Additional Information

I tried to investigate the problem, and I found the problem occurs in the following lines of the readDockerRawStream of the DockerClientExecutor class: DockerClientExecutor.java.

int size = headerBuffer.getInt();

byte[] streamOutputBuffer = new byte[size];
rawSteram.read(streamOutputBuffer);
outputStream.write(streamOutputBuffer);

There are no control about the size thing extracted from the given headerBuffer: I think a better way would be to use a intermediate fixed length buffer to copy data from InputStream to OutputStream. Another way may be to use IOUtils.copy() from commons-io, but it is a transitive dependency from docker-java…

Are you interested by a PR?

Thank you :)

lordofthejars commented 7 years ago

First of all thank you very much for using Arquillian Cube and reporting such a bug. When I implemented this it was more like a feature not really sure that would be used or not. Now it seems it has so sense and needs to be fixed.

So Absolutely for a PR!!!!

foxlegend commented 7 years ago

It seems I made a mistake… I was working against an older Arquillian Cube version which docker-java version was not updated (to be honest, I was working with the Alpha5… It took me some time to notice it).

With the latest version of cube, logs are fetched with a callback class, which writes directly in the outpustream without buffering issues (introduced in newer versions of docker-java)… Then, the bug I submitted does not exist in current Cube release…

I'm so sorry, please accept my apologize about that…

bartoszmajsak commented 7 years ago

Hey @foxlegend no need for apologies. Thank you so much for effort investigating it and coming back with detailed feedback.

arquillian / arquillian-cube