ElasticSearch with custom plugins

[GotoFinal]

How can I start elasticsearch container with some plugins (from zip files) already installed in it? Or I need to manually install them using execInContainer and restart elastic using execInContainer too?

[rnorth]

@GotoFinal I can think of a couple of options:

• probably the neatest would be to create and use your own docker image (e.g. new ElasticsearchContainer("my-image-name:tag")). If you're running your own dockerised Elasticsearch then perhaps you already have such an image available?

• alternatively, if the plugins can be loaded at startup from the file system, volume-mount the necessary files into the container at startup (using .withFileSystemBind(...)

There might be other options - perhaps worth asking in Elasticsearch forums as well?

Please let us know if you find something that works well, as other people might be interested too.

[GotoFinal]

I would need to create new image each time to run a test, as each time this is new .jar of plugin, as I'm testing the plugin itself.
And that is basically what I'm doing right now (but dynamically in code based on original one), but I don't understand why it isn't build in into elasticsearch module of test containers. (As I'm not using it here at all)

public class ElasticsearchWithPluginContainer extends GenericContainer<ElasticsearchWithPluginContainer> {
    private static final int ELASTICSEARCH_DEFAULT_PORT = 9200;
    private static final int ELASTICSEARCH_DEFAULT_TCP_PORT = 9300;
    private String dockerImage;
    private File plugin;
    public ElasticsearchWithPluginContainer(String dockerImageName) {
        this.dockerImage = dockerImageName;
    }

    public ElasticsearchWithPluginContainer withPlugin(File pluginZipFile) {
        plugin = pluginZipFile;
        return this.withFileSystemBind(pluginZipFile.getPath(), "/tmp/plugins/" + pluginZipFile.getName());
    }

    private ImageFromDockerfile prepareImage(String imageName) {
        String pluginContainerPath = plugin == null ? null : ("/tmp/plugins/" + plugin.getName());
        ImageFromDockerfile image = new ImageFromDockerfile()
            .withDockerfileFromBuilder(builder -> {
                builder.from(imageName);
                if (pluginContainerPath != null) {
                    builder.copy(pluginContainerPath, pluginContainerPath);
                    builder.run("bin/elasticsearch-plugin", "install", "file://" + pluginContainerPath);
                }
            });
        if (pluginContainerPath != null) {
            image.withFileFromFile(pluginContainerPath, plugin);
        }
        return image;
    }

    @Override
    protected void configure() {
        withNetworkAliases("elasticsearch-" + Base58.randomString(6));
        withEnv("discovery.type", "single-node");
        addExposedPorts(ELASTICSEARCH_DEFAULT_PORT, ELASTICSEARCH_DEFAULT_TCP_PORT);
        setWaitStrategy(new HttpWaitStrategy()
            .forPort(ELASTICSEARCH_DEFAULT_PORT)
            .forStatusCodeMatching(response -> response == HTTP_OK || response == HTTP_UNAUTHORIZED)
            .withStartupTimeout(Duration.ofMinutes(2)));
        setImage(prepareImage(dockerImage));
    }
    public String getHttpHostAddress() {
        return getContainerIpAddress() + ":" + getMappedPort(ELASTICSEARCH_DEFAULT_PORT);
    }
}

And that is based on code from this repository: https://github.com/testcontainers/testcontainers-java/blob/master/modules/elasticsearch/src/main/java/org/testcontainers/elasticsearch/ElasticsearchContainer.java

And from another one that was "migrated" here: https://github.com/dadoonet/testcontainers-java-module-elasticsearch/blob/master/src/main/java/fr/pilato/elasticsearch/containers/ElasticsearchContainer.java

That does support plugins, so I don't understand why that was skipped here.

And to install a plugin you need to get the file and execute command and restart elasticsearch if it is already started, so it's hard to do it after container is ready.

[bsideup]

I would even recommend using withCopy* API instead of the file system bind.

And, instead of building an image, you can simply run the install command before starting the Elasticsearch process (see withCommand, don't forget to call the original one too)

[GotoFinal]

@bsideup

I would even recommend using withCopy* API instead of the file system bind.

If I try to use something like:

.withCopyFileToContainer(MountableFile.forHostPath(localPath, containerPath)

where localPath is ~build/distributions/plugin.zip and containerPath is /tmp/plugins/plugin.zip I get java.io.IOException: com.sun.jna.LastErrorException: [32] Broken pipe and I can't really find out what is causing this.

Whole error:

org.testcontainers.containers.ContainerLaunchException: Container startup failed
    at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:290)
    at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:272)
    at pl.allegro.tech.elasticsearch.plugin.analysis.morfologik.MorfologikPluginIntegrationTest.setupContainerWithPlugin(MorfologikPluginIntegrationTest.groovy:32)
    at org.spockframework.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:200)
    at org.spockframework.runtime.model.MethodInfo.invoke(MethodInfo.java:113)
    at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
    at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
    at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
    at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
    at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
    at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
    at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:118)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
    at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:175)
    at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:157)
    at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404)
    at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
    at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
    at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:88)
    at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:283)
    ... 25 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
    at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:350)
    at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:285)
    at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
    ... 26 more
Caused by: java.io.IOException: com.sun.jna.LastErrorException: [32] Broken pipe
    at org.testcontainers.shaded.org.scalasbt.ipcsocket.UnixDomainSocket$UnixDomainSocketOutputStream.doWrite(UnixDomainSocket.java:187)
    at org.testcontainers.shaded.org.scalasbt.ipcsocket.UnixDomainSocket$UnixDomainSocketOutputStream.write(UnixDomainSocket.java:171)
    at org.testcontainers.dockerclient.transport.okhttp.UnixSocketFactory$1$2.write(UnixSocketFactory.java:49)
    at org.testcontainers.shaded.okio.Okio$1.write(Okio.java:79)
    at org.testcontainers.shaded.okio.AsyncTimeout$1.write(AsyncTimeout.java:180)
    at org.testcontainers.shaded.okio.RealBufferedSink.emitCompleteSegments(RealBufferedSink.java:179)
    at org.testcontainers.shaded.okio.RealBufferedSink.write(RealBufferedSink.java:42)
    at org.testcontainers.shaded.okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSink.write(Http1ExchangeCodec.java:361)
    at org.testcontainers.shaded.okio.ForwardingSink.write(ForwardingSink.java:35)
    at org.testcontainers.shaded.okhttp3.internal.connection.Exchange$RequestBodySink.write(Exchange.java:231)
    at org.testcontainers.shaded.okio.RealBufferedSink.emitCompleteSegments(RealBufferedSink.java:179)
    at org.testcontainers.shaded.okio.RealBufferedSink.writeAll(RealBufferedSink.java:107)
    at org.testcontainers.dockerclient.transport.okhttp.OkHttpInvocationBuilder$2.writeTo(OkHttpInvocationBuilder.java:247)
    at org.testcontainers.shaded.okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:69)
    at org.testcontainers.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at org.testcontainers.shaded.okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:43)
    at org.testcontainers.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at org.testcontainers.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at org.testcontainers.shaded.okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94)
    at org.testcontainers.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at org.testcontainers.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at org.testcontainers.shaded.okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
    at org.testcontainers.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at org.testcontainers.shaded.okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88)
    at org.testcontainers.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at org.testcontainers.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at org.testcontainers.shaded.okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:221)
    at org.testcontainers.shaded.okhttp3.RealCall.execute(RealCall.java:81)
    at org.testcontainers.dockerclient.transport.okhttp.OkHttpInvocationBuilder.execute(OkHttpInvocationBuilder.java:259)
    at org.testcontainers.dockerclient.transport.okhttp.OkHttpInvocationBuilder.execute(OkHttpInvocationBuilder.java:254)
    at org.testcontainers.dockerclient.transport.okhttp.OkHttpInvocationBuilder.put(OkHttpInvocationBuilder.java:230)
    at com.github.dockerjava.core.exec.CopyArchiveToContainerCmdExec.execute(CopyArchiveToContainerCmdExec.java:32)
    at com.github.dockerjava.core.exec.CopyArchiveToContainerCmdExec.execute(CopyArchiveToContainerCmdExec.java:13)
    at com.github.dockerjava.core.exec.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:21)
    at com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:35)
    at com.github.dockerjava.core.command.CopyArchiveToContainerCmdImpl.exec(CopyArchiveToContainerCmdImpl.java:154)
    at org.testcontainers.containers.GenericContainer.copyFileToContainer(GenericContainer.java:1167)
    at org.testcontainers.containers.GenericContainer.copyFileToContainer(GenericContainer.java:1140)
    at java.base/java.util.HashMap.forEach(HashMap.java:1336)
    at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:307)
    ... 28 more
Caused by: com.sun.jna.LastErrorException: [32] Broken pipe
    at org.testcontainers.shaded.org.scalasbt.ipcsocket.UnixDomainSocket$UnixDomainSocketOutputStream.doWrite(UnixDomainSocket.java:180)
    ... 67 more

[bsideup]

@GotoFinal

O_O are you on Mac or Linux?

[GotoFinal]

@bsideup Mac

[rnorth]

Yikes - please could you share the following when you get a chance:

• Testcontainers version
• Docker version
• JDK distribution and version
• MacOS version

There have been various JNA/native lib issues affecting some versions - admittedly that’s the first time I’ve seen this exact message, but it might help spot a pattern or reproduce.

_{Sent with GitHawk}

[GotoFinal]

Testcontaines: tested 1.12.1 and 1.12.2

Docker:
Server Version: 19.03.2 API Version: 1.40 Operating System: Docker Desktop (2.1.0.3 (38240))

JDK: openjdk version "11.0.4" 2019-07-16 OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.4+11) OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.4+11, mixed mode)

MacOS: 10.14.6