search

testcontainers / testcontainers-java

Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.
https://testcontainers.org
MIT License
7.89k stars 1.62k forks source link

[Bug]: Out-of-the-box ClickHouseContainer throws SQLException on start with Log4j #8761

Closed DFMichelangelo closed 2 weeks ago

DFMichelangelo commented 4 weeks ago

Module

Clickhouse

Testcontainers version

1.19.8

Using the latest Testcontainers version?

Yes

Host OS

Windows 10 with WSL

Host Arch

amd64

Docker version

Client:
 Cloud integration: v1.0.31
 Version:           20.10.23
 API version:       1.41
 Go version:        go1.18.10
 Git commit:        7155243
 Built:             Thu Jan 19 17:43:10 2023
 OS/Arch:           windows/amd64
 Context:           default
 Experimental:      true

Server: Docker Desktop 4.17.0 (99724)
 Engine:
  Version:          20.10.23
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.10
  Git commit:       6051f14
  Built:            Thu Jan 19 17:32:04 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.18
  GitCommit:        2456e983eb9e37e47538f59ea18f2043c9a73640
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

What happened?

Hi all, I've tried to start the most simple Clickhouse test container using the basic setup with clickhouse version 24.5 (the most recent one). Somehow a SQLException is thrown once the container starts claiming that LZ4 is not supported.

I would expect to have the container working "out of the box" when the most basic setup configuration is used.

Relevant log output

12:39:23: Executing ':Main.main()'...

> Task :generateEffectiveLombokConfig UP-TO-DATE
> Task :compileJava UP-TO-DATE
> Task :processResources
> Task :classes
Connected to the target VM, address: 'localhost:58724', transport: 'socket'

> Task :Main.main()
2024-06-09T11:39:24.601975700Z main WARN No Root logger was configured, creating default ERROR-level Root logger with Console appender
12:39:24.850 [main] INFO  org.testcontainers.images.PullPolicy - Image pull policy will be performed by: DefaultPullPolicy()
12:39:24.857 [main] INFO  org.testcontainers.utility.ImageNameSubstitutor - Image name substitution will be performed by: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor')
12:39:25.248 [main] INFO  org.testcontainers.dockerclient.DockerClientProviderStrategy - Loaded org.testcontainers.dockerclient.NpipeSocketClientProviderStrategy from ~/.testcontainers.properties, will try it first
12:39:25.572 [main] INFO  org.testcontainers.dockerclient.DockerClientProviderStrategy - Found Docker environment with local Npipe socket (npipe:////./pipe/docker_engine)
12:39:25.574 [main] INFO  org.testcontainers.DockerClientFactory - Docker host IP address is localhost
12:39:25.598 [main] INFO  org.testcontainers.DockerClientFactory - Connected to docker: 
  Server Version: 20.10.23
  API Version: 1.41
  Operating System: Docker Desktop
  Total Memory: 7828 MB
12:39:25.608 [main] INFO  org.testcontainers.DockerClientFactory - Checking the system...
12:39:25.609 [main] INFO  org.testcontainers.DockerClientFactory - ?? Docker server version should be at least 1.6.0
12:39:25.633 [main] INFO  tc.clickhouse/clickhouse-server:24.5 - Creating container for image: clickhouse/clickhouse-server:24.5
12:39:26.238 [main] INFO  org.testcontainers.utility.RegistryAuthLocator - Credential helper/store (docker-credential-desktop) does not have credentials for https://index.docker.io/v1/
12:39:26.252 [main] INFO  tc.testcontainers/ryuk:0.7.0 - Creating container for image: testcontainers/ryuk:0.7.0
12:39:26.408 [main] INFO  tc.testcontainers/ryuk:0.7.0 - Container testcontainers/ryuk:0.7.0 is starting: 3747682bf38ef172fdb24aa6516af20ea05218a3f94967b2f167ec86e66f4c11
12:39:26.949 [main] INFO  tc.testcontainers/ryuk:0.7.0 - Container testcontainers/ryuk:0.7.0 started in PT0.6970983S
12:39:27.027 [main] INFO  tc.clickhouse/clickhouse-server:24.5 - Container clickhouse/clickhouse-server:24.5 is starting: cb79b1a8f2b02760c9f6d1549114764a3b4989a147b40534efb6e39c7449f8e8
12:39:27.486 [main] INFO  tc.clickhouse/clickhouse-server:24.5 - Waiting for database connection to become available at jdbc:clickhouse://localhost:8123/default using query 'SELECT 1'
12:41:27.584 [main] ERROR tc.clickhouse/clickhouse-server:24.5 - Could not start container
java.lang.IllegalStateException: Container is started, but cannot be accessed by (JDBC URL: jdbc:clickhouse://localhost:8123/default), please check container logs
    at org.testcontainers.containers.JdbcDatabaseContainer.waitUntilContainerStarted(JdbcDatabaseContainer.java:177) ~[jdbc-1.19.8.jar:?]
    at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:500) ~[testcontainers-1.19.8.jar:?]
    at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:354) ~[testcontainers-1.19.8.jar:?]
    at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81) [duct-tape-1.0.8.jar:?]
    at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:344) [testcontainers-1.19.8.jar:?]
    at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:330) [testcontainers-1.19.8.jar:?]
    at org.example.Main.main(Main.java:21) [main/:?]
Caused by: java.sql.SQLException: LZ4 is not supported. Please disable compression(compress=0), modify the algorithm(e.g. compress_algorithm=gzip), or add the missing libraries to the classpath., server ClickHouseNode [uri=http://localhost:8123/default]@514414092
    at com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:85) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.jdbc.SqlExceptionUtils.create(SqlExceptionUtils.java:31) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:90) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.getLastResponse(ClickHouseStatementImpl.java:122) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:474) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.execute(ClickHouseStatementImpl.java:462) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at org.testcontainers.containers.JdbcDatabaseContainer.waitUntilContainerStarted(JdbcDatabaseContainer.java:160) ~[jdbc-1.19.8.jar:?]
    ... 6 more
Caused by: java.lang.UnsupportedOperationException: LZ4 is not supported. Please disable compression(compress=0), modify the algorithm(e.g. compress_algorithm=gzip), or add the missing libraries to the classpath.
    at com.clickhouse.data.ClickHouseCompressionAlgorithm.createInputStream(ClickHouseCompressionAlgorithm.java:61) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.data.ClickHouseInputStream.wrap(ClickHouseInputStream.java:98) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.data.ClickHouseInputStream.of(ClickHouseInputStream.java:294) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.client.ClickHouseClient.getResponseInputStream(ClickHouseClient.java:167) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.client.ClickHouseClient.getAsyncResponseInputStream(ClickHouseClient.java:190) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.client.http.HttpUrlConnectionImpl.buildResponse(HttpUrlConnectionImpl.java:100) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.client.http.HttpUrlConnectionImpl.post(HttpUrlConnectionImpl.java:237) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.client.http.ClickHouseHttpClient.send(ClickHouseHttpClient.java:124) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.client.AbstractClient.execute(AbstractClient.java:280) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.client.ClickHouseClientBuilder$Agent.sendOnce(ClickHouseClientBuilder.java:282) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.client.ClickHouseClientBuilder$Agent.send(ClickHouseClientBuilder.java:294) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.client.ClickHouseClientBuilder$Agent.execute(ClickHouseClientBuilder.java:349) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.client.ClickHouseClient.executeAndWait(ClickHouseClient.java:878) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.client.ClickHouseRequest.executeAndWait(ClickHouseRequest.java:2154) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.getLastResponse(ClickHouseStatementImpl.java:120) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:474) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.execute(ClickHouseStatementImpl.java:462) ~[clickhouse-jdbc-0.6.0.jar:clickhouse-jdbc 0.6.0 (revision: cb8e198)]
    at org.testcontainers.containers.JdbcDatabaseContainer.waitUntilContainerStarted(JdbcDatabaseContainer.java:160) ~[jdbc-1.19.8.jar:?]
    ... 6 more
12:41:27.655 [main] ERROR tc.clickhouse/clickhouse-server:24.5 - Log output from the failed container:
Processing configuration file '/etc/clickhouse-server/config.xml'.
Merging configuration file '/etc/clickhouse-server/config.d/docker_related_config.xml'.
Logging trace to /var/log/clickhouse-server/clickhouse-server.log
Logging errors to /var/log/clickhouse-server/clickhouse-server.err.log

/entrypoint.sh: create database 'default'
Processing configuration file '/etc/clickhouse-server/config.xml'.
Merging configuration file '/etc/clickhouse-server/config.d/docker_related_config.xml'.
Logging trace to /var/log/clickhouse-server/clickhouse-server.log
Logging errors to /var/log/clickhouse-server/clickhouse-server.err.log

Disconnected from the target VM, address: 'localhost:58724', transport: 'socket'
Exception in thread "main" org.testcontainers.containers.ContainerLaunchException: Container startup failed for image clickhouse/clickhouse-server:24.5
    at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:359)
    at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:330)
    at org.example.Main.main(Main.java:21)
Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
    at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:88)
    at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:344)
    ... 2 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
    at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:563)
    at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:354)
    at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
    ... 3 more
Caused by: java.lang.IllegalStateException: Container is started, but cannot be accessed by (JDBC URL: jdbc:clickhouse://localhost:8123/default), please check container logs
    at org.testcontainers.containers.JdbcDatabaseContainer.waitUntilContainerStarted(JdbcDatabaseContainer.java:177)
    at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:500)
    ... 5 more
Caused by: java.sql.SQLException: LZ4 is not supported. Please disable compression(compress=0), modify the algorithm(e.g. compress_algorithm=gzip), or add the missing libraries to the classpath., server ClickHouseNode [uri=http://localhost:8123/default]@514414092
    at com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:85)
    at com.clickhouse.jdbc.SqlExceptionUtils.create(SqlExceptionUtils.java:31)
    at com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:90)
    at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.getLastResponse(ClickHouseStatementImpl.java:122)
    at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:474)
    at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.execute(ClickHouseStatementImpl.java:462)
    at org.testcontainers.containers.JdbcDatabaseContainer.waitUntilContainerStarted(JdbcDatabaseContainer.java:160)
    ... 6 more
Caused by: java.lang.UnsupportedOperationException: LZ4 is not supported. Please disable compression(compress=0), modify the algorithm(e.g. compress_algorithm=gzip), or add the missing libraries to the classpath.
    at com.clickhouse.data.ClickHouseCompressionAlgorithm.createInputStream(ClickHouseCompressionAlgorithm.java:61)
    at com.clickhouse.data.ClickHouseInputStream.wrap(ClickHouseInputStream.java:98)
    at com.clickhouse.data.ClickHouseInputStream.of(ClickHouseInputStream.java:294)
    at com.clickhouse.client.ClickHouseClient.getResponseInputStream(ClickHouseClient.java:167)
    at com.clickhouse.client.ClickHouseClient.getAsyncResponseInputStream(ClickHouseClient.java:190)
    at com.clickhouse.client.http.HttpUrlConnectionImpl.buildResponse(HttpUrlConnectionImpl.java:100)
    at com.clickhouse.client.http.HttpUrlConnectionImpl.post(HttpUrlConnectionImpl.java:237)
    at com.clickhouse.client.http.ClickHouseHttpClient.send(ClickHouseHttpClient.java:124)
    at com.clickhouse.client.AbstractClient.execute(AbstractClient.java:280)
    at com.clickhouse.client.ClickHouseClientBuilder$Agent.sendOnce(ClickHouseClientBuilder.java:282)
    at com.clickhouse.client.ClickHouseClientBuilder$Agent.send(ClickHouseClientBuilder.java:294)
    at com.clickhouse.client.ClickHouseClientBuilder$Agent.execute(ClickHouseClientBuilder.java:349)
    at com.clickhouse.client.ClickHouseClient.executeAndWait(ClickHouseClient.java:878)
    at com.clickhouse.client.ClickHouseRequest.executeAndWait(ClickHouseRequest.java:2154)
    at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.getLastResponse(ClickHouseStatementImpl.java:120)
    ... 9 more

> Task :Main.main() FAILED

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.5/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
4 actionable tasks: 2 executed, 2 up-to-date

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':Main.main()'.
> Process 'command 'C:\Users\dfmic\.jdks\jdk-21.0.2\bin\java.exe'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

BUILD FAILED in 2m 5s
12:41:28: Execution finished ':Main.main()'.

Additional Information

you can reproduce the error by looking at the repository TestingContainers

The issue is not related to LZ4 compression of clickhouse container but to the use of the log4j logger combined with clickhouse

surovenko commented 2 weeks ago

Try to add dependency:

<dependency>
    <groupId>org.lz4</groupId>
    <artifactId>lz4-java</artifactId>
    <version>${lz4.version}</version>
</dependency>
DFMichelangelo commented 2 weeks ago

Try to add dependency:

<dependency>
    <groupId>org.lz4</groupId>
    <artifactId>lz4-java</artifactId>
    <version>${lz4.version}</version>
</dependency>

Thank you @surovenko, it's actually what I've done recently, but I deem that the testcontainrer should include lz4-java as a dependency.

I also needed to add implementation 'org.apache.httpcomponents.client5:httpclient5:5.3.1' to make it work.