search

ClickHouse / clickhouse-java

ClickHouse Java Clients & JDBC Driver
https://clickhouse.com
Apache License 2.0
1.45k stars 534 forks source link

V2 client query fails on initial attempt #1840

Open maxsumrall opened 1 month ago

maxsumrall commented 1 month ago

Describe the bug

It appears that the first queries (SELECTs) to the ClickHouse server fail on the V2 client with an error: com.clickhouse.client.api.ClientException: Failed to get query response. Wrapping the request in a retry loop seems to resolve after the first failed attempt.

Steps to reproduce

  1. Start up application and connect to ClickHouse Cloud. This happens on 24.5, but has not been observed with 24.6 It has not been observed locally.
  2. Execute query:
client.queryAll(QUERY).stream()
        .map(record -> ...record mapping...)
        .collect(toImmutableList());

3.

Expected behaviour

Query succeeds.

Code example

See above.

Error log

com.clickhouse.client.api.ClientException: Failed to get query response
    at com.clickhouse.client.api.Client.queryAll(Client.java:1483)
    ...redacted...
Caused by: com.clickhouse.client.api.ClientException: Failed to get query response
    at com.clickhouse.client.api.Client.lambda$query$10(Client.java:1409)
    at com.clickhouse.client.api.Client.runAsyncOperation(Client.java:1624)
    at com.clickhouse.client.api.Client.query(Client.java:1414)
    at com.clickhouse.client.api.Client.query(Client.java:1284)
    at com.clickhouse.client.api.Client.queryAll(Client.java:1467)
    ... 21 common frames omitted
Caused by: java.util.concurrent.CompletionException: com.clickhouse.client.ClickHouseException: Connection pool shut down
    at com.clickhouse.client.ClickHouseClientBuilder$Agent.retry(ClickHouseClientBuilder.java:240)
    at com.clickhouse.client.ClickHouseClientBuilder$Agent.handle(ClickHouseClientBuilder.java:269)
    at com.clickhouse.client.ClickHouseClientBuilder$Agent.send(ClickHouseClientBuilder.java:296)
    at com.clickhouse.client.ClickHouseClientBuilder$Agent.execute(ClickHouseClientBuilder.java:349)
    at com.clickhouse.client.ClickHouseRequest.execute(ClickHouseRequest.java:2144)
    at com.clickhouse.client.api.Client.lambda$query$10(Client.java:1400)
    ... 25 common frames omitted
Caused by: com.clickhouse.client.ClickHouseException: Connection pool shut down
    at com.clickhouse.client.ClickHouseException.of(ClickHouseException.java:151)
    at com.clickhouse.client.ClickHouseClientBuilder$Agent.handle(ClickHouseClientBuilder.java:251)
    ... 29 common frames omitted
Caused by: java.lang.IllegalStateException: Connection pool shut down
    at org.apache.hc.core5.util.Asserts.check(Asserts.java:38)
    at org.apache.hc.core5.pool.LaxConnPool.lease(LaxConnPool.java:163)
    at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.lease(PoolingHttpClientConnectionManager.java:297)
    at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.acquireEndpoint(InternalExecRuntime.java:103)
    at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:125)
    at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
    at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
    at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
    at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
    at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
    at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
    at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
    at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
    at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:123)
    at com.clickhouse.client.http.ApacheHttpConnectionImpl.post(ApacheHttpConnectionImpl.java:280)
    at com.clickhouse.client.http.ClickHouseHttpClient.send(ClickHouseHttpClient.java:195)
    at com.clickhouse.client.AbstractClient.execute(AbstractClient.java:280)
    at com.clickhouse.client.ClickHouseClientBuilder$Agent.sendOnce(ClickHouseClientBuilder.java:282)
    at com.clickhouse.client.ClickHouseClientBuilder$Agent.send(ClickHouseClientBuilder.java:294)
    ... 28 common frames omitted

Configuration

Environment

ClickHouse server

chernser commented 1 month ago

Good day, @maxsumrall ! Thank you for reporting the issue!

Do you close client at any case? The error indicates that underlying client was closed.

Would you please try configuring client with .useNewImplementation(true) - it will switch client to use new HTTP transport layer that we recently tested and it should not have such issue.

Is the remote server Cloud instance? Is there a proxy between client and the server?

Thanks!

maxsumrall commented 1 month ago

Would you please try configuring client with .useNewImplementation(true) - it will switch client to use new HTTP transport layer that we recently tested and it should not have such issue.

Sure, I'll investigate using it. Is there any docs about the new and old implementation is? N.B. This is marked in the code as: @deprecated - do not use - it is only for development

Is the remote server Cloud instance? Is there a proxy between client and the server?

Yes using CH cloud.

chernser commented 1 month ago

@maxsumrall Initially we have created new client v2 API and had to use old client for testing and trying out the new API. While that time we have developed a new transport layer what is called a "newImplementation". In the future we will remove usage of an old client to deprecate it.