ClickHouse / clickhouse-jdbc-bridge

A JDBC proxy from ClickHouse to external databases
Apache License 2.0
168 stars 60 forks source link

Troubles while getting data from Oracle #168

Closed mehanizm closed 1 year ago

mehanizm commented 1 year ago

I use clickhouse-jdbc-bridge to get data from Oracle. Native query from DataGrip to oracle took long time, approximately 20 minutes.

When I try to query from Clickhouse through clickhouse-jdbc-bridge it works okey for the small queries (with rownum limit).

But I cannot get all data and get different errors with no changes.

Variant 1 In clickhouse-jdbc-bridge log

[2022-12-16 06:16:55] [WARNING] Thread Thread[vert.x-eventloop-thread-0,5,main]=Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 116813 ms, time limit is 15000 ms
[2022-12-16 06:16:56] [WARNING] Thread Thread[vert.x-eventloop-thread-0,5,main]=Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 117813 ms, time limit is 15000 ms
...

And in Clickhouse after exactly 3 minutes

Poco::Exception. Code: 1000, e.code() = 0, Timeout (version 22.10.2.11 (official build))

Variant 2 In clickhouse-jdbc-bridge log

[2022-12-16 12:54:51] [INFO   ] Raw query:
...
[2022-12-16 12:54:51] [INFO   ] Executing query(schema=[mobilen]):
...
[2022-12-16 12:54:51] [WARNING] Failed call setClientInfo due to Invalid or unsupported name for clientInfo.
[2022-12-16 12:57:51] [SEVERE ] Caught exception
[2022-12-16 12:57:51] [INFO   ] Executing query(schema=[mobilen]):
...
[2022-12-16 12:57:5
... exactly every 3 minute I've got the same string
... and no result for a looooong time

Which config did I try

max_exection_time in Clickhouse is set to 12000000 socket_timeout in connection to Clickhouse is set to 12000000 queryTimeout in config/server.json is set to 12000000 maxWorkerExecuteTime in config/vertx.json is set to 12000000

"dataSource": {
            "oracle.jdbc.ReadTimeout": 12000000
        },
"maxLifetime": 12000000,
"connectionTimeout": 300000,

in oracle jdbc-bridge datasource configuration

What should I try next?

mehanizm commented 1 year ago

If it's important I make an insert query:

insert into table
select * from jdbc(...)
mehanizm commented 1 year ago

It was http_receive_timeout option in Clickhouse settings Create PR #170 170

AlexGruPerm commented 7 months ago

Sorry, I have the same problem and can't find where I need set parameter - http_receive_timeout. Could you help me please?