search

itinycheng / flink-connector-clickhouse

Flink SQL connector for ClickHouse. Support ClickHouseCatalog and read/write primary data, maps, arrays to clickhouse.
Apache License 2.0
363 stars 154 forks source link

use options use-local:Unable to establish connection to ClickHouse #38

Closed hehetown closed 2 years ago

hehetown commented 2 years ago

sql: CREATE TABLE sink_table (

 ID String,

 NAME String,

 PRIMARY KEY (ID) NOT ENFORCED) WITH (

'connector' = 'clickhouse', 

'sink.batch-size' =  '500', 

'sink.ignore-delete' =  'false', 

'sink.flush-interval' =  '1000', 

'sink.max-retries' =  '300', 
'use-local' =  'true', 
'url' =  'clickhouse://127.0.0.1:8123', 
'database-name' =  'default', 

'username' =  'clickhouse', 

'password' =  'clickhouse', 

'table-name' =  'demo' )

error: java.io.IOException: Unable to establish connection to ClickHouse at org.apache.flink.connector.clickhouse.internal.ClickHouseShardOutputFormat.open(ClickHouseShardOutputFormat.java:106) at org.apache.flink.streaming.api.functions.sink.OutputFormatSinkFunction.open(OutputFormatSinkFunction.java:65) at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:34) at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:100) at org.apache.flink.table.runtime.operators.sink.SinkOperator.open(SinkOperator.java:58) at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:110) at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:711) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55) at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:687) at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:654) at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958) at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:927) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) at java.lang.Thread.run(Thread.java:750) Caused by: ru.yandex.clickhouse.except.ClickHouseUnknownException: ClickHouse exception, code: 1002, host: 127.0.0.1, port: 9000; Magic is not correct: 60 at ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.getException(ClickHouseExceptionSpecifier.java:92) at ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.specify(ClickHouseExceptionSpecifier.java:56) at ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.specify(ClickHouseExceptionSpecifier.java:25) at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:351) at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:324) at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:319) at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:314) at ru.yandex.clickhouse.ClickHouseConnectionImpl.initConnection(ClickHouseConnectionImpl.java:91) at ru.yandex.clickhouse.ClickHouseConnectionImpl.(ClickHouseConnectionImpl.java:78) at ru.yandex.clickhouse.ClickHouseDriver.connect(ClickHouseDriver.java:62) at ru.yandex.clickhouse.BalancedClickhouseDataSource.getConnection(BalancedClickhouseDataSource.java:195) at org.apache.flink.connector.clickhouse.internal.connection.ClickHouseConnectionProvider.createConnection(ClickHouseConnectionProvider.java:129) at org.apache.flink.connector.clickhouse.internal.connection.ClickHouseConnectionProvider.createAndStoreShardConnection(ClickHouseConnectionProvider.java:95) at org.apache.flink.connector.clickhouse.internal.connection.ClickHouseConnectionProvider.createShardConnections(ClickHouseConnectionProvider.java:82) at org.apache.flink.connector.clickhouse.internal.ClickHouseShardOutputFormat.open(ClickHouseShardOutputFormat.java:85) ... 14 more Caused by: java.io.IOException: Magic is not correct: 60 at ru.yandex.clickhouse.response.ClickHouseLZ4Stream.readNextBlock(ClickHouseLZ4Stream.java:94) at ru.yandex.clickhouse.response.ClickHouseLZ4Stream.checkNext(ClickHouseLZ4Stream.java:75) at ru.yandex.clickhouse.response.ClickHouseLZ4Stream.read(ClickHouseLZ4Stream.java:51) at ru.yandex.clickhouse.response.StreamSplitter.readFromStream(StreamSplitter.java:92) at ru.yandex.clickhouse.response.StreamSplitter.next(StreamSplitter.java:53) at ru.yandex.clickhouse.response.ClickHouseResultSet.(ClickHouseResultSet.java:95) at ru.yandex.clickhouse.ClickHouseStatementImpl.createResultSet(ClickHouseStatementImpl.java:1121) at ru.yandex.clickhouse.ClickHouseStatementImpl.updateResult(ClickHouseStatementImpl.java:224) at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:344) ... 25 more

hehetown commented 2 years ago

query:SELECT shard_num, host_address, port FROM system.clusters WHERE cluster = ? and replica_num = 1 ORDER BY shard_num ASC

I found that there is a node whose host_address is 127.0.0.1, so the network is disconnected. But I don't know how to fix host_address being 127.0.0.1

itinycheng commented 2 years ago

query:SELECT shard_num, host_address, port FROM system.clusters WHERE cluster = ? and replica_num = 1 ORDER BY shard_num ASC

I found that there is a node whose host_address is 127.0.0.1, so the network is disconnected. But I don't know how to fix host_address being 127.0.0.1

system.clusters is a system table, it stores data from the <clickhouse_remote_servers> part in config file /etc/clickhouse-server/config.xml by default.