search

itinycheng / flink-connector-clickhouse

Flink SQL connector for ClickHouse. Support ClickHouseCatalog and read/write primary data, maps, arrays to clickhouse.
Apache License 2.0
374 stars 156 forks source link

e.displayText() = DB::Exception: Mutations are not supported by storage Distributed #39

Closed hehetown closed 2 years ago

hehetown commented 2 years ago

How to solve the ClickHouse distributed write problem? The following error occurs when I use a regular statement: Code: 48, e.displayText() = DB::Exception: There was an error on [chi-bpiih-clickhouse-gxqtcluster-0-0:9000]: Code: 48, e.displayText() = DB::Exception: Mutations are not supported by storage Distributed (version 21.1.3.32) (version 21.1.3.32)

at ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.specify(ClickHouseExceptionSpecifier.java:59) ~[flink-connector-clickhouse-1.14.3-SNAPSHOT.jar:?]
at ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.specify(ClickHouseExceptionSpecifier.java:29) ~[flink-connector-clickhouse-1.14.3-SNAPSHOT.jar:?]
at ru.yandex.clickhouse.ClickHouseStatementImpl.checkForErrorAndThrow(ClickHouseStatementImpl.java:1094) ~[flink-connector-clickhouse-1.14.3-SNAPSHOT.jar:?]
at ru.yandex.clickhouse.ClickHouseStatementImpl.getInputStream(ClickHouseStatementImpl.java:773) ~[flink-connector-clickhouse-1.14.3-SNAPSHOT.jar:?]
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeStatement(ClickHouseStatementImpl.java:255) ~[flink-connector-clickhouse-1.14.3-SNAPSHOT.jar:?]
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeBatch(ClickHouseStatementImpl.java:593) ~[flink-connector-clickhouse-1.14.3-SNAPSHOT.jar:?]
at ru.yandex.clickhouse.ClickHousePreparedStatementImpl.executeBatch(ClickHousePreparedStatementImpl.java:388) ~[flink-connector-clickhouse-1.14.3-SNAPSHOT.jar:?]
at ru.yandex.clickhouse.ClickHousePreparedStatementImpl.executeBatch(ClickHousePreparedStatementImpl.java:364) ~[flink-connector-clickhouse-1.14.3-SNAPSHOT.jar:?]
at org.apache.flink.connector.clickhouse.internal.executor.ClickHouseExecutor.attemptExecuteBatch(ClickHouseExecutor.java:53) ~[flink-connector-clickhouse-1.14.3-SNAPSHOT.jar:?]
at org.apache.flink.connector.clickhouse.internal.executor.ClickHouseUpsertExecutor.executeBatch(ClickHouseUpsertExecutor.java:125) ~[flink-connector-clickhouse-1.14.3-SNAPSHOT.jar:?]
itinycheng commented 2 years ago

Distributed table don't support update/delete statement, We can directly write data to local table, set conf use-local=true and do some tests. In addition, we'd better don't use upsert mode but insert to overwrite data in the MergeTree table engine instead.

hehetown commented 2 years ago

Thank you very much. Let's try to replace the upsert mode with insert.

hehetown commented 2 years ago

use-local=true, report error:ClickHouse exception, code: 341, host: 10.233.79.20, port: 8123; Code: 341, e.displayText() = DB::Exception: There was an error on [demo-clickhouse-1-0:9000]: Cannot execute replicated DDL query, maximum retires exceeded (version 21.1.3.32)

I don't know what to do with it. Ask for help..

itinycheng commented 2 years ago

use-local=true, report error:ClickHouse exception, code: 341, host: 10.233.79.20, port: 8123; Code: 341, e.displayText() = DB::Exception: There was an error on [demo-clickhouse-1-0:9000]: Cannot execute replicated DDL query, maximum retires exceeded (version 21.1.3.32)

I don't know what to do with it. Ask for help..

This is an exception thrown by the clickhouse server, you'd better get the more useful info from the server log. DDL statement is generated under upsert mode, you can do some breakpoint tests to get what DDL executed.