search

itinycheng / flink-connector-clickhouse

Flink SQL connector for ClickHouse. Support ClickHouseCatalog and read/write primary data, maps, arrays to clickhouse.
Apache License 2.0
346 stars 148 forks source link

[Bug]: 经常出现刷新异常 #140

Open ysq5202121 opened 1 week ago

ysq5202121 commented 1 week ago

What happened?

Caused by: java.lang.RuntimeException: Flush exception found. at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkFlushException(AbstractClickHouseOutputFormat.java:117) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkBeforeFlush(AbstractClickHouseOutputFormat.java:86) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.flush(ClickHouseBatchOutputFormat.java:93) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.lambda$scheduledFlush$0(AbstractClickHouseOutputFormat.java:73) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131] ... 1 more Caused by: java.lang.RuntimeException: Flush exception found. at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkFlushException(AbstractClickHouseOutputFormat.java:117) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkBeforeFlush(AbstractClickHouseOutputFormat.java:86) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.flush(ClickHouseBatchOutputFormat.java:93) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.lambda$scheduledFlush$0(AbstractClickHouseOutputFormat.java:73) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131] ... 1 more Caused by: java.lang.RuntimeException: Flush exception found. at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkFlushException(AbstractClickHouseOutputFormat.java:117) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkBeforeFlush(AbstractClickHouseOutputFormat.java:86) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.flush(ClickHouseBatchOutputFormat.java:93) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.lambda$scheduledFlush$0(AbstractClickHouseOutputFormat.java:73) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131] ... 1 more Caused by: java.lang.RuntimeException: Flush exception found. at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkFlushException(AbstractClickHouseOutputFormat.java:117) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkBeforeFlush(AbstractClickHouseOutputFormat.java:86) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.flush(ClickHouseBatchOutputFormat.java:93) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.lambda$scheduledFlush$0(AbstractClickHouseOutputFormat.java:73) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131] ... 1 more Caused by: java.lang.RuntimeException: Flush exception found. at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkFlushException(AbstractClickHouseOutputFormat.java:117) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkBeforeFlush(AbstractClickHouseOutputFormat.java:86) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.flush(ClickHouseBatchOutputFormat.java:93) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.lambda$scheduledFlush$0(AbstractClickHouseOutputFormat.java:73) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131] ... 1 more Caused by: java.lang.RuntimeException: Flush exception found. at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkFlushException(AbstractClickHouseOutputFormat.java:117) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkBeforeFlush(AbstractClickHouseOutputFormat.java:86) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.flush(ClickHouseBatchOutputFormat.java:93) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.lambda$scheduledFlush$0(AbstractClickHouseOutputFormat.java:73) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131] ... 1 more Caused by: java.lang.RuntimeException: Flush exception found. at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkFlushException(AbstractClickHouseOutputFormat.java:117) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkBeforeFlush(AbstractClickHouseOutputFormat.java:86) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.flush(ClickHouseBatchOutputFormat.java:93) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.lambda$scheduledFlush$0(AbstractClickHouseOutputFormat.java:73) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131] ... 1 more Caused by: java.lang.RuntimeException: Flush exception found. at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkFlushException(AbstractClickHouseOutputFormat.java:117) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkBeforeFlush(AbstractClickHouseOutputFormat.java:86) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.flush(ClickHouseBatchOutputFormat.java:93) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.lambda$scheduledFlush$0(AbstractClickHouseOutputFormat.java:73) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131] ... 1 more Caused by: java.lang.RuntimeException: Flush exception found. at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkFlushException(AbstractClickHouseOutputFormat.java:117) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkBeforeFlush(AbstractClickHouseOutputFormat.java:86) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.flush(ClickHouseBatchOutputFormat.java:93) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.lambda$scheduledFlush$0(AbstractClickHouseOutputFormat.java:73) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131] ... 1 more Caused by: java.lang.RuntimeException: Flush exception found. at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkFlushException(AbstractClickHouseOutputFormat.java:117) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkBeforeFlush(AbstractClickHouseOutputFormat.java:86) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.flush(ClickHouseBatchOutputFormat.java:93) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.lambda$scheduledFlush$0(AbstractClickHouseOutputFormat.java:73) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131] ... 1 more Caused by: java.lang.RuntimeException: Flush exception found. at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkFlushException(AbstractClickHouseOutputFormat.java:117) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkBeforeFlush(AbstractClickHouseOutputFormat.java:86) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.flush(ClickHouseBatchOutputFormat.java:93) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.lambda$scheduledFlush$0(AbstractClickHouseOutputFormat.java:73) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131] ... 1 more Caused by: java.lang.RuntimeException: Flush exception found. at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkFlushException(AbstractClickHouseOutputFormat.java:117) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkBeforeFlush(AbstractClickHouseOutputFormat.java:86) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.flush(ClickHouseBatchOutputFormat.java:93) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.lambda$scheduledFlush$0(AbstractClickHouseOutputFormat.java:73) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131] ... 1 more Caused by: java.lang.RuntimeException: Flush exception found. at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkFlushException(AbstractClickHouseOutputFormat.java:117) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkBeforeFlush(AbstractClickHouseOutputFormat.java:86) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.flush(ClickHouseBatchOutputFormat.java:93) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.lambda$scheduledFlush$0(AbstractClickHouseOutputFormat.java:73) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131] ... 1 more Caused by: java.lang.RuntimeException: Flush exception found. at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkFlushException(AbstractClickHouseOutputFormat.java:117) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkBeforeFlush(AbstractClickHouseOutputFormat.java:86) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.flush(ClickHouseBatchOutputFormat.java:93) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.lambda$scheduledFlush$0(AbstractClickHouseOutputFormat.java:73) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131] ... 1 more Caused by: java.io.IOException: java.sql.SQLException: Attempt to execute batch failed, exhausted retry times = 3 at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkBeforeFlush(AbstractClickHouseOutputFormat.java:90) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.flush(ClickHouseBatchOutputFormat.java:93) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.lambda$scheduledFlush$0(AbstractClickHouseOutputFormat.java:73) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131] ... 1 more Caused by: java.sql.SQLException: Attempt to execute batch failed, exhausted retry times = 3 at org.apache.flink.connector.clickhouse.internal.executor.ClickHouseExecutor.attemptExecuteBatch(ClickHouseExecutor.java:54) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.executor.ClickHouseUpsertExecutor.executeBatch(ClickHouseUpsertExecutor.java:127) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkBeforeFlush(AbstractClickHouseOutputFormat.java:88) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.flush(ClickHouseBatchOutputFormat.java:93) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.lambda$scheduledFlush$0(AbstractClickHouseOutputFormat.java:73) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131] ... 1 more Caused by: org.apache.flink.shaded.clickhouse.ru.yandex.clickhouse.except.ClickHouseUnknownException: ClickHouse exception, code: 1002, host: clickhouse.bestwo.net, port: 8123; clickhouse.bestwo.net at org.apache.flink.shaded.clickhouse.ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.getException(ClickHouseExceptionSpecifier.java:92) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.specify(ClickHouseExceptionSpecifier.java:56) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.specify(ClickHouseExceptionSpecifier.java:25) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.ru.yandex.clickhouse.ClickHouseStatementImpl.sendStream(ClickHouseStatementImpl.java:1071) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.ru.yandex.clickhouse.ClickHouseStatementImpl.sendStream(ClickHouseStatementImpl.java:1026) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.ru.yandex.clickhouse.ClickHouseStatementImpl.sendStream(ClickHouseStatementImpl.java:1019) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.ru.yandex.clickhouse.ClickHousePreparedStatementImpl.executeBatch(ClickHousePreparedStatementImpl.java:381) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.ru.yandex.clickhouse.ClickHousePreparedStatementImpl.executeBatch(ClickHousePreparedStatementImpl.java:364) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.connection.ClickHouseStatementWrapper.executeBatch(ClickHouseStatementWrapper.java:24) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.executor.ClickHouseExecutor.attemptExecuteBatch(ClickHouseExecutor.java:48) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.executor.ClickHouseUpsertExecutor.executeBatch(ClickHouseUpsertExecutor.java:127) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkBeforeFlush(AbstractClickHouseOutputFormat.java:88) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.flush(ClickHouseBatchOutputFormat.java:93) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.lambda$scheduledFlush$0(AbstractClickHouseOutputFormat.java:73) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131] ... 1 more Caused by: java.net.UnknownHostException: clickhouse.bestwo.net at java.net.InetAddress.getAllByName0(InetAddress.java:1280) ~[?:1.8.0_131] at java.net.InetAddress.getAllByName(InetAddress.java:1192) ~[?:1.8.0_131] at java.net.InetAddress.getAllByName(InetAddress.java:1126) ~[?:1.8.0_131] at org.apache.flink.shaded.clickhouse.org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.ru.yandex.clickhouse.util.IpVersionPriorityResolver.resolve(IpVersionPriorityResolver.java:29) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.ru.yandex.clickhouse.ClickHouseStatementImpl.sendStream(ClickHouseStatementImpl.java:1059) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.ru.yandex.clickhouse.ClickHouseStatementImpl.sendStream(ClickHouseStatementImpl.java:1026) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.ru.yandex.clickhouse.ClickHouseStatementImpl.sendStream(ClickHouseStatementImpl.java:1019) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.ru.yandex.clickhouse.ClickHousePreparedStatementImpl.executeBatch(ClickHousePreparedStatementImpl.java:381) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.shaded.clickhouse.ru.yandex.clickhouse.ClickHousePreparedStatementImpl.executeBatch(ClickHousePreparedStatementImpl.java:364) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.connection.ClickHouseStatementWrapper.executeBatch(ClickHouseStatementWrapper.java:24) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.executor.ClickHouseExecutor.attemptExecuteBatch(ClickHouseExecutor.java:48) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.executor.ClickHouseUpsertExecutor.executeBatch(ClickHouseUpsertExecutor.java:127) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.checkBeforeFlush(AbstractClickHouseOutputFormat.java:88) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.flush(ClickHouseBatchOutputFormat.java:93) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.lambda$scheduledFlush$0(AbstractClickHouseOutputFormat.java:73) ~[flink-connector-clickhouse-1.16.0-SNAPSHOT.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131] ... 1 more

Affects Versions

1.16.0

What are you seeing the problem on?

No response

How to reproduce

No response

Relevant log output

No response

Anything else

No response

Are you willing to submit a PR?

Code of Conduct

czy006 commented 1 week ago

@ysq5202121 Do you have any cases that can be triggered. Is it writing or reading? Are you using the latest Connector test? The old version is no longer within the scope of maintenance

ysq5202121 commented 1 week ago

偶发性的，不是最新版本，因为我的flink是1.16.2版本。

czy006 commented 1 week ago

偶发性的，不是最新版本，因为我的flink是1.16.2版本。

use new version to test it, Versions lower than Flink 1.18 are no longer supported.

itinycheng commented 1 week ago

@ysq5202121 Data is written into ClickHouse in batches. If a batch write fails, subsequent writes will raise an exception directly, you need to check the first error, the content is Caused by: java.net.UnknownHostException: clickhouse.bestwo.net