search

itinycheng / flink-connector-clickhouse

Flink SQL connector for ClickHouse. Support ClickHouseCatalog and read/write primary data, maps, arrays to clickhouse.
Apache License 2.0
349 stars 149 forks source link

It doesn't work with Flink 1.14.0 #6

Closed shouxinjk closed 2 years ago

shouxinjk commented 2 years ago

Got below error message. Caused by: java.lang.NoClassDefFoundError: org/apache/flink/runtime/util/ExecutorThreadFactory at org.apache.flink.connector.clickhouse.internal.AbstractClickHouseOutputFormat.scheduledFlush(AbstractClickHouseOutputFormat.java:59) at org.apache.flink.connector.clickhouse.internal.ClickHouseBatchOutputFormat.open(ClickHouseBatchOutputFormat.java:68)

It seems org/apache/flink/runtime/util/ExecutorThreadFactory has been removed from Flink 1.14.0. Do you have a plan to upgrade to Flink 1.14.0?

shouxinjk commented 2 years ago

Just replaced with org.apache.flink.util.concurrent.ExecutorThreadFactory. It works now though no sufficient tests.

itinycheng commented 2 years ago

Recently, I only worked on this project in my free time at weekends, I don't have a certain time about when to upgrade to flink 1.14.0, maybe it takes a few weeks.

The connector runs stably in my company's flink 1.13.2 cluster. In my opinion, after upgrading to flink 1.14.0 and fixing some visible problems, it should be run well in flink 1.14.0 cluster.

Currently, there are three things I need to do, in order:

  1. ClickHouseCatalog supports upsert mode.
  2. Connector supports source function.
  3. Upgrade flink to version 1.14.0.

I will complete these features as soon as possible. I will be very appreciative if you can help with some code.

itinycheng commented 2 years ago

The master branch has been upgraded to flink 1.14.3.