search

itinycheng / flink-connector-clickhouse

Flink SQL connector for ClickHouse. Support ClickHouseCatalog and read/write primary data, maps, arrays to clickhouse.
Apache License 2.0
349 stars 149 forks source link

Refactor the code to use AbstractDialect #12

Closed snuyanzin closed 9 months ago

snuyanzin commented 2 years ago

In https://issues.apache.org/jira/browse/FLINK-24253 internal jdbc dialect classes have been made public. It makes sense to refactor the code to extend them. However it requires upgrade to 1.15.x (should be released in Feb 2022)

I would be happy to help here, however first I would like to know if there are any blockers for such an update?

itinycheng commented 2 years ago

In my plan, the next work is to upgrade Flink to 1.14.

About refactoring the code to fit the flink-connector-jdbc module, I'm not optimistic about that:

  1. We can directly read/write the local tables of the distributed tables, which can't compatible with flink-connector-jdbc, only extending classes like AbstractDialect brings few benefits.
  2. Source function of flink-connector-jdbc doesn't implement the SupportsFilterPushDown interface.

If there is a better way, we can discuss it or open a new branch to try it alone?

snuyanzin commented 2 years ago

Thank for your feedback and highlights, I like your idea about try with a branch. I will try it in my sandbox and once I get some positive result I will come back here

itinycheng commented 2 years ago

OK, welcome any follow-up feedback.