search

Altinity / clickhouse-sink-connector

Replicate data from MySQL, Postgres and MongoDB to ClickHouse
https://www.altinity.com
Apache License 2.0
220 stars 50 forks source link

Support MySQL Polygon #819

Open poweroftrue opened 1 week ago

poweroftrue commented 1 week ago

Hello there,

This already implemented but have the following breaking issues:

  1. Nullable(polygon) if source DB has the Polygon null will throw an error: java.sql.SQLException: Code: 43. DB::Exception: Nested type Polygon cannot be inside Nullable type. (ILLEGAL_TYPE_OF_ARGUMENT) (version 24.7.1.1).
  2. If 1 is fixed manually still data won't get inserted:

java.lang.ClassCastException: class [B cannot be cast to class java.nio.ByteBuffer ([B and java.nio.ByteBuffer are in module java.base of loader 'bootstrap') at com.altinity.clickhouse.sink.connector.converters.ClickHouseDataTypeMapper.convert(ClickHouseDataTypeMapper.java:233) ~[app.jar:?] at com.altinity.clickhouse.sink.connector.db.batch.PreparedStatementExecutor.insertPreparedStatement(PreparedStatementExecutor.java:280) ~[app.jar:?] at com.altinity.clickhouse.sink.connector.db.batch.PreparedStatementExecutor.lambda$executePreparedStatement$0(PreparedStatementExecutor.java:143) ~[app.jar:?] at java.lang.Iterable.forEach(Iterable.java:75) [?:?] at com.altin


3. If 2 is fixed manually still the current implementation tries to store Polygon as HEX in a Clickhouse String, while maybe there's a use case for this, I believe the best way is to store it as Clickhouse Polygon for advance analytics in Clickhouse.

I will fix these issues and PR it today.
poweroftrue commented 1 week ago

Related partially https://github.com/Altinity/clickhouse-sink-connector/issues/79