apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
7.76k stars 1.73k forks source link

[Bug] [Module Name] Bug title mysql cdc to clickhouse ILIKE '%allow_experimental_lightweight_delete%'. Expected PROFILES (version 20.8.3.18) #5324

Closed Achilles1996 closed 1 year ago

Achilles1996 commented 1 year ago

Search before asking

What happened

Caused by: org.apache.seatunnel.common.utils.SeaTunnelException: org.apache.seatunnel.connectors.seatunnel.clickhouse.exception.ClickhouseConnectorException: ErrorCode:[COMMON-08], ErrorDescription:[Sql operation failed, such as (execute,addBatch,close) etc...] - java.sql.SQLException: Code: 62, e.displayText() = DB::Exception: Syntax error: failed at position 15 ('ILIKE'): ILIKE '%allow_experimental_lightweight_delete%'. Expected PROFILES (version 20.8.3.18)

because my clickhouse version 的太低not support allow_experimental_lightweight_delete?

SeaTunnel Version

2.3.2

SeaTunnel Config

env {
  execution.parallelism = 2
  job.mode = "STREAMING"
  checkpoint.interval = 2000
}

source {
  MySQL-CDC {
    result_table_name = "fake"
    parallelism = 1
    server-id = 5656
    username = "mysql"
    password = "jjY@xklby1025"
    table-names = ["txjm_ppf.t_bp_salepos_d"]
    base-url = "jdbc:mysql://192.168.208.100:3316/txjm_ppf"
  }
}

sink {
  Clickhouse {
    host = "192.168.12.34:8123"
    database = "mysql_JM"
    table = "t_bp_salepos_d"
    username = "default"
    password = "CK@123"
    # cdc options
    primary_key = "bill"
    support_upsert = true
    allow_experimental_lightweight_delete = true
  }
}

Running Command

./bin/seatunnel.sh --config ./config/v2.streaming.mysql2ck -e local

Error Exception

Caused by: org.apache.seatunnel.common.utils.SeaTunnelException: org.apache.seatunnel.connectors.seatunnel.clickhouse.exception.ClickhouseConnectorException: ErrorCode:[COMMON-08], ErrorDescription:[Sql operation failed, such as (execute,addBatch,close) etc...] - java.sql.SQLException: Code: 62, e.displayText() = DB::Exception: Syntax error: failed at position 15 ('ILIKE'): ILIKE '%allow_experimental_lightweight_delete%'. Expected PROFILES (version 20.8.3.18)

because my clickhouse  version 的太低not support allow_experimental_lightweight_delete?

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

Code of Conduct

hailin0 commented 1 year ago

Recommended fix logic

https://github.com/apache/seatunnel/blob/dev/seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/client/ClickhouseSinkWriter.java#L211

image