apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
8.03k stars 1.82k forks source link

[Bug] [mysql-cdc] Generate Splits for table #7298

Closed 13813586515 closed 2 months ago

13813586515 commented 3 months ago

Search before asking

What happened

mysql-cdc ->doris exception

SeaTunnel Version

2.3.5

SeaTunnel Config

env {
  parallelism = 1
  job.mode = "STREAMING"
  checkpoint.interval = 10000
}

source {
  MySQL-CDC {
    result_table_name = "customers_mysql_cdc"
    base-url = "jdbc:mysql://xxxx:3306/abc"
    username = "root"
    password = "password"
    table-names = ["abc.seatunnel_cdc_test"]
    table-names-config = [
      {
        table = "abc.seatunnel_cdc_test"
        primaryKeys = ["id"]
      }
    ]
    startup.mode = "initial"
  }
}

sink {
  Doris {
    fenodes = "xxxx:58030"
    username = root
    password = "password"
    database = "ods_xxx"
    table = "ods_xxx"
    sink.label-prefix = "ods_xxx_seatunnel_cdc_test"
    sink.enable-2pc = "true"
    sink.enable-delete = "true"
    doris.config {
      format = "json"
      read_json_by_line = "true"
    }

  }
}

Running Command

mysql create table sql:
CREATE TABLE `xxx`.`seatunnel_cdc_test`  (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL,
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 2 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

Error Exception

java.util.concurrent.ExecutionException: java.lang.RuntimeException: Generate Splits for table xxx.seatunnel_cdc_test error

Zeta or Flink or Spark Version

Zeta

Java or Scala Version

java1.8

Screenshots

No response

Are you willing to submit PR?

Code of Conduct

github-actions[bot] commented 2 months ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] commented 2 months ago

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.