[Bug] [Seatunnel] jdbc-mysql to doris error

[luban-130]

Search before asking

• [X] I had searched in the issues and found no similar issues.

What happened

当我使用seatunnel将mysql数据同步至doris数据库时报错 When I use Seatunnel to synchronize MySQL data to Doris database, an error occurs.

SeaTunnel Version

v2.3.5

SeaTunnel Config

env {
  # You can set flink configuration here
  execution.parallelism = 1
  job.mode = "BATCH"
}
source{
    Jdbc {
        url = "jdbc:mysql:/*****:3306/indicator"
        driver = "com.mysql.cj.jdbc.Driver"
        connection_check_timeout_sec = 100
        user = "***"
        password = "****"
        table_list = [
          {  
             table_path = "indicator.dwd_cost_deptcost_chare_result_df"
          }  
          {  
             table_path = "indicator.dwd_hrp_employee_mt"
          }
        ]
    }
}

sink {
  Doris {
    fenodes = "$ip:8030"
    query-port = 9030
    username = ****
    password = "****"
    database = "indicator"
    table = "${table_name}_from_mysql"
    sink.label-prefix = "test_json"
    sink.enable-2pc = "true"
    doris.config {
      format = "json"
      read_json_by_line = "true"
    }
  }
}

Running Command

sh bin/seatunnel.sh --config ./config/mysqltodoris.conf

Error Exception

2024-06-18 16:54:34,953 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - 
===============================================================================

Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:202)
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.common.exception.SeaTunnelRuntimeException: ErrorCode:[API-09], ErrorDescription:[Handle save mode failed]
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.handleSaveMode(MultipleTableJobConfigParser.java:655)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.createSinkAction(MultipleTableJobConfigParser.java:641)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSink(MultipleTableJobConfigParser.java:553)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:200)
        at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:88)
        at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:156)
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:149)
        ... 2 more
Caused by: org.apache.seatunnel.api.table.catalog.exception.CatalogException: ErrorCode:[API-03], ErrorDescription:[Catalog initialize failed] - create table statement execute failed
        at org.apache.seatunnel.connectors.doris.catalog.DorisCatalog.createTable(DorisCatalog.java:291)
        at org.apache.seatunnel.api.sink.DefaultSaveModeHandler.createTable(DefaultSaveModeHandler.java:181)
        at org.apache.seatunnel.api.sink.DefaultSaveModeHandler.createSchemaWhenNotExist(DefaultSaveModeHandler.java:108)
        at org.apache.seatunnel.api.sink.DefaultSaveModeHandler.handleSchemaSaveMode(DefaultSaveModeHandler.java:69)
        at org.apache.seatunnel.api.sink.SaveModeHandler.handleSaveMode(SaveModeHandler.java:38)
        at org.apache.seatunnel.api.sink.SaveModeExecuteWrapper.execute(SaveModeExecuteWrapper.java:36)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.handleSaveMode(MultipleTableJobConfigParser.java:653)
        ... 8 more
Caused by: java.sql.SQLException: errCode = 2, detailMessage = Syntax error in line 89:
 UNIQUE KEY ()
             ^
Encountered: )
Expected: IDENTIFIER

        at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:129)
        at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
        at com.mysql.cj.jdbc.StatementImpl.executeInternal(StatementImpl.java:763)
        at com.mysql.cj.jdbc.StatementImpl.execute(StatementImpl.java:648)
        at org.apache.seatunnel.connectors.doris.catalog.DorisCatalog.createTable(DorisCatalog.java:289)
        ... 14 more

Zeta or Flink or Spark Version

zeta

Java or Scala Version

java -version java version "1.8.0_291" Java(TM) SE Runtime Environment (build 1.8.0_291-b10) Java HotSpot(TM) 64-Bit Server VM (build 25.291-b10, mixed mode)

Screenshots

No response

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[Carl-Zhou-CN]

It appears that the lack of a primary key in the upstream table is causing this issue

[luban-130]

Indeed, one of the upstream tables lacks a primary key. After replacing the table and attempting to synchronize again, an error was encountered with ErrorCode: [COMMON-15] and ErrorDescription: [Class load operation failed] - JDBC-Class not found. - com.mysql.cj.jdbc.Driver. However, the MySQL driver is already present in the lib directory under the home directory of seatunnel.

[luban-130]

Thank you for your answer. The issue has been resolved by replacing mysql-connector-java-8.0.28 with 8.0.30. Additionally, I would like to inquire if Seatunnel supports full database synchronization instead of using the tablelist approach. Thank you.

[Carl-Zhou-CN]

Thank you for your answer. The issue has been resolved by replacing mysql-connector-java-8.0.28 with 8.0.30. Additionally, I would like to inquire if Seatunnel supports full database synchronization instead of using the tablelist approach. Thank you.

This is currently not supported and requires enumerating all table names