apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
7.59k stars 1.67k forks source link

SQLservercdc将配置文件所有表读取出来了,无视配置文件 #4476

Open bigdata-liu opened 1 year ago

bigdata-liu commented 1 year ago

Search before asking

What happened

读取配置文件时候把库里面的所有表都读取出来了,然后按照第一张表进行增量读取

SeaTunnel Version

2.3.1

SeaTunnel Config

env {
 execution.parallelism = 1
}   
source {
  SqlServer-CDC {
    hostname = "192.168.0.74"
    port = "1434"
    username = "sa"
    password = "123456aA??"
    database-names = ["testdb"]
    table-name = "testdb.dbo.sqlserver-cdc-oracle"
  }
}
transform {
}
sink {
  Jdbc {
    url = "jdbc:oracle:thin:@192.168.0.74:1521:helowin"
    driver = "oracle.jdbc.OracleDriver"
    user = "system"
    database = "HR"
    password = "helowin"
    table = SQLSERVER_CDC_TO_ORACLE
    primary_keys = ["ID"]
  }
}

Running Command

bin/seatunnel.sh --config ./config/sqlserver-cdc-oracle.conf -e local

Error Exception

2023-04-02 14:20:24,602 INFO  org.apache.seatunnel.connectors.seatunnel.cdc.sqlserver.source.utils.TableDiscoveryUtils -     list of available databases is: [master, tempdb, model, msdb, testdb]
2023-04-02 14:20:24,602 INFO  org.apache.seatunnel.connectors.seatunnel.cdc.sqlserver.source.utils.TableDiscoveryUtils - Read list of available tables in each database
2023-04-02 14:20:25,076 INFO  org.apache.seatunnel.connectors.seatunnel.cdc.sqlserver.source.utils.TableDiscoveryUtils -     including 'testdb.dbo.sqlserver_cdc_to_greenplum_test' for further processing
2023-04-02 14:20:25,076 INFO  org.apache.seatunnel.connectors.seatunnel.cdc.sqlserver.source.utils.TableDiscoveryUtils -     including 'testdb.dbo.sqlserver_cdc_to_oracle' for further processing
2023-04-02 14:20:25,079 INFO  io.debezium.jdbc.JdbcConnection - Connection gracefully closed
2023-04-02 14:20:25,134 INFO  org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - received enough reader, starting enumerator...
2023-04-02 14:20:25,179 INFO  org.apache.seatunnel.connectors.seatunnel.cdc.sqlserver.source.source.eumerator.SqlServerChunkSplitter - Start splitting table testdb.dbo.sqlserver_cdc_to_greenplum_test into chunks

Flink or Spark Version

2.3.1

Java or Scala Version

jdk 8

Screenshots

image

Are you willing to submit PR?

Code of Conduct

hk-lrzy commented 1 year ago

configuration should be table-names

davidzollo commented 1 year ago

good job Can you replace the title and description in English?