apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
7.86k stars 1.77k forks source link

Null Pointer Exception while synchronising data from mysql to mysql using mysql cdc connector. #4371

Open sangeetha1706 opened 1 year ago

sangeetha1706 commented 1 year ago

Search before asking

What happened

Null Pointer Exception while synchronising data from mysql to mysql using mysql cdc connector.

SeaTunnel Version

apache-seatunnel-incubating-2.3.0

SeaTunnel Config

env {
   job.mode = "STREAMING"
  #jars = "/opt/RND/SANGEETHA/APACHE_SEATUNNEL/apache-seatunnel-incubating-2.3.0/connectors/seatunnel/connector-cdc-mysql-2.3.0.jar"
}
source {
  Mysql-CDC {
    result_table_name = "SINK_USER"
    parallelism = 1
    port = 3306
    server-id=5656
    hostname = "10.0.0.82"
    user = "hauser"
    password = "Hauser@7ujm"
    database-name = "test"
    #driver = "com.mysql.cj.jdbc.Driver"
    table-name = "SOURCE_USER"
    base-url = "jdbc:mysql://10.0.0.82:3306"
}}

transform {
}
sink {
jdbc {

url = "jdbc:mysql://10.0.0.82:3306/test?serverTimezone=GMT%2b8"
        driver = "com.mysql.cj.jdbc.Driver"
        user = "hauser"
        password = "Hauser@7ujm"
        primary_keys = ["ID"]
       table = SINK_USER
}}
~

Running Command

/bin/seatunnel.sh -c ./config/MYSQL-CDC.conf -e LOCAL

Error Exception

2023-03-19 22:49:37,086 ERROR org.apache.seatunnel.core.starter.Seatunnel - Reason:null 

2023-03-19 22:49:37,090 ERROR org.apache.seatunnel.core.starter.Seatunnel - Exception StackTrace:java.lang.NullPointerException
    at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:212)
    at org.apache.seatunnel.connectors.seatunnel.cdc.mysql.config.MySqlSourceConfigFactory.create(MySqlSourceConfigFactory.java:61)
    at org.apache.seatunnel.connectors.seatunnel.cdc.mysql.source.MySqlDialect.<init>(MySqlDialect.java:61)
    at org.apache.seatunnel.connectors.seatunnel.cdc.mysql.source.MySqlIncrementalSource.createDataSourceDialect(MySqlIncrementalSource.java:80)
    at org.apache.seatunnel.connectors.cdc.base.source.IncrementalSource.prepare(IncrementalSource.java:88)
    at org.apache.seatunnel.engine.core.parse.ConnectorInstanceLoader.loadSourceInstance(ConnectorInstanceLoader.java:60)
    at org.apache.seatunnel.engine.core.parse.JobConfigParser.sampleAnalyze(JobConfigParser.java:314)
    at org.apache.seatunnel.engine.core.parse.JobConfigParser.parse(JobConfigParser.java:125)
    at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.getLogicalDag(JobExecutionEnvironment.java:129)
    at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.execute(JobExecutionEnvironment.java:121)
    at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:91)
    at org.apache.seatunnel.core.starter.Seatunnel.run(Seatunnel.java:39)
    at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:31)

Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

Code of Conduct

laglangyue commented 1 year ago

you lost some conf,such as a query sql.

laglangyue commented 1 year ago

please refer to the document to complete necessary configuration

lightzhao commented 1 year ago

Your Mysql-CDC module configuration is missing the 'database.user' parameter