apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
7.86k stars 1.77k forks source link

[Question] Synchronizing data from PostgreSQL to MySQL with error: not a valid URL #7051

Closed xjdAlan closed 1 month ago

xjdAlan commented 3 months ago

Search before asking

What happened

I expect to synchronize data tables from PostgreSQL to MySQL, but executing tasks always results in an error message. The jdbc URL is invalid:not a valid URL

SeaTunnel Version

2.3.5

SeaTunnel Config

# Defining the runtime environment
env {
  parallelism = 4
  job.mode = "BATCH"
}
source{
    Jdbc {
        url = "jdbc:postgresql://192.168.0.221:5432/studio"
        driver = "org.postgresql.Driver"
        user = "postgres"
        password = "******"
        table_path = "studio.public.account_pptt00"
    }
}

transform {
    # If you would like to get more information about how to configure seatunnel and see full list of transform plugins,
    # please go to https://seatunnel.apache.org/docs/transform-v2/sql
}

sink {
    jdbc {
        # if you would use json or jsonb type insert please add jdbc url stringtype=unspecified option
        url = "jdbc:mysql://192.168.0.63:3306/studio_tar?serverTimezone=GMT%2b8&useUnicode=true&characterEncoding=UTF-8&rewriteBatchedStatements=true"
        driver = "com.mysql.cj.jdbc.Driver"
        user = "datacenter"
        password = "****"

        generate_sink_sql = true
        #You need to configure both database and table
        database = "studio_tar"
        table = "${table_name} "
    }

}

Running Command

./bin/seatunnel.sh --config config/v2.stream.single.pgsql-2-mysql -e local

Error Exception

Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:202)
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.api.table.factory.FactoryException: ErrorCode:[API-06], ErrorDescription:[Factory initialize failed] - Unable to create a source for identifier 'Jdbc'.
        at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:100)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSource(MultipleTableJobConfigParser.java:332)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:188)
        at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:88)
        at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:156)
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:149)
        ... 2 more
Caused by: org.apache.seatunnel.api.table.catalog.exception.CatalogException: ErrorCode:[API-03], ErrorDescription:[Catalog initialize failed] - Failed connecting to jdbc:postgresql://192.168.0.221:5432/studio via JDBC.
        at org.apache.seatunnel.connectors.seatunnel.jdbc.catalog.AbstractJdbcCatalog.getConnection(AbstractJdbcCatalog.java:123)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.catalog.AbstractJdbcCatalog.open(AbstractJdbcCatalog.java:129)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.utils.JdbcCatalogUtils.getTables(JdbcCatalogUtils.java:78)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.source.JdbcSource.<init>(JdbcSource.java:57)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.source.JdbcSourceFactory.lambda$createSource$0(JdbcSourceFactory.java:78)
        at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:112)
        at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSource(FactoryUtil.java:73)
        ... 7 more
Caused by: java.sql.SQLException: not a valid url
        at com._4paradigm.openmldb.jdbc.SQLDriver.parseAndMergeClusterProps(SQLDriver.java:84)
        at com._4paradigm.openmldb.jdbc.SQLDriver.connect(SQLDriver.java:52)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:247)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.catalog.AbstractJdbcCatalog.getConnection(AbstractJdbcCatalog.java:119)
        ... 13 more

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

Code of Conduct

Carl-Zhou-CN commented 3 months ago

Doesn't seem to be PG driven

xjdAlan commented 3 months ago

Doesn't seem to be PG driven

I placed the postgresql-42.7.3. jar package in ${SEATUNNEL_HOME}/lib and ${SEATUNNEL_HOME}/connectors. If synchronizing from MySQL to postgreSQL, it can be successful, but if synchronizing from postgreSQL to MySQL, this error will be reported. I tried to delete the connector openmldb 2.3.5. jar package under ${SEATUNNEL_HOME}/lib, but encountered an error: java.sql.SQLException: No suitable driver found for jdbc:postgresql://192.168.0.221:5432/studio. I feel like I didn't use the Postgresql-42.7.3. jar driver

github-actions[bot] commented 2 months ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] commented 1 month ago

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.