[Bug] [Flink] Flink engine, source is LocalFile, sink is jdbc, get "No suitable driver" Exception

luoyongsir commented 5 months ago

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened

Flink engine, source is LocalFile, sink is jdbc, get "No suitable driver" Exception. Before this, I had already placed the driver in "${SEATUNNEL_HOME}/plugins/jdbc/lib/" directory

SeaTunnel Version

2.3.5

SeaTunnel Config

env {
  job.mode = "BATCH"
  parallelism = 1
}

source {
  LocalFile {
  }
}

transform {
}

sink {
  jdbc {
  }
}

Running Command

${SEATUNNEL_HOME}/bin/start-seatunnel-flink-15-connector-v2.sh --config xxx.conf -e run --target remote

Error Exception

Caused by: org.apache.seatunnel.connectors.seatunnel.jdbc.exception.JdbcConnectorException: ErrorCode:[JDBC-04], ErrorDescription:[Connector database failed] - unable to open JDBC writer
        at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcOutputFormat.open(JdbcOutputFormat.java:69)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.sink.JdbcSinkWriter.tryOpen(JdbcSinkWriter.java:117)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.sink.JdbcSinkWriter.write(JdbcSinkWriter.java:128)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.sink.JdbcSinkWriter.write(JdbcSinkWriter.java:47)
        at org.apache.seatunnel.translation.flink.sink.FlinkSinkWriter.write(FlinkSinkWriter.java:91)
        at org.apache.flink.streaming.api.transformations.SinkV1Adapter$SinkWriterV1Adapter.write(SinkV1Adapter.java:136)
        at org.apache.flink.streaming.runtime.operators.sink.SinkWriterOperator.processElement(SinkWriterOperator.java:160)
        at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:233)
        at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134)
        at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105)
        at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:542)
        at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780)
        at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935)
        at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914)
        at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
        at java.lang.Thread.run(Thread.java:748)
    Caused by: java.lang.RuntimeException: Failed to get driver instance for jdbcUrl=jdbc:mysql://xxxx
        at org.apache.seatunnel.shade.com.zaxxer.hikari.util.DriverDataSource.<init>(DriverDataSource.java:114)
        at org.apache.seatunnel.shade.com.zaxxer.hikari.pool.PoolBase.initializeDataSource(PoolBase.java:331)
        at org.apache.seatunnel.shade.com.zaxxer.hikari.pool.PoolBase.<init>(PoolBase.java:114)
        at org.apache.seatunnel.shade.com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:108)
        at org.apache.seatunnel.shade.com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.sink.ConnectionPoolManager.lambda$getConnection$0(ConnectionPoolManager.java:45)
        at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.sink.ConnectionPoolManager.getConnection(ConnectionPoolManager.java:41)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.connection.SimpleJdbcConnectionPoolProviderProxy.getOrEstablishConnection(SimpleJdbcConnectionPoolProviderProxy.java:57)
        at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcOutputFormat.open(JdbcOutputFormat.java:67)
        ... 19 more
    Caused by: java.sql.SQLException: No suitable driver
        at java.sql.DriverManager.getDriver(DriverManager.java:315)
        at org.apache.seatunnel.shade.com.zaxxer.hikari.util.DriverDataSource.<init>(DriverDataSource.java:106)
        ... 28 more

Zeta or Flink or Spark Version

Flink 1.6.1

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

[x] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

ljl568419709 commented 5 months ago

你需要把驱动包和connectorjar包方在seatunnel 的lib目录下，才可以的

luoyongsir commented 5 months ago

的lib目录下试过了，没效果。而且官方文档说的就是放 "${SEATUNNEL_HOME}/plugins/jdbc/lib/" 目录

ljl568419709 commented 5 months ago

的lib目录下试过了，没效果。而且官方文档说的就是放 "${SEATUNNEL_HOME}/plugins/jdbc/lib/" 目录

怎么可能呢，我都试过10多种数据源的测试了，虽然有些有问题，但总归是能调起的，还有jdbc的connector包也要丢进去

luoyongsir commented 5 months ago

请问你的source 是LocalFile，sink是jdbc，并且在Flink上运行吗？我其他场景也用了十来种数据源，几千个任务了，都没问题

ljl568419709 commented 5 months ago

请问你的source 是LocalFile，sink是jdbc，并且在Flink上运行吗？我其他场景也用了十来种数据源，几千个任务了，都没问题你说的那个localFile 我们没测试过，可能有bug，可以加我vx：lijialing8811 后面一起交流

github-actions[bot] commented 3 months ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

apache / seatunnel