如果sink 是mysql-jdbc 的时候，mysql 连接会存在8小时中断的问题。

lovegq commented 2 weeks ago

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened

com.mysql.cj.exceptions.StatementIsClosedException: No operations allowed after statement closed.

SeaTunnel Version

2.3.8

SeaTunnel Config

从mysql 到mysql

Running Command

通过seaunnl-web 界面启动

Error Exception

Caused by: com.mysql.cj.exceptions.StatementIsClosedException: No operations allowed after statement closed.
    at jdk.internal.reflect.GeneratedConstructorAccessor35.newInstance(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
    at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) ~[?:?]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:480) ~[?:?]
    at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:62) ~[mysql-connector-j-8.0.33.jar:8.0.33]
    at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:86) ~[mysql-connector-j-8.0.33.jar:8.0.33]
    at com.mysql.cj.jdbc.StatementImpl.checkClosed(StatementImpl.java:336) ~[mysql-connector-j-8.0.33.jar:8.0.33]
    at com.mysql.cj.jdbc.ClientPreparedStatement.setLong(ClientPreparedStatement.java:1542) ~[mysql-connector-j-8.0.33.jar:8.0.33]
    at org.apache.seatunnel.shade.com.zaxxer.hikari.pool.HikariProxyPreparedStatement.setLong(HikariProxyPreparedStatement.java) ~[?:?]
    at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.executor.FieldNamedPreparedStatement.setLong(FieldNamedPreparedStatement.java:98) ~[?:?]
    at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.converter.AbstractJdbcRowConverter.toExternal(AbstractJdbcRowConverter.java:217) ~[?:?]
    ... 35 more

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

lovegq commented 2 weeks ago

因为mysql 存在8小时自动杀死连接的问题，而seatunnel 组件提供的jdbc连接池选择是hikaricp，虽然快但是没有探活机制保证，一旦连接失败，任务就执行失败。

wu-a-ge commented 2 weeks ago

有sink_retry解决

Xuzhengz commented 1 week ago

有sink_retry解决

在哪里有说明？请告知一下谢谢

lovegq commented 1 week ago

即便设置 sink 的max-retry 也会出现这个问题 Caused by: com.mysql.cj.exceptions.CJCommunicationsException: The last packet successfully received from the server was 20,539,370 milliseconds ago. The last packet sent successfully to the server was 20,539,379 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem. at jdk.internal.reflect.GeneratedConstructorAccessor36.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:480) ~[?:?] at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:62) ~[mysql-connector-j-8.0.33.jar:8.0.33] at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105) ~[mysql-connector-j-8.0.33.jar:8.0.33] at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:150) ~[mysql-connector-j-8.0.33.jar:8.0.33] at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:166) ~[mysql-connector-j-8.0.33.jar:8.0.33] at com.mysql.cj.protocol.a.NativeProtocol.send(NativeProtocol.java:629) ~[mysql-connector-j-8.0.33.jar:8.0.33] at com.mysql.cj.protocol.a.NativeProtocol.sendCommand(NativeProtocol.java:684) ~[mysql-connector-j-8.0.33.jar:8.0.33] at com.mysql.cj.protocol.a.NativeProtocol.sendCommand(NativeProtocol.java:156) ~[mysql-connector-j-8.0.33.jar:8.0.33] at com.mysql.cj.NativeSession.queryServerVariable(NativeSession.java:589) ~[mysql-connector-j-8.0.33.jar:8.0.33] at com.mysql.cj.jdbc.ConnectionImpl.isReadOnly(ConnectionImpl.java:1366) ~[mysql-connector-j-8.0.33.jar:8.0.33] ... 18 more Caused by: java.net.SocketException: Broken pipe at sun.nio.ch.NioSocketImpl.implWrite(NioSocketImpl.java:425) ~[?:?] at sun.nio.ch.NioSocketImpl.write(NioSocketImpl.java:445) ~[?:?] at sun.nio.ch.NioSocketImpl$2.write(NioSocketImpl.java:831) ~[?:?] at java.net.Socket$SocketOutputStream.write(Socket.java:1035) ~[?:?] at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:81) ~[?:?] at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:142) ~[?:?] at com.mysql.cj.protocol.a.SimplePacketSender.send(SimplePacketSender.java:55) ~[mysql-connector-j-8.0.33.jar:8.0.33] at com.mysql.cj.protocol.a.TimeTrackingPacketSender.send(TimeTrackingPacketSender.java:50) ~[mysql-connector-j-8.0.33.jar:8.0.33] at com.mysql.cj.protocol.a.NativeProtocol.send(NativeProtocol.java:620) ~[mysql-connector-j-8.0.33.jar:8.0.33] at com.mysql.cj.protocol.a.NativeProtocol.sendCommand(NativeProtocol.java:684) ~[mysql-connector-j-8.0.33.jar:8.0.33] at com.mysql.cj.protocol.a.NativeProtocol.sendCommand(NativeProtocol.java:156) ~[mysql-connector-j-8.0.33.jar:8.0.33] at com.mysql.cj.NativeSession.queryServerVariable(NativeSession.java:589) ~[mysql-connector-j-8.0.33.jar:8.0.33] at com.mysql.cj.jdbc.ConnectionImpl.isReadOnly(ConnectionImpl.java:1366) ~[mysql-connector-j-8.0.33.jar:8.0.33] ... 18 more

codethereforam commented 1 week ago

mysql到mysql cdc，同样遇到此问题，设置autoReconnect=true后也没有用，seatunel运行1~2天进程就会挂掉

CJCommunicationsException: The last packet successfully received from the server was 85,426,089 milliseconds ago. The last packet sent successfully to the server was 85,426,093 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.

chenqianwen commented 3 days ago

同样遇到了这个问题

jacklicn commented 2 days ago

add socketTimeout=30000 to jdbc url.

apache / seatunnel