MyCATApache / Mycat-Server

GNU General Public License v2.0
9.52k stars 3.85k forks source link

java连接mycat Communications link failure #2461

Open muoubumu opened 4 years ago

muoubumu commented 4 years ago

环境描述: 1.java批量执行sql,sql中包含insert,update,并且对同一个表操作 2.使用java1.8->mycat1.6(默认配置无优化,配置4个处理器)->mysql(主从) 3.项目原因,java驱动为mysql-connector-5.18,用高版本无法连接上mycat 4.直连mysql跑批速度慢,连mycat速度非常快,但是经常出现link failure 5.请大神直解,有经验的麻烦帮下忙

java错误堆栈: ERROR com.dtroad.ieasweb.ieasweb.rule.po.RuleExecutor.processSqlQueue(RuleExecutor.java:416) - 批量sql执行,异常 org.springframework.dao.DataAccessResourceFailureException: StatementCallback; Communications link failure

The last packet successfully received from the server was 120 milliseconds ago. The last packet sent successfully to the server was 230 milliseconds ago.; nested exception is java.sql.BatchUpdateException: Communications link failure

The last packet successfully received from the server was 120 milliseconds ago. The last packet sent successfully to the server was 230 milliseconds ago. at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:107) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81) at org.springframework.jdbc.core.JdbcTemplate.translateException(JdbcTemplate.java:1444) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:388) at com.dtroad.ieasweb.core.support.InternalService.batchExecuteSql(InternalService.java:69) at com.dtroad.ieasweb.core.support.InternalService$$FastClassBySpringCGLIB$$82047ce3.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at com.baomidou.dynamic.datasource.aop.DynamicDataSourceAnnotationInterceptor.invoke(DynamicDataSourceAnnotationInterceptor.java:52) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) at com.dtroad.ieasweb.core.support.InternalService$$EnhancerBySpringCGLIB$$d49db11e.batchExecuteSql() at com.dtroad.ieasweb.ieasweb.rule.po.RuleExecutor.processSqlQueue(RuleExecutor.java:413) at com.dtroad.ieasweb.ieasweb.rule.po.RuleExecutor.executeRules(RuleExecutor.java:130) at com.dtroad.ieasweb.ieasweb.rule.po.RuleExecutor.execute(RuleExecutor.java:83) at com.dtroad.ieasweb.ieasweb.rule.po.RuleExecutor$$FastClassBySpringCGLIB$$a3d1808d.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at com.baomidou.dynamic.datasource.aop.DynamicDataSourceAnnotationInterceptor.invoke(DynamicDataSourceAnnotationInterceptor.java:52) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) at com.dtroad.ieasweb.ieasweb.rule.po.RuleExecutor$$EnhancerBySpringCGLIB$$f88c07bc.execute() at com.dtroad.ieasweb.config.profile.TestController.ruleExecutor_execute(TestController.java:180) at com.Test2.ieasweb(Test2.java:94) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.springframework.test.context.junit4.statements.RunBeforeTestExecutionCallbacks.evaluate(RunBeforeTestExecutionCallbacks.java:74) at org.springframework.test.context.junit4.statements.RunAfterTestExecutionCallbacks.evaluate(RunAfterTestExecutionCallbacks.java:84) at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75) at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86) at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:251) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61) at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:190) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) Caused by: java.sql.BatchUpdateException: Communications link failure

The last packet successfully received from the server was 120 milliseconds ago. The last packet sent successfully to the server was 230 milliseconds ago. at com.mysql.jdbc.StatementImpl.executeBatchUsingMultiQueries(StatementImpl.java:1316) at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:1084) at com.alibaba.druid.pool.DruidPooledStatement.executeBatch(DruidPooledStatement.java:778) at com.dtroad.ieasweb.core.support.InternalService$1.doInStatement(InternalService.java:76) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:376) ... 52 common frames omitted INFO com.dtroad.ieasweb.core.support.InternalService$1.doInStatement(InternalService.java:77) - 批量执行sql耗时=165,数量=5000 ERROR com.alibaba.druid.pool.DruidDataSource.handleFatalError(DruidDataSource.java:1602) - discard connection java.sql.BatchUpdateException: Communications link failure

The last packet successfully received from the server was 27 milliseconds ago. The last packet sent successfully to the server was 26 milliseconds ago. at com.mysql.jdbc.StatementImpl.executeBatchUsingMultiQueries(StatementImpl.java:1316) at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:1084) at com.alibaba.druid.pool.DruidPooledStatement.executeBatch(DruidPooledStatement.java:778) at com.dtroad.ieasweb.core.support.InternalService$1.doInStatement(InternalService.java:76) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:376) at com.dtroad.ieasweb.core.support.InternalService.batchExecuteSql(InternalService.java:69) at com.dtroad.ieasweb.core.support.InternalService$$FastClassBySpringCGLIB$$82047ce3.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at com.baomidou.dynamic.datasource.aop.DynamicDataSourceAnnotationInterceptor.invoke(DynamicDataSourceAnnotationInterceptor.java:52) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) at com.dtroad.ieasweb.core.support.InternalService$$EnhancerBySpringCGLIB$$d49db11e.batchExecuteSql() at com.dtroad.ieasweb.ieasweb.rule.po.RuleExecutor.processSqlQueue(RuleExecutor.java:413) at com.dtroad.ieasweb.ieasweb.rule.po.RuleExecutor.executeRules(RuleExecutor.java:130) at com.dtroad.ieasweb.ieasweb.rule.po.RuleExecutor.execute(RuleExecutor.java:83) at com.dtroad.ieasweb.ieasweb.rule.po.RuleExecutor$$FastClassBySpringCGLIB$$a3d1808d.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at com.baomidou.dynamic.datasource.aop.DynamicDataSourceAnnotationInterceptor.invoke(DynamicDataSourceAnnotationInterceptor.java:52) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) at com.dtroad.ieasweb.ieasweb.rule.po.RuleExecutor$$EnhancerBySpringCGLIB$$f88c07bc.execute() at com.dtroad.ieasweb.config.profile.TestController.ruleExecutor_execute(TestController.java:180) at com.Test2.ieasweb(Test2.java:94) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.springframework.test.context.junit4.statements.RunBeforeTestExecutionCallbacks.evaluate(RunBeforeTestExecutionCallbacks.java:74) at org.springframework.test.context.junit4.statements.RunAfterTestExecutionCallbacks.evaluate(RunAfterTestExecutionCallbacks.java:84) at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75) at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86) at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:251) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61) at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:190) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

mycat错误堆栈: 2020-04-15 16:25:37.618 WARN [$_NIOREACTOR-2-RW] (io.mycat.server.NonBlockingSession.execute(NonBlockingSession.java:135)) - ServerConnection [id=11495, schema=XXX, host=XXX, user=XXX,txIsolation=3, autocommit=true, schema=XXX], route={ 1 -> dn16{insert into XXX 这里隐藏,批量insert操作} } java.lang.NullPointerException at io.mycat.net.AbstractConnection.checkWriteBuffer(AbstractConnection.java:456) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.mysql.ErrorPacket.write(ErrorPacket.java:118) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.mysql.ErrorPacket.write(ErrorPacket.java:136) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.backend.mysql.nio.handler.SingleNodeHandler.backConnectionErr(SingleNodeHandler.java:255) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.backend.mysql.nio.handler.SingleNodeHandler.executeException(SingleNodeHandler.java:216) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.backend.mysql.nio.handler.SingleNodeHandler._execute(SingleNodeHandler.java:205) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.backend.mysql.nio.handler.SingleNodeHandler.connectionAcquired(SingleNodeHandler.java:190) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.backend.datasource.PhysicalDatasource.takeCon(PhysicalDatasource.java:367) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.backend.datasource.PhysicalDatasource.getConnection(PhysicalDatasource.java:404) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.backend.datasource.PhysicalDBNode.getConnection(PhysicalDBNode.java:134) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.backend.mysql.nio.handler.SingleNodeHandler.execute(SingleNodeHandler.java:182) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.server.NonBlockingSession.execute(NonBlockingSession.java:132) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.server.ServerConnection.routeEndExecuteSQL(ServerConnection.java:288) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.server.ServerConnection.execute(ServerConnection.java:222) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.server.ServerQueryHandler.query(ServerQueryHandler.java:136) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.FrontendConnection.query(FrontendConnection.java:317) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.FrontendConnection.query(FrontendConnection.java:337) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.handler.FrontendCommandHandler.handle(FrontendCommandHandler.java:71) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.FrontendConnection.rawHandle(FrontendConnection.java:478) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.FrontendConnection.handle(FrontendConnection.java:460) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.AbstractConnection.onReadData(AbstractConnection.java:321) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.NIOSocketWR.asynRead(NIOSocketWR.java:190) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.AbstractConnection.asynRead(AbstractConnection.java:273) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.NIOReactor$RW.run(NIOReactor.java:102) ~[Mycat-server-1.6-RELEASE.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.7.0_79] 2020-04-15 16:25:37.618 WARN [$_NIOREACTOR-2-RW] (io.mycat.net.NIOReactor$RW.run(NIOReactor.java:107)) - caught err: java.lang.NullPointerException at io.mycat.net.AbstractConnection.checkWriteBuffer(AbstractConnection.java:456) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.mysql.ErrorPacket.write(ErrorPacket.java:118) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.mysql.ErrorPacket.write(ErrorPacket.java:136) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.FrontendConnection.writeErrMessage(FrontendConnection.java:207) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.FrontendConnection.writeErrMessage(FrontendConnection.java:199) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.server.NonBlockingSession.execute(NonBlockingSession.java:136) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.server.ServerConnection.routeEndExecuteSQL(ServerConnection.java:288) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.server.ServerConnection.execute(ServerConnection.java:222) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.server.ServerQueryHandler.query(ServerQueryHandler.java:136) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.FrontendConnection.query(FrontendConnection.java:317) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.FrontendConnection.query(FrontendConnection.java:337) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.handler.FrontendCommandHandler.handle(FrontendCommandHandler.java:71) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.FrontendConnection.rawHandle(FrontendConnection.java:478) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.FrontendConnection.handle(FrontendConnection.java:460) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.AbstractConnection.onReadData(AbstractConnection.java:321) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.NIOSocketWR.asynRead(NIOSocketWR.java:190) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.AbstractConnection.asynRead(AbstractConnection.java:273) ~[Mycat-server-1.6-RELEASE.jar:?] at io.mycat.net.NIOReactor$RW.run(NIOReactor.java:102) ~[Mycat-server-1.6-RELEASE.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.7.0_79] 2020-04-15 16:25:37.618 INFO [$_NIOREACTOR-2-RW] (io.mycat.net.AbstractConnection.close(AbstractConnection.java:508)) - close connection,reason:program err:java.lang.NullPointerException ,ServerConnection [id=11495, schema=ieasweb, host=11.18.15.238, user=eco,txIsolation=3, autocommit=true, schema=ieasweb]

junwen12221 commented 4 years ago

mycat不使用批处理,一条一条插入

muoubumu commented 4 years ago

mycat不使用批处理,一条一条插入

这样的话就失去跑批功能了,说不过去

funnyAnt commented 4 years ago

@muoubumu 1.6 支持批量insert的,不支持批量update,delete 。驱动改成5.1.35,高于这个版本就不行了。