xuxueli / xxl-job

A distributed task scheduling framework.(分布式任务调度平台XXL-JOB)
http://www.xuxueli.com/xxl-job/
GNU General Public License v3.0
27.46k stars 10.86k forks source link

当心跳和调度同一时刻执行,会报com.xxl.rpc.util.XxlRpcException: xxl-rpc, request timeout at #1009

Closed jesse-tang closed 5 years ago

jesse-tang commented 5 years ago

eureka 心跳日志一直有,说明执行器客户端的服务进程一直是存在,而偶尔出现这个心跳超时,正好是发生在任务调度的那一刻,很有可能是当任务和心跳同一时刻同一连接执行的时候,这个心跳请求被终止,同一个连接被任务执行调度覆盖,导致get不到 XxlRpcResponse xxlRpcResponse = futureResponse.get(timeout, TimeUnit.MILLISECONDS);

INFO c.x.r.r.i.r.XxlRpcReferenceBean - >>>>>>>>>>> xxl-rpc, invoke error, address:172.17.30.158:7401, XxlRpcRequestXxlRpcRequest{requestId='908b3f35-e11d-4247-a28d-2956bb0768ad', createMillisTime=1562288400214, accessToken='', className='com.xxl.job.core.biz.ExecutorBiz', methodName='beat', parameterTypes=[], parameters=null, version='null'} 09:00:10.216 xxl-job-admin [xxl-job, admin JobTriggerPoolHelper-fastTriggerPool-1132169857] ERROR c.x.j.a.core.route.ExecutorRouter - xxl-rpc, request timeout at:1562288410214, request:XxlRpcRequest{requestId='908b3f35-e11d-4247-a28d-2956bb0768ad', createMillisTime=1562288400214, accessToken='', className='com.xxl.job.core.biz.ExecutorBiz', methodName='beat', parameterTypes=[], parameters=null, version='null'} com.xxl.rpc.util.XxlRpcException: xxl-rpc, request timeout at:1562288410214, request:XxlRpcRequest{requestId='908b3f35-e11d-4247-a28d-2956bb0768ad', createMillisTime=1562288400214, accessToken='', className='com.xxl.job.core.biz.ExecutorBiz', methodName='beat', parameterTypes=[], parameters=null, version='null'} at com.xxl.rpc.remoting.net.params.XxlRpcFutureResponse.get(XxlRpcFutureResponse.java:117) at com.xxl.rpc.remoting.invoker.reference.XxlRpcReferenceBean$1.invoke(XxlRpcReferenceBean.java:219) at com.sun.proxy.$Proxy81.beat(Unknown Source) at com.xxl.job.admin.core.route.strategy.ExecutorRouteFailover.route(ExecutorRouteFailover.java:26) at com.xxl.job.admin.core.trigger.XxlJobTrigger.processTrigger(XxlJobTrigger.java:137) at com.xxl.job.admin.core.trigger.XxlJobTrigger.trigger(XxlJobTrigger.java:74) at com.xxl.job.admin.core.thread.JobTriggerPoolHelper$3.run(JobTriggerPoolHelper.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

ERROR c.x.r.r.n.i.n.c.NettyHttpClientHandler - >>>>>>>>>>> xxl-rpc netty_http client caught exception java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at sun.nio.ch.IOUtil.read(IOUtil.java:192) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) at io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:288) at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1125) at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:347) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:682) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:617) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:534) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748)收起

xuxueli commented 5 years ago

你好,问题无法发现,从日志 “Connection reset by peer” 来看应该是网络问题,可以参考这篇文章:https://segmentfault.com/q/1010000015858284