xuxueli / xxl-job

A distributed task scheduling framework.(分布式任务调度平台XXL-JOB)
http://www.xuxueli.com/xxl-job/
GNU General Public License v3.0
27.46k stars 10.86k forks source link

集群部署xxl-job-admin后调度失败次数增多,而单机部署xxl-job-admin只有少量调度失败 #697

Closed dongzehong closed 5 years ago

dongzehong commented 5 years ago

Please answer some questions before submitting your issue. Thanks!

Which version of XXL-JOB do you using?

release v2.0.1

Expected behavior

集群部署xxl-job-admin后调度失败次数增多,而单机部署xxl-job-admin只有少量调度失败。期望集群部署xxl-job-admin后也应该和单机部署一样只有少量失败或者不失败。

Actual behavior

集群部署xxl-job-admin后调度失败次数增多。

Steps to reproduce the behavior

xxl-job-admin从单机部署变成集群部署,调度失败增多。

Other information

单机部署xxl-job-admin时,每天只有几笔调度失败。当在两台机器部署xxl-job-admin后,1小时约有10笔调度失败的情况。执行器确实是运行着的,因为下一次调度就成功了。而且单机部署几乎没有调度失败的情况。调度失败时报错信息如下(略去了ip和端口信息): ERROR [pool-3-thread-13] c.x.j.a.core.trigger.XxlJobTrigger [XxlJobTrigger.java : 193] >>>>>>>>>>> xxl-job trigger error, please check if the executor is running. com.xxl.rpc.util.XxlRpcException: java.io.EOFException: HttpConnectionOverHTTP@70d8f44d[HttpChannelOverHTTP@2bf75b18(exchange=HttpExchange@32bb3f3c req=TERMINATED/null@null res=PENDING/null@null)[send=HttpSenderOverHTTP@537f91f(req=QUEUED,snd=COMPLETED,failure=null)[HttpGenerator{s=START}],recv=HttpReceiverOverHTTP@32fa0f68(rsp=IDLE,failure=null)[HttpParser{s=CLOSED,0 of -1}]]] at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.earlyEOF(HttpReceiverOverHTTP.java:277) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1305) at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.shutdown(HttpReceiverOverHTTP.java:182) at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.process(HttpReceiverOverHTTP.java:129) at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.receive(HttpReceiverOverHTTP.java:69) at org.eclipse.jetty.client.http.HttpChannelOverHTTP.receive(HttpChannelOverHTTP.java:90) at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onFillable(HttpConnectionOverHTTP.java:174) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:745)

    at com.xxl.rpc.remoting.invoker.reference.XxlRpcReferenceBean$1.invoke(XxlRpcReferenceBean.java:161)
    at com.sun.proxy.$Proxy80.run(Unknown Source)
    at com.xxl.job.admin.core.trigger.XxlJobTrigger.runExecutor(XxlJobTrigger.java:191)
    at com.xxl.job.admin.core.trigger.XxlJobTrigger.processTrigger(XxlJobTrigger.java:142)
    at com.xxl.job.admin.core.trigger.XxlJobTrigger.trigger(XxlJobTrigger.java:70)
    at com.xxl.job.admin.core.thread.JobTriggerPoolHelper$1.run(JobTriggerPoolHelper.java:35)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
mail6562 commented 5 years ago

我的也遇到这个问题, 任务触发类型:Cron触发 调度机器:192.168.1.2 执行器-注册方式:自动注册 执行器-地址列表:[192.168.1.1:9997, 192.168.1.3:9997] 路由策略:轮询 阻塞处理策略:单机串行 任务超时时间:0 失败重试次数:0

触发调度<<<<<<<<<<< 触发调度: address:192.168.1.1:9997 code:500 msg:com.xxl.rpc.util.XxlRpcException: java.io.EOFException: HttpConnectionOverHTTP@3ba86311::SocketChannelEndPoint@3f7eb234{/192.168.1.1:9997<->/192.168.1.2:45117,ISHUT,fill=-,flush=-,to=56/0}{io=0/0,kio=0,kro=1}->HttpConnectionOverHTTP@3ba86311(l:/192.168.1.2:45117 <-> r:/192.168.1.1:9997,closed=false)=>HttpChannelOverHTTP@22510f27(exchange=HttpExchange@519a280d req=TERMINATED/null@null res=PENDING/null@null)[send=HttpSenderOverHTTP@2d8d1e5c(req=QUEUED,snd=COMPLETED,failure=null)[HttpGenerator@67a8e6cb{s=START}],recv=HttpReceiverOverHTTP@4daae52e(rsp=IDLE,failure=null)[HttpParser{s=CLOSED,0 of -1}]] at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.earlyEOF(HttpReceiverOverHTTP.java:338) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1551) at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.shutdown(HttpReceiverOverHTTP.java:209) at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.process(HttpReceiverOverHTTP.java:147) at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.receive(HttpReceiverOverHTTP.java:73) at org.eclipse.jetty.client.http.HttpChannelOverHTTP.receive(HttpChannelOverHTTP.java:133) at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onFillable(HttpConnectionOverHTTP.java:155) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:745) at com.xxl.rpc.remoting.invoker.reference.XxlRpcReferenceBean$1.invoke(XxlRpcReferenceBean.java:161) at com.sun.proxy.$Proxy78.run(Unknown Source) at com.xxl.job.admin.core.trigger.XxlJobTrigger.runExecutor(XxlJobTrigger.java:192) at com.xxl.job.admin.core.trigger.XxlJobTrigger.processTrigger(XxlJobTrigger.java:145) at com.xxl.job.admin.core.trigger.XxlJobTrigger.trigger(XxlJobTrigger.java:75) at com.xxl.job.admin.core.thread.JobTriggerPoolHelper$1.run(JobTriggerPoolHelper.java:35) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)

mail6562 commented 5 years ago

楼主的问题解决了吗?

dongzehong commented 5 years ago

楼主的问题解决了吗?

我后来使用v1.9.2版本,没有报这个错了。这个问题的原因还未知。

xuxueli commented 5 years ago

合并至:https://github.com/xuxueli/xxl-job/issues/688