alibaba / jstorm

Enterprise Stream Process Engine
http://jstorm.io
Apache License 2.0
3.91k stars 1.81k forks source link

jstorm 任务卡住 #551

Closed dengbo closed 7 years ago

dengbo commented 7 years ago

整个任务处于卡住状态,spout和bolt的emit都变成0,日志里出现大量的: Task-3 is dead on 10.8.162.90:8014, 20170719145520, at 2017-07-19 14:55 KafkaSpout:352-CtrlQueue is full , at 2017-07-20 10:22 Failed to publish timer event to KafkaSpout:383_taskHeartbeat shade.storm.com.lmax.disruptor.InsufficientCapacityException: null Report error to /ZK/taskerrors/tlm_mengxiangrui_appmonitor.test-1259-1500437756/383: No response from Task-383, last report time(sec) is 1500446190 设置里TOPOLOGY_MAX_SPOUT_PENDING是3000,topology.disruptor.wait.timeout是 1,从output的时延看,很低,不到10ms,任务刚开始的时候也是正常的,跑着跑着就卡住了,麻烦帮忙看看 @longdafeng

qianshanding commented 7 years ago

我也碰到同样的问题,只能重启才能解决

dengbo commented 7 years ago

553