search

alibaba / jstorm

Enterprise Stream Process Engine
http://jstorm.io
Apache License 2.0
3.92k stars 1.8k forks source link

三个节点，提交topology时设置两个worker的时候，其中一个worker报反序列化的错误 #583

Closed dooonabe closed 6 years ago

dooonabe commented 6 years ago
  1. spout从kafka消费数据，设置一个worker的时候，不会有反序列化的错误，但是bolt经常消费一会就失去心跳被kill。设置两个worker的时候，其中一个worker报序列化的错误，但另一个worker的bolt就不会失去心跳，可以一直正常消费。
  2. bolt超时信息 [INFO 2017-10-19 12:56:33 c.a.j.t.a.Acker:130 __acker:2-BoltExecutors] Acker's timeout item size:0 [INFO 2017-10-19 12:56:35 b.s.u.StormBoundedExponentialBackoffRetry:44 pool-13-thread-1] The baseSleepTimeMs [1000] the maxSleepTimeMs [30000] the maxRetries [20] [INFO 2017-10-19 12:56:35 b.s.u.StormBoundedExponentialBackoffRetry:44 pool-13-thread-1] The baseSleepTimeMs [2000] the maxSleepTimeMs [60000] the maxRetries [5] [INFO 2017-10-19 12:57:35 b.s.u.StormBoundedExponentialBackoffRetry:44 pool-13-thread-1] The baseSleepTimeMs [1000] the maxSleepTimeMs [30000] the maxRetries [20] [INFO 2017-10-19 12:57:35 b.s.u.StormBoundedExponentialBackoffRetry:44 pool-13-thread-1] The baseSleepTimeMs [2000] the maxSleepTimeMs [60000] the maxRetries [5] 超时时间设置的大一点会不会有用？ 3.[INFO b.s.s.KryoTupleDeserializer:102 device:6-deserializer] Deserialize error:targetTaskId:6,creationTimeStamp:1508383403005,taskId:8,streamId:6,componentName:kafka,streamName:default,MessageId{-4865592488643089442=5714022020194277048} java.lang.NullPointerException: null at java.io.FileInputStream.(FileInputStream.java:130) ~[na:1.8.0_45] at java.io.FileInputStream.(FileInputStream.java:93) ~[na:1.8.0_45] at com.utils.AvroUtil.getSchema(AvroUtil.java:56) ~[stormjar.jar:na] at com.utils.AvroSerializer.getSchema(AvroSerializer.java:67) ~[stormjar.jar:na] at com.utils.AvroSerializer.read(AvroSerializer.java:52) ~[stormjar.jar:na] at com.utils.AvroSerializer.read(AvroSerializer.java:22) ~[stormjar.jar:na] at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:732) ~[kryo-2.21.jar:na] at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:109) ~[kryo-2.21.jar:na] at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18) ~[kryo-2.21.jar:na] at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:629) ~[kryo-2.21.jar:na] at backtype.storm.serialization.KryoValuesDeserializer.deserializeFrom(KryoValuesDeserializer.java:38) ~[jstorm-core-2.1.1.jar:na] at backtype.storm.serialization.KryoTupleDeserializer.deserialize(KryoTupleDeserializer.java:85) [jstorm-core-2.1.1.jar:na] at backtype.storm.serialization.KryoTupleDeserializer.deserialize(KryoTupleDeserializer.java:65) [jstorm-core-2.1.1.jar:na] at backtype.storm.serialization.KryoTupleDeserializer.deserializeBatch(KryoTupleDeserializer.java:116) [jstorm-core-2.1.1.jar:na] at com.alibaba.jstorm.task.TaskBatchReceiver$DeserializeBatchRunnable.deserialize(TaskBatchReceiver.java:70) [jstorm-core-2.1.1.jar:na] at com.alibaba.jstorm.task.TaskReceiver$DeserializeRunnable.onEvent(TaskReceiver.java:163) [jstorm-core-2.1.1.jar:na] at com.alibaba.jstorm.task.TaskBatchReceiver$DeserializeBatchRunnable.onEvent(TaskBatchReceiver.java:46) [jstorm-core-2.1.1.jar:na] at backtype.storm.utils.DisruptorQueueImpl.consumeBatchToCursor(DisruptorQueueImpl.java:193) [jstorm-core-2.1.1.jar:na] at backtype.storm.utils.DisruptorQueueImpl.consumeBatchWhenAvailable(DisruptorQueueImpl.java:161) [jstorm-core-2.1.1.jar:na] at com.alibaba.jstorm.task.TaskReceiver$DeserializeRunnable.run(TaskReceiver.java:187) [jstorm-core-2.1.1.jar:na] at com.alibaba.jstorm.task.TaskBatchReceiver$DeserializeBatchRunnable.run(TaskBatchReceiver.java:46) [jstorm-core-2.1.1.jar:na] at com.alibaba.jstorm.callback.AsyncLoopRunnable.run(AsyncLoopRunnable.java:95) [jstorm-core-2.1.1.jar:na] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45] [ERROR c.a.j.t.TaskBatchReceiver:78 device:6-deserializer] device:6 recv thread error 00 00 00 21 00 00 00 06 00 00 01 5F 32 A8 2B FD 08 06 01 BC 79 F1 69 1C 9E AB DE 4F 4C 48 两个worker时，其中一个worker一直在报这样的错误 谢谢！