search

WallarooLabs / pony-kafka

:horse: Pure Pony Kafka client
Other
57 stars 4 forks source link

Crash during message consumption from Kafka #49

Closed niamster closed 6 years ago

niamster commented 6 years ago

Kafka driver is used as Wallaroo source. At some point it crashes with following error messages:

2018-03-06 18:50:19: ERROR: Kafka Client encountered unrecoverable error! KafkaErrorReport: [ status = Client(ClientErrorDecode/-9001): Error decoding data. error decoding remaining message bytes - /home/vagrant/wallaroo/examples/go/ace/.deps/WallarooLabs/pony-kafka/pony-kafka/kafka_api.pony:805:70, topic = xxx, partition = 5 ]

2018-03-06 18:50:19: ERROR: SHUTTING DOWN CLIENT!
ERROR: Kafka client encountered an unrecoverable error! KafkaErrorReport: [ status = Client(ClientErrorDecode/-9001): Error decoding data. error decoding remaining message bytes - /home/vagrant/wallaroo/examples/go/ace/.deps/WallarooLabs/pony-kafka/pony-kafka/kafka_api.pony:805:70, topic = xxx, partition = 5 ]

This should never happen: failure in /home/vagrant/wallaroo/lib/wallaroo/core/source/kafka_source/kafka_source_listener.pony at line 194

It's a systematic crash, not reproduced with other Kafka drivers.

dipinhora commented 6 years ago

@niamster Thanks for the report.

Can you provide any additional information about what version of Wallaroo you're running and what application you're running (if it is one of the samples)?

Any other information about how to reproduce the error would also be helpful.

niamster commented 6 years ago

@dipinhora thanks for a quick response

I'm using Wallaroo v.0.4.0. I have a custom application that reads from kafka and does some computation without any sink ATM. Unfortunately I can't provide information about application but I would love to help debugging the crash.

niamster commented 6 years ago

@dipinhora can we have a fix in the official repository?

dipinhora commented 6 years ago

Resolved by 0b8c3c1902520d57d96d3a35694dc118f3690071.