presto-0.100 could not fetch data from Kafka, error code is '6'

[lvxin1986]

Hello,Presto:

I'm using Presto-kafka,But when i executed a query,an error occoured:

presto:default> use kafka_test.default; presto:default> select * from mytopic;

Query 20150416_055844_00057_ngb8v, FAILED, 2 nodes Splits: 1 total, 0 done (0.00%) 0:01 [0 rows, 0B] [0 rows/s, 0B/s]

Query 20150416_055844_00057_ngb8v failed: could not fetch data from Kafka, error code is '6'

the table : mytopic info is:

presto:default> describe mytopic; Column | Type | Null | Partition Key | Comment
-------------------+---------+------+---------------+--------------------------------------------- _partition_id | bigint | true | false | Partition Id
_partition_offset | bigint | true | false | Offset for the message within the partition _segment_start | bigint | true | false | Segment start offset
_segment_end | bigint | true | false | Segment end offset
_segment_count | bigint | true | false | Running message count per segment
_key | varchar | true | false | Key text
_key_corrupt | boolean | true | false | Key data is corrupt
_key_length | bigint | true | false | Total number of key bytes
_message | varchar | true | false | Message text
_message_corrupt | boolean | true | false | Message data is corrupt
_message_length | bigint | true | false | Total number of message bytes
(11 rows)

The when I get the message from the topic using kafka cosumer,It's OK,and the messages are:

[hadp@localhost kafka_2.9.2-0.8.2.1]$ bin/kafka-console-consumer.sh --zookeeper zk1:2181 --from-beginning --topic mytopic "rowNumber":5,"customerKey":5,"name":"Customer#000000005","address":"KvpyuHCplrB84WgAiGV6sYpZq7Tj","nationKey":3,"phone":"13-750-942-6364","accountBalance":794.47,"marketSegment":"HOUSEHOLD","comment":"n accounts will have to unwind. foxes cajole accor" {"rowNumber":1,"customerKey":1,"name":"Customer#000000001","address":"IVhzIApeRb ot,c,E","nationKey":15,"phone":"25-989-741-2988","accountBalance":711.56,"marketSegment":"BUILDING","comment":"to the even, regular platelets. regular, ironic epitaphs nag e"} {"rowNumber":100,"customerKey":100,"name":"Customer#test","address":"IVhzIApeRb ot,c,E","nationKey":15,"phone":"25-989-741-2988","accountBalance":711.56,"marketSegment":"BUILDING","comment":"to the even, regular platelets. regular, ironic epitaphs nag e"} {"rowNumber":100,"customerKey":100,"name":"shilei","address":"IVhzIApeRb ot,c,E","nationKey":15,"phone":"25-989-741-2988","accountBalance":711.56,"marketSegment":"BUILDING","comment":"to the even, regular platelets. regular, ironic epitaphs nag e"}

The exception stack is:

com.facebook.presto.kafka.KafkaRecordSet$KafkaRecordCursor.openFetchRequest(KafkaRecordSet.java:313)", "com.facebook.presto.kafka.KafkaRecordSet$KafkaRecordCursor.advanceNextPosition(KafkaRecordSet.java:162)", "com.facebook.presto.spi.RecordPageSource.getNextPage(RecordPageSource.java:93)", "com.facebook.presto.operator.TableScanOperator.getOutput(TableScanOperator.java:230)", "com.facebook.presto.operator.Driver.processInternal(Driver.java:351)", "com.facebook.presto.operator.Driver.processFor(Driver.java:283)", "com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:563)", "com.facebook.presto.execution.TaskExecutor$PrioritizedSplitRunner.process(TaskExecutor.java:443)", "com.facebook.presto.execution.TaskExecutor$Runner.run(TaskExecutor.java:577)", "java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)", "java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)", "java.lang.Thread.run(Thread.java:745)

but when I executed other queries from other topic,It's OK.

the environment information as follows：

（1）presto-0.100 （2）kafka_2.9.2-0.8.2.1

I don't know why?Can you resolve this problem？

[dain]

@hgschmie any idea?

[hgschmie]

Not really. We saw the error 6 a couple of times. Need to dig into that a bit when I find time.

[dain]

@hgschmie there is an old issue about this #1940 which claimed to have a fix in progress, but it was never submitted.

[danoyoung]

Has any progress been made on this? We're seeing the same issue with presto 0.108.....

presto:prod> select _message from clickstream limit 200;

Query 20150626_152255_00002_ka34h, FAILED, 1 node Splits: 41 total, 24 done (58.54%) 0:02 [0 rows, 0B] [0 rows/s, 0B/s]

Query 20150626_152255_00002_ka34h failed: could not fetch data from Kafka, error code is '6'

[yuananf]

@danoyoung you can try #3000 , we also met this problem and the patch fixed this problem in my case.

[danoyoung]

@yuananf BTW this worked, thank you! Does anyone know if this issue is going to be resolved in 108 or 109?

[yuananf]

@danoyoung @hgschmie is reviewing the patch, but I think he may be busy recently.

[danoyoung]

@hgschmie any update on getting this merged in?

[fabregas4you]

Hi. I got a same error " Kafka, error code is '6' " with presto 0.115.

---

2015-08-24T19:30:40.340+0900 WARN 20150824_103040_00018_n8jzd.1.1-0-57 com.facebook.presto.kafka.KafkaRecordSet Fetch response has error: 6 2015-08-24T19:30:40.354+0900 ERROR SplitRunner-11-57 com.facebook.presto.execution.TaskExecutor Error processing Split 20150824_103040_00018_n8jzd.1.1-0 (start = 1440412240333, wall = 18 ms, cpu = 0 ms, calls = 1): KAFKA_SPLIT_ERROR: could not fetch data from Kafka, error code is '6' 2015-08-24T19:30:40.366+0900 ERROR Stage-20150824_103040_00018_n8jzd.1-112 com.facebook.presto.execution.StageStateMachine Stage 20150824_103040_00018_n8jzd.1 failed com.facebook.presto.spi.PrestoException: could not fetch data from Kafka, error code is '6' at com.facebook.presto.kafka.KafkaRecordSet$KafkaRecordCursor.openFetchRequest(KafkaRecordSet.java:319) at com.facebook.presto.kafka.KafkaRecordSet$KafkaRecordCursor.advanceNextPosition(KafkaRecordSet.java:162) at com.facebook.presto.spi.RecordPageSource.getNextPage(RecordPageSource.java:93) at com.facebook.presto.operator.TableScanOperator.getOutput(TableScanOperator.java:239) at com.facebook.presto.operator.Driver.processInternal(Driver.java:380) at com.facebook.presto.operator.Driver.processFor(Driver.java:303) at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:587) at com.facebook.presto.execution.TaskExecutor$PrioritizedSplitRunner.process(TaskExecutor.java:505) at com.facebook.presto.execution.TaskExecutor$Runner.run(TaskExecutor.java:639) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

any clue ?

[danoyoung]

@fabregas4you yes, we still see this issue in the latest version. @yuananf patch hasn't made it into master yet I believe. For us, this is a pretty important patch. It's nice to be able to use Presto for this as well as querying our Parquet data on S3.

[fabregas4you]

@danoyoung OK. I'll wait and see.

[hgschmie]

I wrote some reply on #3000. I hope to find some time to hack on this but I will make no promises.

[danoyoung]

@hgschmie ok, thank you.

[fabregas4you]

@hgschmie Thank you!

[danoyoung]

@hgschmie This still seems to be broken in 0.118. Still unable to query a Kafka Topic.

[z00]

.124 same problem as well, blocked on it. Do nightly builds have the fix?

[saiprasadmishra]

.126 also has the same issue

[nezihyigitbasi]

A fix has just been merged [79198697cc1847c2144aaee85f99ee8325069d6d] and this issue should be resolved now.