search

dibbhatt / kafka-spark-consumer

High Performance Kafka Connector for Spark Streaming.Supports Multi Topic Fetch, Kafka Security. Reliable offset management in Zookeeper. No Data-loss. No dependency on HDFS and WAL. In-built PID rate controller. Support Message Handler . Offset Lag checker.
Apache License 2.0
635 stars 318 forks source link

No Data reciver Apache kafka apache spark #13

Closed sidahmedbenkhaoua closed 9 years ago

sidahmedbenkhaoua commented 9 years ago

hi i'am using apache kafka and apache spark 1.3 and i test this code def main(args: Array[String]) { val ctx = new SparkContext("local[4]", "Spark Streamin") val kafkaParams = Map("metadata.broker.list" -> "127.0.1:2182") val topics = Set("page_visits") val kafkaStream = KafkaUtils.createDirectStream[String, String, StringDecoder, StringDecoder](ssc, kafkaParams, topics) kafkaStream.foreachRDD(rdd => print()) ssc.start() ssc.awaitTermination() } and my tomic its work when i start java programme too produce and consume data but in streaming no reciver data Help pleas!

dibbhatt commented 9 years ago

I could not understand the exact problem . If you are facing issue with KafkaUtils.createDirectStream please ask the question in Spark user group .

This low level consumer is an alternative Receiver based consumer and have no relation to the API which you are using.

sidahmedbenkhaoua commented 9 years ago

ok thanks for your replay

sidahmedbenkhaoua commented 9 years ago

please telle me where i find kafka.consumer.id ?