we have setup all the tunning params for kafka and strom still getting very high latency

miguno / kafka-storm-starter

[PROJECT IS NO LONGER MAINTAINED] Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

http://www.michael-noll.com/blog/2014/05/27/kafka-storm-integration-example-tutorial/

Other

725 stars 330 forks source link

we have setup all the tunning params for kafka and strom still getting very high latency #7

Closed imvipul closed 9 years ago

imvipul commented 9 years ago

we have 3 node kafka cluster....3 node zookeeper and 1 node storm cluster with 8 CPU per node....there is enough memory and disk space available...we are using kafkaspout ...for testing we have a topology with two spout reading message from two topics and sending it to bolt which is outputting it to kafka topic....we are facing very high latency

miguno commented 9 years ago

@imvipul I am afraid that support for Storm/Kafka is out of scope for this code project. :-(