Broken example Kafka Topology

[jakekdodd]

I tried running the example Kafka topology with the suggested additions to the topology_dependencies.xml, but the dependencies are out of date.

I was, however, able to make some changes that got RedStorm and the Kafka topology working with storm-0.9.1-incubating and kafka_2.9.2-0.8.1.1. Would you be interested in a fork and pull request to update the suggested topology_dependencies.xml? Also, maybe I could make some changes to the readme to help people get up-and-running with the Kafka topology.

[colinsurprenant]

Yes! I would definitely appreciate this. Thanks in advance for taking the time to contribute this!

[colinsurprenant]

For the record, another user reported some success with these dependencies:

<dependency org="org.scala-lang" name="scala-library" rev="2.9.2" conf="default" transitive="false"/>
<dependency org="org.apache.kafka" name="kafka_2.9.2" rev="0.8.0" conf="default" transitive="false"/>
<dependency org="net.wurstmeister.storm" name="storm-kafka-0.8-plus" rev="0.4.0" conf="default" transitive="false"/>

[jakekdodd]

Awesome, that's almost identical to the dependencies that I added, with the exception of Kafka 0.8.1.1 instead of 0.8.0.

Also, I was getting a java.lang.NoClassDefFoundError: com/yammer/metrics/Metrics error at runtime, so I needed to explicitly add the com.yammer.metrics dependency:

<dependency org="com.yammer.metrics" name="metrics-core" rev="2.2.0"/>

This seems to me like a bug with the Kafka repo, since both 0.8.0 and 0.8.1.1 explicitly depend on com.yammer.metrics (according to here)

I think there was also a minor change that I needed to make to kafka_topology.rb to get it working--I'll have to check.

[colinsurprenant]

Great stuff. thanks again!