search

MeltwaterArchive / dropwizard-extra

A set of miscellaneous and common Dropwizard utilities
109 stars 45 forks source link

Making dropwizard-extra-kafka work with Kafka 0.7.2 #6

Closed mbknor closed 11 years ago

mbknor commented 11 years ago

Hi,

This pullrequest contains some small changes to make dropwizard-extra-kafka work with Kafka 0.7.2 which is the latest stable version here: http://kafka.apache.org/downloads.html

I use scala 2.8.0 since this is the same version as found in Kafka source.

I've deployed kafka to my local repo, by using

./sbt make-pom

and some copying :)

I hope you can pull it or upgrade to 0.7.2 in some other way

nicktelford commented 11 years ago

I'm probably going to remove dropwizard-extra-kafka soon, at least until kafka 0.8 is released. Until they have artifacts in Maven Central, the module reduces the portability of the build.

That said, my main motivation for wanting to remove this module is that I'm not at all happy with it. We've had a number of problems that are attributed to its poor design and it's the single biggest thing holding me back from announcing Dropwizard Extra to a wider audience.

As it stands, I can't merge this as I'm unable to get Kafka's POM installed on my machine and I'm not happy downgrading Scala support to 2.8.0 as I fear it will break the dropwizard-extra-scala module.

mbknor commented 11 years ago

No problem at.

Thanks for sharing your thoughts about the module.

olvesh commented 11 years ago

We (me and @mbknor ) might take a copy of that Kafka code, but it would be interesting if you could shed some light on what problems you have had with the kafka module, and what design-choices are poor?

nicktelford commented 11 years ago

@olvesh (sorry for the slow response!)

I'm actually in two minds about the dropwizard-extra-kafka module. On the one hand, it provides reasonable integration of Kafka's high-level API with Dropwizard. On the other, we've found the underlying high-level Kafka API to be somewhat lacking in the granularity of control it gives us.

I may keep the existing module, but move things around a little and provide an integration for Kafka's low-level API so that implementations have a choice. Though that'll likely prove difficult.

Finally, my biggest gripe is build portability. Depending on Kafka requires that the appropriate JAR is manually installed to the local Maven repo before you can build DW Extra. Until Kafka lands on Maven Central, which I believe is scheduled for 0.8, I'd prefer to keep it out of the build.

If you're using this module (or any of the others!) I'd greatly appreciate any feedback and ideas you have.

olvesh commented 11 years ago

Late answer here. The project I planned using the kafka module has been postponed until now. I will be using kafka, and will see if I can get you that feedback some time :-)