Kafka 0.8.2+ producer

[joekiller]

I recently implemented the 0.8.2 producer in jruby-kafka (https://github.com/joekiller/jruby-kafka/issues/24) and it could be useful for this project.

[mheese]

+1

[talevy]

WIP branch with updated producer. Still figuring out how to best integrate this in a backwards compatible way. https://github.com/talevy/logstash-output-kafka/tree/new-producer

[joekiller]

Being that 1.5 is not released yet it could be argued to just make this the default. The producer is noted to be 0.8.X compatible here: http://blog.confluent.io/2014/12/02/whats-coming-in-apache-kafka-0-8-2/ On Apr 27, 2015 6:09 PM, "Tal Levy" notifications@github.com wrote:

WIP branch with updated producer. Still figuring out how to best integrate this in a backwards compatible way. https://github.com/talevy/logstash-output-kafka/tree/new-producer

— Reply to this email directly or view it on GitHub https://github.com/logstash-plugins/logstash-output-kafka/issues/15#issuecomment-96836014 .

[ggrossetie]

Being that 1.5 is not released yet it could be argued to just make this the default.

I agree :+1:

[keithchambers]

:+1:

[allenmchan]

Logstash is already on 1.5.x. Is it currently using the 0.8.2 producer? This issue hasnt been touched in 3 months

[allenmchan]

Nevermind, looks like the new producer was merged in https://github.com/logstash-plugins/logstash-output-kafka/pull/21

This issue should be closed to avoid confusion?

[talevy]

@allenmchan it is in master, but it has not been officially released in rubygems yet for compatibility reasons. That is why this has not yet been closed.

but, since it is in master. I will go ahead and close this. to install, you must build the gem yourself and install it locally.

[allenmchan]

ok now i am confused why you cannot push the update to rubygems. Joekiller's comment said it is backward compatible. Asking everyone who uses this to build our own gem does not make sense

[talevy]

@allenmchan, Joekiller's library is backwards compatible. but the logstash plugin is not. mainly for the reason of simplicity.

[talevy]

@allenmchan we are finalizing how we handle pre-release plugins like beta releases within Logstash, and then we will go ahead and release this. We do not wish for people to run bin/plugin update logstash-output-kafka and have everything break on them without them being aware of the changes.

[suyograo]

@allenmchan this definitely is not backwards compatible with LS 1.x. Most of the config options have changed.

[allenmchan]

@suyograo is there a github issue or pull request that i can track to follow when this is actually available on rubygems?

[suyograo]

@allenmchan #24