3.0 - Cannot get past iterator.hasNext

[ghost]

In 3.0, it appears the process gets locked when calling iterator.hasNext and I can no longer consume messages.

[joekiller]

hasNext is a blocking call so it would appear locked without messages until the consumer.timeout.ms hits (default is -1 so blocking). Did this just happen when you upgraded? Have you changed the number of topic partitions? Got a test to replicate the issue?

What Java are you running? Jruby version, etc. Also check Kafka 0.9.0 issues: https://issues.apache.org/jira/browse/KAFKA/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel

Maybe this? https://issues.apache.org/jira/plugins/servlet/mobile#issue/KAFKA-3172 On Feb 16, 2016 10:40 AM, "geojr" notifications@github.com wrote:

In 3.0, it appears the process gets locked when calling iterator.hasNext and I can no longer consume messages.

— Reply to this email directly or view it on GitHub https://github.com/joekiller/jruby-kafka/issues/46.

[ghost]

Yeah, I definitely did not give you a lot of info to go on. Sorry about that.

So I can confirm that on other kafka clients, messages are flowing so that's not the problem. When I downgrade the gem to 2.0 with the exact same configuration, I get messages and everything works as normal.

Java = OpenJDK 1.8.0_66 / Ubuntu 15.10 Kafka 0.8.20 Is 3.0 supposed to be Kafka 0.9 compatible only?

[joekiller]

3.0 is a 0.9.0 build but it's supposed to be backwards compatible. If you want it to work now, stick with the 2.X release as it is 0.8.2.2 as well.

If you can write a test that creates the problem I could take it from there.

What's your broker running for Java/Kafka? and same questions for the producer. On Feb 16, 2016 1:36 PM, "geojr" notifications@github.com wrote:

Yeah, I definitely did not give you a lot of info to go on. Sorry about that.

So I can confirm that on other kafka clients, messages are flowing so that's not the problem. When I downgrade the gem to 2.0 with the exact same configuration, I get messages and everything works as normal.

Java = OpenJDK 1.8.0_66 / Ubuntu 15.10 Kafka 0.8.20 Is 3.0 supposed to be Kafka 0.9 compatible only?

— Reply to this email directly or view it on GitHub https://github.com/joekiller/jruby-kafka/issues/46#issuecomment-184818441 .

[ghost]

Honestly, it's the sample consumer code in your readme.md that's not working (with the exception of me using a topic that is local to us)

A coworker advised me his consumer was working which is how I figured out that under 2.0 things were working fine but 3.0 it was not.

The broker is running 0.8.1 but with Java 7. Edit: Producer is running the same as the broker as it turns out.

[joekiller]

I'll see if I can replicate but if you can make a fork/PR with an example test that blows up it'll help the most :). I'm guessing it has something to do with Java 7/0.8.1 and the 0.9.0 Java 8 client.

[joekiller]

I tested this and got the same problem. I observed some zookeeper client errors. Will dog deeper. On Feb 16, 2016 11:54 AM, "geojr" notifications@github.com wrote:

Honestly, it's the sample consumer code in your readme.md that's not working (with the exception of me using a topic that is local to us)

A coworker advised me his consumer was working which is how I figured out that under 2.0 things were working fine but 3.0 it was not.

The broker is running 0.8.1 but with Java 7.

On Tue, Feb 16, 2016 at 1:49 PM, Joseph Lawson notifications@github.com wrote:

3.0 is a 0.9.0 build but it's supposed to be backwards compatible. If you want it to work now, stick with the 2.X release as it is 0.8.2.2 as well.

If you can write a test that creates the problem I could take it from there.

What's your broker running for Java/Kafka? and same questions for the producer. On Feb 16, 2016 1:36 PM, "geojr" notifications@github.com wrote:

Yeah, I definitely did not give you a lot of info to go on. Sorry about that.

So I can confirm that on other kafka clients, messages are flowing so that's not the problem. When I downgrade the gem to 2.0 with the exact same configuration, I get messages and everything works as normal.

Java = OpenJDK 1.8.0_66 / Ubuntu 15.10 Kafka 0.8.20 Is 3.0 supposed to be Kafka 0.9 compatible only?

— Reply to this email directly or view it on GitHub <

https://github.com/joekiller/jruby-kafka/issues/46#issuecomment-184818441>

.

— Reply to this email directly or view it on GitHub < https://github.com/joekiller/jruby-kafka/issues/46#issuecomment-184850259> .

J.R. Smith Software Developer

Main 972.546.3878 https://www-staging.geoforce.net/uploads/downloads/esignature.html Fax 972.546.3871 https://www-staging.geoforce.net/uploads/downloads/esignature.html Toll Free 888.574.3878 https://www-staging.geoforce.net/uploads/downloads/esignature.html Geoforce.com http://www.geoforce.com/

Geoforce 750 Canyon Dr. Suite 140 Coppell, Texas 75019

https://www.facebook.com/geoforceinc https://twitter.com/geoforce https://www.linkedin.com/company/geoforce-inc https://instagram.com/geoforce_tracking/ https://oilpro.com/company/152135/geoforce

CONFIDENTIAL - If you are not the intended recipient, please delete this message without disclosing it.

— Reply to this email directly or view it on GitHub https://github.com/joekiller/jruby-kafka/issues/46#issuecomment-184852454 .

[joekiller]

So I was confused. The 0.9.X consumer and producers cannot talk to 0.8 brokers. A 0.9.X broker can be configured to talk 0.8 bit not vice versa. So use the 2.X stuff.

[ghost]

Ahh I was not aware of that either. I was under the impression that it was all backwards compatible. Good to know. Thank you so much for your help and response time. 2.x has been working well in my tests. On Feb 18, 2016 9:50 PM, "Joseph Lawson" notifications@github.com wrote:

So I was confused. The 0.9.X consumer and producers cannot talk to 0.8 brokers. A 0.9.X broker can be configured to talk 0.8 bit not vice versa. So use the 2.X stuff.

— Reply to this email directly or view it on GitHub https://github.com/joekiller/jruby-kafka/issues/46#issuecomment-186040140 .

[joekiller]

Good deal. I just lol'd at most of my responses. They all seem to be victim of autocorrect. On Feb 18, 2016 7:59 PM, "geojr" notifications@github.com wrote:

Ahh I was not aware of that either. I was under the impression that it was all backwards compatible. Good to know. Thank you so much for your help and response time. 2.x has been working well in my tests. On Feb 18, 2016 9:50 PM, "Joseph Lawson" notifications@github.com wrote:

So I was confused. The 0.9.X consumer and producers cannot talk to 0.8 brokers. A 0.9.X broker can be configured to talk 0.8 bit not vice versa. So use the 2.X stuff.

— Reply to this email directly or view it on GitHub < https://github.com/joekiller/jruby-kafka/issues/46#issuecomment-186040140> .

— Reply to this email directly or view it on GitHub https://github.com/joekiller/jruby-kafka/issues/46#issuecomment-186041323 .