linkedin / Burrow

Kafka Consumer Lag Checking
Apache License 2.0
3.76k stars 801 forks source link

Issue with Burrow from 1.6.0 to 1.7.0/1.8.0 #817

Open grigassukys opened 6 months ago

grigassukys commented 6 months ago

Hello,

I've encountered an issue when updating the Burrow package from version 1.6.0 to 1.7.0/1.8.0. The configuration remains exactly the same as it was in version 1.6.0, but starting from version 1.7.0, it no longer works as expected. If I rollback to 1.6.0, everything works the same as before.

Errors I'm getting when updating to 1.7.0/1.8.0:

{"level":"debug","ts":1715166414.740178,"msg":"Error while performing SASL Auth IP:PORT","name":"sarama"} {"level":"debug","ts":1715166414.7402897,"msg":"Closed connection to broker IP:PORT","name":"sarama"} {"level":"debug","ts":1715166414.74035,"msg":"Error while sending ApiVersionsRequest to broker IP:PORT: EOF","name":"sarama"} {"level":"debug","ts":1715166414.740367,"msg":"client/metadata got error from broker -1 while fetching metadata:EOF","name":"sarama"}

Or like this {"level":"error","ts":1715163313.996046,"msg":"failed to start client","type":"module","coordinator":"cluster","class":"kafka","name":"MYCLUSTER","error":"kafka: client has run out of available brokers to talk to: 3 errors occurred:\n\t* EOF\n\t* EOF\n\t* EOF\n"}

Steps to Reproduce:

  1. Update the Burrow package from version 1.6.0 to 1.7.0/1.8.0.
  2. Use the same configuration as in version 1.6.0.

Expected Behavior:

The Burrow package should work as expected with the same configuration as in version 1.6.0.

Actual Behavior:

The Burrow package does not work as expected starting from version 1.7.0.

Environment:

lukasertl commented 6 months ago

Is your kafka-version setting in the client-profile up-to-date? I had similar problems and fixed it by updating this setting.

grigassukys commented 6 months ago

@lukasertl kafka-version is set to 3.5.2 as this is our version of the cluster

nkostoulas commented 5 months ago

Having the same issue when using SASL/SCRAM for authentication.

grigassukys commented 5 months ago

@bai maybe you could take a look at this? thank you!

cdenneen commented 5 months ago

Having same issue... had to revert back to 1.6.0

My 1.6.0 config has a fake tls with unverify=true which has worked for years. I thought this was the issue with the EOF so I actually removed that and went back to plaintext instead. This still rendered the same EOF issue on 1.8.0. So I was able to re-enable the TLS back on 1.6.0 but no idea why tls or plaintext both keep getting these EOF issues on latest burrow.

grigassukys commented 3 weeks ago

Hello, I still haven't figured out the issue.

grigassukys commented 1 week ago

And it's still relevant