search

linkedin / Burrow

Kafka Consumer Lag Checking
Apache License 2.0
3.72k stars 796 forks source link

Blacklisting console-consumer* does not seem to work #436

Open xefke opened 6 years ago

xefke commented 6 years ago

Hi,

I have been struggling with this for a while now, and no matter what I try, the console-consumer-* entries stay in the list.

The config is as following:

[consumer.kfkcrp]
class-name="kafka"
cluster="kfkcrp"
servers=[ "kfkcrpa1.uat.corp.telenet.be:9092","kfkcrpb1.uat.corp.telenet.be:9092", "kfkcrpc1.uat.corp.telenet.be:9092" ]
client-profile="burrow"
group-blacklist="^(console-consumer-|python-kafka-consumer-|quick-).*$"
group-whitelist=""

Am I doing something wrong? I also tried a simpler blacklist by just using: group-blacklist="^console-consumer-.*$"

Best regards, Kristof

vvuibert commented 6 years ago

Hello. The group-blacklist does not work for me either with this endpoint /v3/kafka/(cluster)/consumer, but it works with this one /v3/kafka/(cluster)/consumer/(group)

DivsDibs commented 6 years ago

For me, if I try to show the console_consumers (tried disabling the blacklist or omitting the ^) , they don't work. And as per this different thread, they are not run more than 7 days back. So not sure what's wrong.

I am using Burrow 1.1.0 with /v3 API endpoints

xefke commented 6 years ago

@vvuibert, I do not understand your point, as it would be normal that if you specify a group with /v3/kafka/(cluster)/consumer/(group). You would not see all groups, thus you would not see the 'console-consumer-' groups anyway as you chose the details about one group in particular.

dtboctor commented 6 years ago

Hello, I'm seeing something similar to this but not exact

if I start up a console consumer, and then I look at the list of consumers, it shows that console consumer there, ignoring the blacklist. If I then try to look a the details for that consumer, burrow tells me that consumer doesn't exist. THEN, if I look at the list of consumers, it is gone.

vvuibert commented 6 years ago

@dtboctor I had not noticed that, but I just tried it and the same thing happens to me

cnfree0355 commented 5 years ago

I have the same issue as above when use burrow 1.1.0

vvuibert commented 5 years ago

https://github.com/linkedin/Burrow/pull/404 this PR fixes this issue

bai commented 5 years ago

👍https://github.com/linkedin/Burrow/pull/404 was merged a few minutes ago.