linkedin / Burrow

Kafka Consumer Lag Checking
Apache License 2.0
3.76k stars 800 forks source link

client not authorized error #202

Closed cl0udgeek closed 7 years ago

cl0udgeek commented 7 years ago

When I try to start burrow...I see the following in my log output...

2017-04-11 13:37:02 [INFO] Starting Zookeeper client
2017-04-11 13:37:02 [INFO] Starting Offsets Storage module
2017-04-11 13:37:02 [INFO] Starting HTTP server
2017-04-11 13:37:02 [INFO] Starting Zookeeper client for cluster asdp
2017-04-11 13:37:02 [INFO] Starting Kafka client for cluster asdp
2017-04-11 13:37:02 [CRITICAL] Cannot start Kafka client for cluster asdp: kafka server: The client is not authorized to access this topic.

my config file looks like this:

[general]
logdir=/home/ec2-user/burrowlogs
logconfig=/home/ec2-user/log.cfg
pidfile=burrow.pid
client-id=burrow-lagchecker
group-blacklist=^(console-consumer-|python-kafka-consumer-).*$

[clientprofile "asdp"]
tls=true
tls-noverify=true
tls-certfilepath=/home/ec2-user/ssl/client.cert
tls-keyfilepath=/home/ec2-user/ssl/client.key
tls-cafilepath=/home/ec2-user/ssl/ca.pem

[zookeeper]
hostname=kafka1.com
hostname=kafka2.com
hostname=kafka3.com
port=2181
timeout=6
lock-path=/burrow/notifier

[kafka]
broker=kafka1.com
broker=kafka2.com
broker=kafka3.com
broker-port=9050
zookeeper=kafka1.com
zookeeper=kafka2.com
zookeeper=kafka3.com
zookeeper-port=2181
zookeeper-path=/kafkanpi
zookeeper-offsets=false
offsets-topic=__consumer_offsets
client-profile=asdp

[notify]
interval=10

[lagcheck]
intervals=10
expire-group=604800
min-distance=1
zookeeper-interval=60
zk-group-refresh=300

my acls for __consumer_offsets looks like this:

Current ACLs for resource `Topic:__consumer_offsets`: 
     User:burrow-lagchecker has Allow permission for operations: Describe from hosts: *
    User:burrow-lagchecker has Allow permission for operations: Read from hosts: *
    User:ANONYMOUS has Allow permission for operations: Describe from hosts: *
    User:ANONYMOUS has Allow permission for operations: Read from hosts: * 

Do I need to do anything special with ACLs on the brokers>?

wobuxiangtong commented 7 years ago

hello,can you tell me how to generate “tls-certfilepath=/home/ec2-user/ssl/client.cert tls-keyfilepath=/home/ec2-user/ssl/client.key” this two file?

bencoughlan commented 7 years ago

any joy with this? I'm getting the same issues as you.

cl0udgeek commented 7 years ago

yeah...just make sure the DN you use the generate the cert is the DN you use in the User part of the ACL

so if you use something like CN=burrow@company.com,O=Tech,L=something when generating and cert....your ACL would look like this:

kafka-acls --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=zk01.cloud.company.com:2181/kafkacluster --add --allow-principal User:burrow@company.com

then, also make sure you drop the cert and key files into the kafka store too