ACL not working in Kcraft mode, error: Topic authorization failed

innubies commented 12 months ago

Name and Version

bitnami/kafka:3.5

What architecture are you using?

None

What steps will reproduce the bug?

Kraft cluster, deploy via ansible
Config

kafka_docker:
  user: root
  name: kafka
  image: bitnami/kafka:3.5
  publish:
    - 9092:9092
    - 9093:9093
    - 9094:9094
  env:
    - SERVICE_NAME=kafka
    - BITNAMI_DEBUG=yes
    - KAFKA_ENABLE_KRAFT=yes
    - KAFKA_CFG_NUM_PARTITIONS=3
    - KAFKA_CFG_NODE_ID={{kafka_node_id}}
    - KAFKA_CFG_PROCESS_ROLES=broker,controller
    - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
    - KAFKA_CFG_LISTENERS=BROKER://:9092,CONTROLLER://:9093,EXTERNAL://:9094
    - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=BROKER:SASL_SSL,CONTROLLER:SASL_SSL,EXTERNAL:SASL_PLAINTEXT
    - KAFKA_CFG_ADVERTISED_LISTENERS=BROKER://{{ansible_fqdn}}:9092,EXTERNAL://{{ansible_fqdn}}:9094
    - KAFKA_BROKER_ID={{kafka_node_id}}
    - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka-1:9093,1@kafka-2:9093,2@kafka-3:9093
    - KAFKA_KRAFT_CLUSTER_ID=Ytavvxgsx1xxashc
    - KAFKA_CFG_SECURITY_PROTOCOL=SASL_SSL
    - KAFKA_CFG_SSL_KEYSTORE_LOCATION=/opt/bitnami/kafka/config/certs/kafka.keystore.jks
    - KAFKA_CFG_SSL_TRUSTSTORE_LOCATION=/opt/bitnami/kafka/config/certs/kafka.truststore.jks
    - KAFKA_CFG_SSL_TRUSTSTORE_PASSWORD=strongpassword
    - KAFKA_CFG_SSL_KEYSTORE_PASSWORD=strongpassword
    - KAFKA_CFG_SSL_KEY_PASSWORD=strongpassword
    - KAFKA_HEAP_OPTS='-Xmx4G -Xms4G'
    - KAFKA_CLIENT_USERS=client_user
    - KAFKA_CLIENT_PASSWORDS=client_password
    - KAFKA_CFG_SASL_MECHANISM_CONTROLLER_PROTOCOL=PLAIN
    - KAFKA_CFG_SASL_MECHANISM_INTER_BROKER_PROTOCOL=PLAIN
    - KAFKA_INTER_BROKER_LISTENER_NAME=BROKER
    - KAFKA_INTER_BROKER_USER=broker_user
    - KAFKA_INTER_BROKER_PASSWORD=broker_password
    - KAFKA_CONTROLLER_USER=controller_user
    - KAFKA_CONTROLLER_PASSWORD=controller_password
    - KAFKA_CFG_SUPER_USERS='User:controller_user;User:broker_user;'
    - KAFKA_CFG_AUTHORIZER_CLASS_NAME=org.apache.kafka.metadata.authorizer.StandardAuthorizer
    - KAFKA_CFG_EARLY_START_LISTENERS=CONTROLLER
  volume:
    - /opt/docker/kafka/data:/bitnami/kafka/data
    - /opt/docker/kafka/logs:/opt/bitnami/kafka/logs
    - /opt/docker/kafka/ssl/kafka.keystore.jks:/opt/bitnami/kafka/config/certs/kafka.keystore.jks:ro
    - /opt/docker/kafka/ssl/kafka.truststore.jks:/opt/bitnami/kafka/config/certs/kafka.truststore.jks:ro

Run stack and create acl:

kafka-acls.sh --bootstrap-server kafka-1:9092 --command-config /tmp/test2.conf --add --allow-principal USER:client_user --operation all --topic test

Try to read topic and see error:

kcat -b kafka-1:9094  -X security.protocol=SASL_PLAINTEXT -X sasl.mechanisms=PLAIN   -X sasl.username=client_user -X sasl.password='client_password' -C -t test -o 0 -e

% ERROR: Topic test error: Broker: Topic authorization failed

kafka's log:

INFO Principal = User:client_user is Denied operation = DESCRIBE from host = localhost on resource = Topic:LITERAL:test for request = Metadata with resourceRefCount = 1 based on rule DefaultDeny (kafka.authorizer.logger)

What is the expected behavior?

Topic will be able for client_user.

What do you see instead?

I expect to see something like this:

kcat -b kafka-1:9094  -X security.protocol=SASL_PLAINTEXT -X sasl.mechanisms=PLAIN   -X sasl.username=client_user -X sasl.password='client_password' -C -t test -o 0 -e

% Reached end of topic test [2] at offset 0
% Reached end of topic test [1] at offset 0
% Reached end of topic test [0] at offset 0: exiting

Additional information

I've tried to add KAFKA_CFG_ALLOW_EVERYONE_IF_NO_ACL_FOUND = true but it's not recommended for production clusters. Also, I've tried to add KAFKA_CFG_SUPER_USERS: "User:ANONYMOUS" but it doesn't work for me.

carrodher commented 12 months ago

The issue may not be directly related to the Bitnami container image or Helm chart, but rather to how the application is being utilized or configured in your specific environment.

Having said that, if you think that's not the case and are interested in contributing a solution, we welcome you to create a pull request. The Bitnami team is excited to review your submission and offer feedback. You can find the contributing guidelines here.

Your contribution will greatly benefit the community. Feel free to reach out if you have any questions or need assistance.

If you have any questions about the application itself, customizing its content, or questions about technology and infrastructure usage, we highly recommend that you refer to the forums and user guides provided by the project responsible for the application or technology.

With that said, we'll keep this ticket open until the stale bot automatically closes it, in case someone from the community contributes valuable insights.

innubies commented 11 months ago

Hello! The problem was in command of topic's creation. The principal must be User, not USER. After I've fixed command, ACL worked fine. I think we can close the issue. Thanks.

bitnami / containers