KafkaAdminClient acl list not working

[jfinzel]

We are using ccloud kafka, and see a lot of promise in KafkaAdminClient. I recognize this API is in an unstable state, but I have gotten to the point of understanding (I think) what describe_acls is requesting. I would even be willing to expand docs here myself if I could get help with this problem.

I am attempting to show an acl that I clearly see in the ccloud cli. Here is the example in ccloud cli, and the corresponding example in KafkaAdminClient which returns an empty list.

ccloud cli:

$ ccloud kafka acl list | head -n3
  ServiceAccountId | Permission |    Operation     | Resource |           Name            |  Type
+------------------+------------+------------------+----------+---------------------------+---------+
  User:38947       | ALLOW      | READ             | GROUP    | my-group | LITERAL

KafkaAdminClient:

from kafka import KafkaAdminClient
from kafka.admin.acl_resource import ACLOperation, ACLPermissionType, ACLFilter, ACL, ResourcePattern, ResourceType, ACLResourcePatternType, ResourcePatternFilter

client  = KafkaAdminClient(
    bootstrap_servers="path-to-kafka.confluent.cloud:9092",
    security_protocol='SASL_SSL',
    sasl_mechanism='PLAIN',
    sasl_plain_username="<key>",
    sasl_plain_password="<secret>"
)

rpfilter = ResourcePatternFilter(ResourceType.GROUP, 'my-group', ACLResourcePatternType.LITERAL)
filter = ACLFilter('User:38947', '4.5.6.7', ACLOperation.READ, ACLPermissionType.ALLOW, rpfilter)
print(client.describe_acls(filter))

result: ([], <class 'kafka.errors.NoError'>)

As you can see, nothing at all shows up. I have tried other things as well, such as suggested None for principal to remove filter on principal, but with the same result. I also tried another case of a topic permission, again with an empty return value.

Please advise! Thanks!!

[jeffwidman]

Thanks for including the code you're using. At first glance, it appears to me that you're using the code as intended.

Can you enable DEBUG-level logging of kafka-python and paste the output here? That will let us see exactly what kafka-python is sending and then what is being returned from the cluster. Hopefully then we can start to isolate whether the problem is in what is being sent, or in what the cluster is responding with, or in how the response is being processed by kafka-python.

[jfinzel]

@jeffwidman sorry, could you please clarify how I can enable DEBUG logging? Is that a setting in KafkaAdminClient or a different one of the libraries?

[jeffwidman]

This is the gist:

import logging
logger = logging.getLogger("kafka")
logger.addHandler(logging.StreamHandler(sys.stdout))
logger.setLevel(logging.DEBUG)

And may I suggest you invest two hours in the python logging tutorial? https://docs.python.org/3.8/howto/logging.html

Seriously, I waited 4 years into my python career to learn about logging, but when I did finally take the time it was one of the highest leverage things I did...

[AnamikaN]

Below is my code, which works as expected, try this:

creating resource pattern group filter to print acl

            resource_pattern_filter_group = ResourcePatternFilter(resource_type=ResourceType.GROUP,
                                                                  resource_name="*",
                                                                  pattern_type=ACLResourcePatternType.LITERAL)
            acl_filter = ACLFilter(principal=user_name,
                                   host="*",
                                   operation=ACLOperation.ANY,
                                   permission_type=ACLPermissionType.ANY,
                                   resource_pattern=resource_pattern_filter_group)
            print(self.admin_client.describe_acls(acl_filter))

[userakhila]

I am also seeing the same issue, any update on this? I am using same code provided by @AnamikaN ([], <class 'kafka.errors.NoError'>) below are the debug level logs

Sending request DescribeAclsRequest_v1(resource_type=<ResourceType.ANY: 1>, resource_name='*', resource_pattern_type_filter=<ACLResourcePatternType.ANY: 1>, principal='*', host='*', operation=<ACLOperation.ANY: 1>, permission_type=<ACLPermissionType.ANY: 1>)
<BrokerConnection node_id=2 host=host:9094 <connected> [IPv4 ('ip', 9094)]> Request 3: DescribeAclsRequest_v1(resource_type=<ResourceType.ANY: 1>, resource_name='*', resource_pattern_type_filter=<ACLResourcePatternType.ANY: 1>, principal='*', host='*', operation=<ACLOperation.ANY: 1>, permission_type=<ACLPermissionType.ANY: 1>)
Received correlation id: 3
Processing response DescribeAclsResponse_v1
<BrokerConnection node_id=2 host=host:9094 <connected> [IPv4 ('ip', 9094)]> Response 3 (101.5470027923584 ms): DescribeAclsResponse_v1(throttle_time_ms=0, error_code=0, error_message=None, resources=[])
([], <class 'kafka.errors.NoError'>)
Sending request DescribeAclsRequest_v1(resource_type=<ResourceType.ANY: 1>, resource_name='*', resource_pattern_type_filter=<ACLResourcePatternType.ANY: 1>, principal='*', host='*', operation=<ACLOperation.ANY: 1>, permission_type=<ACLPermissionType.ANY: 1>)
<BrokerConnection node_id=2 host=host:9094 <connected> [IPv4 ('ip', 9094)]> Request 4: DescribeAclsRequest_v1(resource_type=<ResourceType.ANY: 1>, resource_name='*', resource_pattern_type_filter=<ACLResourcePatternType.ANY: 1>, principal='*', host='*', operation=<ACLOperation.ANY: 1>, permission_type=<ACLPermissionType.ANY: 1>)
Received correlation id: 4
Processing response DescribeAclsResponse_v1
<BrokerConnection node_id=2 host=host:9094 <connected> [IPv4 ('ip', 9094)]> Response 4 (114.5930290222168 ms): DescribeAclsResponse_v1(throttle_time_ms=0, error_code=0, error_message=None, resources=[])
([], <class 'kafka.errors.NoError'>)

[kebyn]

@jeffwidman I guess the user sasl_plain_username="<key>" does not have permission to list the permissions of User:38947 @userakhila The log shows same problem, ('ip', 9094) and principal='*', Please see here acl_resource, if you want to matching any principal, set principal to None.