search

confluentinc / confluent-kafka-python

Confluent's Kafka Python Client
http://docs.confluent.io/current/clients/confluent-kafka-python
Other
3.72k stars 882 forks source link

AdminClient does not work with logger #1734

Closed pavelschon closed 1 month ago

pavelschon commented 2 months ago

Description

Log messages are not logged via logger.

How to reproduce

import logging
import sys

from confluent_kafka.admin import AdminClient

logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
logger=logging.getLogger("AdminClient")
logger.info("Running...")

client = AdminClient({
    "bootstrap.servers": "localhost:9999",
    "logger": logger
})

client.list_topics(timeout=1)

I would expect to see a log record such as:

%3|1713881891.083|FAIL|rdkafka#producer-1| [thrd:localhost:9999/bootstrap]: localhost:9999/bootstrap: Connect to ipv4#127.0.0.1:9999 failed: Connection refused (after 0ms in state CONNECT)

Python: 3.12.1 Confluent-Kafka version: ('2.3.0', 33751040)

pranavrth commented 2 months ago

Yes there is some issue in logger with AdminClient. Its similar to https://github.com/confluentinc/confluent-kafka-python/issues/1699. Marking it as bug to investigate it further. Hopefully, we will be able to take this up later this quarter.

pranavrth commented 1 month ago

You can use admin_client.poll to serve the logs through the custom logger.

I have updated the other thread where more discussion will continue. Closing this ticket so that all the discussion can happen at one place.