Open Narzhan opened 1 year ago
Can you please the debug logs? It can be enabled by setting "debug": "all"
in the config.
@Narzhan
Following up on this issue. Is it still an issue you are facing? If you could please provide debug logs as @pranavrth mentioned earlier, it can help us investigate later.
If this issue is no longer needed, please advise we can mark it closed
@nhaq-confluent I had slightly similar issue. I had a sync code which produces hardcoded event to kafka with that config:
Producer(
{
"bootstrap.servers": config.bootstrap_servers,
# "debug": "all",
"security.protocol": config.security_protocol,
"ssl.ca.location": config.ssl_ca_location,
"ssl.key.location": config.ssl_key_location,
"ssl.certificate.location": config.ssl_certificate_location,
"enable.ssl.certificate.verification": False,
"api.version.request": False,
"message.timeout.ms": config.message_timeout,
"default.topic.config": {
"topic.metadata.refresh.interval.ms": config.topic_metadata_refresh_interval,
}
}
)
Kafka is under SSL without security checks. I tried to produce event like that:
# Trigger available delivery callbacks from previous produce() calls
self.producer.poll(timeout=0)
self.producer.produce(
topic=topic, value=value,
on_delivery=lambda err, msg: self._callback(err, msg, {})
)
# callback on_delivery (use print() to be sure that logging settings doesn't skip printing by level)
def _callback(self, err: Any, *args: Any, **kwargs: Any) -> None:
if err is not None:
err_string = str(err).replace('"', '\\"')
print(f"KAFKA: Message with context {args} was not sent: {err_string}")
else:
print(f"KAFKA: Message with context {args} was sent successfully")
And I got no errors, _callback
was not triggered, and no data was posted to kafka.
What I checked:
producer.list_topics()
, I got it - so, connection to Kafka established well;confluentinc/cp-kafka:6.0.14
Docker, tried to produce event there with PLAINTEXT
- everything posted, so problem is not in the code;Nothing helped - kafka is available, code works, but data didn't appear there.
Everything seemed as fixed, after I had set debug: all
in config as @pranavrth advised. I got the following logs and event appeared in topic (see logs at the end). Also what is strange, data was thransfered successfully, but my prints KAFKA: Message...
are not in logs.
When I disabled debug and tried again, event still was produced sucessfully. I have no idea what changed, and why it started to work, but maybe it gives you some thoughts.
$ python manage.py test-kafka
%7|1708588257.013|OPENSSL|rdkafka#producer-1| [thrd:app]: Using statically linked OpenSSL version OpenSSL 3.0.8 7 Feb 2023 (0x30000080, librdkafka built with 0x30000080)
%7|1708588257.016|CACERTS|rdkafka#producer-1| [thrd:app]: Setting default CA certificate location to /etc/ssl/certs/ca-certificates.crt, override with ssl.ca.location
%7|1708588257.075|WAKEUPFD|rdkafka#producer-1| [thrd:app]: ssl://some-kafka-address-1:9094/bootstrap: Enabled low-latency ops queue wake-ups
%7|1708588257.075|BRKMAIN|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Enter main broker thread
%7|1708588257.075|BROKER|rdkafka#producer-1| [thrd:app]: ssl://some-kafka-address-1:9094/bootstrap: Added new broker with NodeId -1
%7|1708588257.075|WAKEUPFD|rdkafka#producer-1| [thrd:app]: ssl://some-kafka-address-2:9094/bootstrap: Enabled low-latency ops queue wake-ups
%7|1708588257.075|BRKMAIN|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enter main broker thread
%7|1708588257.075|BROKER|rdkafka#producer-1| [thrd:app]: ssl://some-kafka-address-2:9094/bootstrap: Added new broker with NodeId -1
%7|1708588257.075|WAKEUPFD|rdkafka#producer-1| [thrd:app]: ssl://some-kafka-address-3:9094/bootstrap: Enabled low-latency ops queue wake-ups
%7|1708588257.075|BROKER|rdkafka#producer-1| [thrd:app]: ssl://some-kafka-address-3:9094/bootstrap: Added new broker with NodeId -1
%7|1708588257.075|BRKMAIN|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Enter main broker thread
%7|1708588257.075|CONNECT|rdkafka#producer-1| [thrd:app]: ssl://some-kafka-address-1:9094/bootstrap: Selected for cluster connection: bootstrap servers added (broker has 0 connection attempt(s))
%7|1708588257.075|INIT|rdkafka#producer-1| [thrd:app]: librdkafka v2.2.0 (0x20200ff) rdkafka#producer-1 initialized (builtin.features gzip,snappy,ssl,sasl,regex,lz4,sasl_plain,sasl_scram,plugins,zstd,sasl_oauthbearer,http,oidc, STRIP STATIC_LINKING GCC GXX PKGCONFIG INSTALL GNULD LIBDL PLUGINS ZLIB SSL ZSTD CURL HDRHISTOGRAM SYSLOG SNAPPY SOCKEM SASL_SCRAM SASL_OAUTHBEARER OAUTHBEARER_OIDC CRC32C_HW, debug 0xfffff)
%7|1708588257.075|BRKMAIN|rdkafka#producer-1| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/bootstrap: Enter main broker thread
%7|1708588257.075|CONF|rdkafka#producer-1| [thrd:app]: Client configuration:
%7|1708588257.076|CONF|rdkafka#producer-1| [thrd:app]: client.software.name = confluent-kafka-python
%7|1708588257.075|CONNECT|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Received CONNECT op
%7|1708588257.076|STATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Broker changed state INIT -> TRY_CONNECT
%7|1708588257.076|BROADCAST|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: Broadcasting state change
%7|1708588257.076|CONNECT|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: broker in state TRY_CONNECT connecting
%7|1708588257.076|STATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Broker changed state TRY_CONNECT -> CONNECT
%7|1708588257.076|BROADCAST|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: Broadcasting state change
%7|1708588257.076|CONF|rdkafka#producer-1| [thrd:app]: client.software.version = 2.2.0-rdkafka-2.2.0
%7|1708588257.076|CONF|rdkafka#producer-1| [thrd:app]: metadata.broker.list = some-kafka-address-1:9094,some-kafka-address-2:9094,some-kafka-address-3:9094
%7|1708588257.076|CONF|rdkafka#producer-1| [thrd:app]: topic.metadata.refresh.interval.ms = 20000
%7|1708588257.076|CONF|rdkafka#producer-1| [thrd:app]: debug = generic,broker,topic,metadata,feature,queue,msg,protocol,cgrp,security,fetch,interceptor,plugin,consumer,admin,eos,mock,assignor,conf,all
%7|1708588257.076|CONF|rdkafka#producer-1| [thrd:app]: error_cb = 0x7f9c3cf30670
%7|1708588257.076|CONF|rdkafka#producer-1| [thrd:app]: opaque = 0x7f9c146ad620
%7|1708588257.076|CONF|rdkafka#producer-1| [thrd:app]: default_topic_conf = 0x557af9edb7d0
%7|1708588257.076|CONF|rdkafka#producer-1| [thrd:app]: api.version.request = true
%7|1708588257.076|CONF|rdkafka#producer-1| [thrd:app]: security.protocol = ssl
%7|1708588257.076|CONF|rdkafka#producer-1| [thrd:app]: ssl.key.location = [redacted]
%7|1708588257.076|CONF|rdkafka#producer-1| [thrd:app]: enable.ssl.certificate.verification = false
%7|1708588257.076|CONF|rdkafka#producer-1| [thrd:app]: dr_msg_cb = 0x7f9c3cf2e0d0
%7|1708588257.076|CONF|rdkafka#producer-1| [thrd:app]: Default topic configuration:
%7|1708588257.076|CONF|rdkafka#producer-1| [thrd:app]: message.timeout.ms = 60000
%7|1708588257.205|CONNECT|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Connecting to ipv4#10.200.4.72:9094 (ssl) with socket 12
%7|1708588257.282|CONNECT|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Connected to ipv4#10.200.4.72:9094
%7|1708588257.282|STATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Broker changed state CONNECT -> SSL_HANDSHAKE
%7|1708588257.282|BROADCAST|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: Broadcasting state change
%7|1708588257.282|ENDPOINT|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabled endpoint identification using hostname some-kafka-address-1
{"asctime": "2024-02-22 11:50:57", "levelname": "WARNING", "name": "app_log", "service_name": "ads_core_backend", "process": 106846, "threadName": "MainThread", "user_id": "-", "request_id": "-", "message": "", "event": {"message": "Mocked client used to sync with ads-banner"}, "logger": "app_log", "level": "warning"}
{"asctime": "2024-02-22 11:50:57", "levelname": "WARNING", "name": "app_log", "service_name": "ads_core_backend", "process": 106846, "threadName": "MainThread", "user_id": "-", "request_id": "-", "message": "", "event": {"message": "Mocked client used to sync with users-microservice"}, "logger": "app_log", "level": "warning"}
{"asctime": "2024-02-22 11:50:57", "levelname": "WARNING", "name": "app_log", "service_name": "ads_core_backend", "process": 106846, "threadName": "MainThread", "user_id": "-", "request_id": "-", "message": "", "event": {"message": "Mocked client used to send ads to moderation"}, "logger": "app_log", "level": "warning"}
{"asctime": "2024-02-22 11:50:57", "levelname": "WARNING", "name": "app_log", "service_name": "ads_core_backend", "process": 106846, "threadName": "MainThread", "user_id": "-", "request_id": "-", "message": "", "event": {"message": "Mocked client used to sync with ads-authgate"}, "logger": "app_log", "level": "warning"}
{"asctime": "2024-02-22 11:50:57", "levelname": "WARNING", "name": "app_log", "service_name": "ads_core_backend", "process": 106846, "threadName": "MainThread", "user_id": "-", "request_id": "-", "message": "", "event": {"message": "Mocked client used to sync with ads-billing"}, "logger": "app_log", "level": "warning"}
{"asctime": "2024-02-22 11:50:57", "levelname": "DEBUG", "name": "root", "service_name": "ads_core_backend", "process": 106846, "threadName": "MainThread", "user_id": null, "request_id": "-", "message": "Using manage.py cli"}
>> Send to kafka at 2024-02-22 07:50:57.705857+00:00
%7|1708588257.710|OPENSSL|rdkafka#producer-2| [thrd:app]: Using statically linked OpenSSL version OpenSSL 3.0.8 7 Feb 2023 (0x30000080, librdkafka built with 0x30000080)
%7|1708588257.711|CACERTS|rdkafka#producer-2| [thrd:app]: Setting default CA certificate location to /etc/ssl/certs/ca-certificates.crt, override with ssl.ca.location
%7|1708588257.752|WAKEUPFD|rdkafka#producer-2| [thrd:app]: ssl://some-kafka-address-1:9094/bootstrap: Enabled low-latency ops queue wake-ups
%7|1708588257.752|BROKER|rdkafka#producer-2| [thrd:app]: ssl://some-kafka-address-1:9094/bootstrap: Added new broker with NodeId -1
%7|1708588257.752|WAKEUPFD|rdkafka#producer-2| [thrd:app]: ssl://some-kafka-address-2:9094/bootstrap: Enabled low-latency ops queue wake-ups
%7|1708588257.752|BROKER|rdkafka#producer-2| [thrd:app]: ssl://some-kafka-address-2:9094/bootstrap: Added new broker with NodeId -1
%7|1708588257.753|WAKEUPFD|rdkafka#producer-2| [thrd:app]: ssl://some-kafka-address-3:9094/bootstrap: Enabled low-latency ops queue wake-ups
%7|1708588257.753|BROKER|rdkafka#producer-2| [thrd:app]: ssl://some-kafka-address-3:9094/bootstrap: Added new broker with NodeId -1
%7|1708588257.753|CONNECT|rdkafka#producer-2| [thrd:app]: ssl://some-kafka-address-1:9094/bootstrap: Selected for cluster connection: bootstrap servers added (broker has 0 connection attempt(s))
%7|1708588257.753|BRKMAIN|rdkafka#producer-2| [thrd::0/internal]: :0/internal: Enter main broker thread
%7|1708588257.753|BRKMAIN|rdkafka#producer-2| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Enter main broker thread
%7|1708588257.753|BRKMAIN|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enter main broker thread
%7|1708588257.753|CONNECT|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Received CONNECT op
%7|1708588257.753|STATE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Broker changed state INIT -> TRY_CONNECT
%7|1708588257.753|BROADCAST|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: Broadcasting state change
%7|1708588257.753|CONNECT|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: broker in state TRY_CONNECT connecting
%7|1708588257.753|STATE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Broker changed state TRY_CONNECT -> CONNECT
%7|1708588257.753|BROADCAST|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: Broadcasting state change
%7|1708588257.753|BRKMAIN|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/bootstrap: Enter main broker thread
%7|1708588257.753|INIT|rdkafka#producer-2| [thrd:app]: librdkafka v2.2.0 (0x20200ff) rdkafka#producer-2 initialized (builtin.features gzip,snappy,ssl,sasl,regex,lz4,sasl_plain,sasl_scram,plugins,zstd,sasl_oauthbearer,http,oidc, STRIP STATIC_LINKING GCC GXX PKGCONFIG INSTALL GNULD LIBDL PLUGINS ZLIB SSL ZSTD CURL HDRHISTOGRAM SYSLOG SNAPPY SOCKEM SASL_SCRAM SASL_OAUTHBEARER OAUTHBEARER_OIDC CRC32C_HW, debug 0xfffff)
%7|1708588257.753|CONF|rdkafka#producer-2| [thrd:app]: Client configuration:
%7|1708588257.753|CONF|rdkafka#producer-2| [thrd:app]: client.software.name = confluent-kafka-python
%7|1708588257.753|CONF|rdkafka#producer-2| [thrd:app]: client.software.version = 2.2.0-rdkafka-2.2.0
%7|1708588257.753|CONF|rdkafka#producer-2| [thrd:app]: metadata.broker.list = some-kafka-address-1:9094,some-kafka-address-2:9094,some-kafka-address-3:9094
%7|1708588257.753|CONF|rdkafka#producer-2| [thrd:app]: topic.metadata.refresh.interval.ms = 20000
%7|1708588257.753|CONF|rdkafka#producer-2| [thrd:app]: debug = generic,broker,topic,metadata,feature,queue,msg,protocol,cgrp,security,fetch,interceptor,plugin,consumer,admin,eos,mock,assignor,conf,all
%7|1708588257.753|CONF|rdkafka#producer-2| [thrd:app]: error_cb = 0x7f9c3cf30670
%7|1708588257.753|CONF|rdkafka#producer-2| [thrd:app]: opaque = 0x7f9c11a287c0
%7|1708588257.753|CONF|rdkafka#producer-2| [thrd:app]: default_topic_conf = 0x557af84c5210
%7|1708588257.753|CONF|rdkafka#producer-2| [thrd:app]: api.version.request = true
%7|1708588257.753|CONF|rdkafka#producer-2| [thrd:app]: security.protocol = ssl
%7|1708588257.753|CONF|rdkafka#producer-2| [thrd:app]: ssl.key.location = [redacted]
%7|1708588257.753|CONF|rdkafka#producer-2| [thrd:app]: enable.ssl.certificate.verification = false
%7|1708588257.753|CONF|rdkafka#producer-2| [thrd:app]: dr_msg_cb = 0x7f9c3cf2e0d0
%7|1708588257.753|CONF|rdkafka#producer-2| [thrd:app]: Default topic configuration:
%7|1708588257.753|CONF|rdkafka#producer-2| [thrd:app]: message.timeout.ms = 60000
>>>> produce my__topic b'{"author_id":"00000000-0000-0000-0000-000000000000","campaign_id":"00000000-0000-0000-0000-000000000000","ad_group_id":"00000000-0000-0000-0000-000000000000","ad_id":"00000000-0000-0000-0000-000000000000","avideo_id":"external_media_id","objective":"reach","bid_for_one_action":0.001,"campaign_limit":0.0,"campaign_limit_type":"no","updated_at":"2024-02-22T07:50:57.705857"}'
%7|1708588257.754|TOPIC|rdkafka#producer-2| [thrd:app]: New local topic: my__topic
%7|1708588257.754|TOPPARNEW|rdkafka#producer-2| [thrd:app]: NEW my__topic [-1] 0x557afa2620f0 refcnt 0x557afa262180 (at rd_kafka_topic_new0:472)
%7|1708588257.754|CONF|rdkafka#producer-2| [thrd:app]: Topic "my__topic" configuration (default_topic_conf):
%7|1708588257.754|CONF|rdkafka#producer-2| [thrd:app]: message.timeout.ms = 60000
>> FINISHED 2024-02-22 07:50:57.705857+00:00
%7|1708588257.755|CONNECT|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Connecting to ipv4#10.200.4.72:9094 (ssl) with socket 21
%7|1708588257.830|CONNECT|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Connected to ipv4#10.200.4.72:9094
%7|1708588257.830|STATE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Broker changed state CONNECT -> SSL_HANDSHAKE
%7|1708588257.830|BROADCAST|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: Broadcasting state change
%7|1708588257.830|ENDPOINT|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabled endpoint identification using hostname some-kafka-address-1
%7|1708588258.075|CONNECT|rdkafka#producer-1| [thrd:main]: ssl://some-kafka-address-2:9094/bootstrap: Selected for cluster connection: no cluster connection (broker has 0 connection attempt(s))
%7|1708588258.075|CONNECT|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Received CONNECT op
%7|1708588258.075|STATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Broker changed state INIT -> TRY_CONNECT
%7|1708588258.075|BROADCAST|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: Broadcasting state change
%7|1708588258.075|CONNECT|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: broker in state TRY_CONNECT connecting
%7|1708588258.075|STATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Broker changed state TRY_CONNECT -> CONNECT
%7|1708588258.075|BROADCAST|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: Broadcasting state change
%7|1708588258.152|CONNECT|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Connecting to ipv4#10.200.5.29:9094 (ssl) with socket 22
%7|1708588258.226|CONNECT|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Connected to ipv4#10.200.5.29:9094
%7|1708588258.227|STATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Broker changed state CONNECT -> SSL_HANDSHAKE
%7|1708588258.227|BROADCAST|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: Broadcasting state change
%7|1708588258.227|ENDPOINT|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Enabled endpoint identification using hostname some-kafka-address-2
%7|1708588258.388|CONNECTED|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Connected (#1)
%7|1708588258.388|FEATURE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Updated enabled protocol features +ApiVersion to ApiVersion
%7|1708588258.388|STATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Broker changed state SSL_HANDSHAKE -> APIVERSION_QUERY
%7|1708588258.388|BROADCAST|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: Broadcasting state change
%7|1708588258.389|CONNECTED|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Connected (#1)
%7|1708588258.389|FEATURE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Updated enabled protocol features +ApiVersion to ApiVersion
%7|1708588258.389|STATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Broker changed state SSL_HANDSHAKE -> APIVERSION_QUERY
%7|1708588258.389|BROADCAST|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: Broadcasting state change
%7|1708588258.389|SEND|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Sent ApiVersionRequest (v3, 66 bytes @ 0, CorrId 1)
%7|1708588258.389|SEND|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Sent ApiVersionRequest (v3, 66 bytes @ 0, CorrId 1)
%7|1708588258.390|CONNECTED|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Connected (#1)
%7|1708588258.390|FEATURE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Updated enabled protocol features +ApiVersion to ApiVersion
%7|1708588258.390|STATE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Broker changed state SSL_HANDSHAKE -> APIVERSION_QUERY
%7|1708588258.390|BROADCAST|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: Broadcasting state change
%7|1708588258.390|SEND|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Sent ApiVersionRequest (v3, 66 bytes @ 0, CorrId 1)
%7|1708588258.459|RECV|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Received ApiVersionResponse (v3, 6 bytes, CorrId 1, rtt 70.25ms)
%7|1708588258.459|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiVersionRequest v3 failed due to UNSUPPORTED_VERSION: retrying with v0
%7|1708588258.459|SEND|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Sent ApiVersionRequest (v0, 21 bytes @ 0, CorrId 2)
%7|1708588258.462|RECV|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Received ApiVersionResponse (v3, 6 bytes, CorrId 1, rtt 72.72ms)
%7|1708588258.462|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiVersionRequest v3 failed due to UNSUPPORTED_VERSION: retrying with v0
%7|1708588258.462|SEND|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Sent ApiVersionRequest (v0, 21 bytes @ 0, CorrId 2)
%7|1708588258.520|RECV|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Received ApiVersionResponse (v3, 6 bytes, CorrId 1, rtt 130.43ms)
%7|1708588258.520|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiVersionRequest v3 failed due to UNSUPPORTED_VERSION: retrying with v0
%7|1708588258.521|SEND|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Sent ApiVersionRequest (v0, 21 bytes @ 0, CorrId 2)
%7|1708588258.529|RECV|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Received ApiVersionResponse (v0, 270 bytes, CorrId 2, rtt 70.19ms)
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Broker API support:
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey Produce (0) Versions 0..7
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey Fetch (1) Versions 0..10
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey ListOffsets (2) Versions 0..5
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey Metadata (3) Versions 0..7
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey LeaderAndIsr (4) Versions 0..2
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey StopReplica (5) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey UpdateMetadata (6) Versions 0..5
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey ControlledShutdown (7) Versions 0..2
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey OffsetCommit (8) Versions 0..6
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey OffsetFetch (9) Versions 0..5
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey FindCoordinator (10) Versions 0..2
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey JoinGroup (11) Versions 0..4
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey Heartbeat (12) Versions 0..2
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey LeaveGroup (13) Versions 0..2
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey SyncGroup (14) Versions 0..2
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey DescribeGroups (15) Versions 0..2
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey ListGroups (16) Versions 0..2
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey SaslHandshake (17) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey ApiVersion (18) Versions 0..2
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey CreateTopics (19) Versions 0..3
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey DeleteTopics (20) Versions 0..3
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey DeleteRecords (21) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey InitProducerId (22) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey OffsetForLeaderEpoch (23) Versions 0..2
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey AddPartitionsToTxn (24) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey AddOffsetsToTxn (25) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey EndTxn (26) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey WriteTxnMarkers (27) Versions 0..0
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey TxnOffsetCommit (28) Versions 0..2
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey DescribeAcls (29) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey CreateAcls (30) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey DeleteAcls (31) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey DescribeConfigs (32) Versions 0..2
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey AlterConfigs (33) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey AlterReplicaLogDirs (34) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey DescribeLogDirs (35) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey SaslAuthenticate (36) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey CreatePartitions (37) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey CreateDelegationToken (38) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey RenewDelegationToken (39) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey ExpireDelegationToken (40) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey DescribeDelegationToken (41) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey DeleteGroups (42) Versions 0..1
%7|1708588258.530|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey ElectLeadersRequest (43) Versions 0..0
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature MsgVer1: Produce (2..2) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature MsgVer1: Fetch (2..2) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature MsgVer1
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature MsgVer2: Produce (3..3) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature MsgVer2: Fetch (4..4) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature MsgVer2
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature ApiVersion: ApiVersion (0..0) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature ApiVersion
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature BrokerGroupCoordinator: FindCoordinator (0..0) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature BrokerGroupCoordinator
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature BrokerBalancedConsumer: FindCoordinator (0..0) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature BrokerBalancedConsumer: OffsetCommit (1..2) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature BrokerBalancedConsumer: OffsetFetch (1..1) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature BrokerBalancedConsumer: JoinGroup (0..0) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature BrokerBalancedConsumer: SyncGroup (0..0) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature BrokerBalancedConsumer: Heartbeat (0..0) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature BrokerBalancedConsumer: LeaveGroup (0..0) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature BrokerBalancedConsumer
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature ThrottleTime: Produce (1..2) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature ThrottleTime: Fetch (1..2) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature ThrottleTime
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature Sasl: JoinGroup (0..0) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature Sasl
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature SaslHandshake: SaslHandshake (0..0) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature SaslHandshake
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature LZ4: FindCoordinator (0..0) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature LZ4
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature OffsetTime: ListOffsets (1..1) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature OffsetTime
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature IdempotentProducer: InitProducerId (0..0) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature IdempotentProducer
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature ZSTD: Produce (7..7) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature ZSTD: Fetch (10..10) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature ZSTD
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature SaslAuthReq: SaslHandshake (1..1) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature SaslAuthReq: SaslAuthenticate (0..1) supported by broker
%7|1708588258.531|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature SaslAuthReq
%7|1708588258.531|FEATURE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Updated enabled protocol features to MsgVer1,ApiVersion,BrokerBalancedConsumer,ThrottleTime,Sasl,SaslHandshake,BrokerGroupCoordinator,LZ4,OffsetTime,MsgVer2,IdempotentProducer,ZSTD,SaslAuthReq
%7|1708588258.531|STATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Broker changed state APIVERSION_QUERY -> UP
%7|1708588258.531|BROADCAST|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: Broadcasting state change
%7|1708588258.532|METADATA|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Request metadata for brokers only: connected
%7|1708588258.532|SEND|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Sent MetadataRequest (v7, 26 bytes @ 0, CorrId 3)
%7|1708588258.533|RECV|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Received ApiVersionResponse (v0, 270 bytes, CorrId 2, rtt 70.85ms)
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Broker API support:
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey Produce (0) Versions 0..7
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey Fetch (1) Versions 0..10
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey ListOffsets (2) Versions 0..5
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey Metadata (3) Versions 0..7
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey LeaderAndIsr (4) Versions 0..2
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey StopReplica (5) Versions 0..1
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey UpdateMetadata (6) Versions 0..5
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey ControlledShutdown (7) Versions 0..2
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey OffsetCommit (8) Versions 0..6
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey OffsetFetch (9) Versions 0..5
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey FindCoordinator (10) Versions 0..2
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey JoinGroup (11) Versions 0..4
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey Heartbeat (12) Versions 0..2
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey LeaveGroup (13) Versions 0..2
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey SyncGroup (14) Versions 0..2
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey DescribeGroups (15) Versions 0..2
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey ListGroups (16) Versions 0..2
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey SaslHandshake (17) Versions 0..1
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey ApiVersion (18) Versions 0..2
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey CreateTopics (19) Versions 0..3
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey DeleteTopics (20) Versions 0..3
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey DeleteRecords (21) Versions 0..1
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey InitProducerId (22) Versions 0..1
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey OffsetForLeaderEpoch (23) Versions 0..2
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey AddPartitionsToTxn (24) Versions 0..1
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey AddOffsetsToTxn (25) Versions 0..1
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey EndTxn (26) Versions 0..1
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey WriteTxnMarkers (27) Versions 0..0
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey TxnOffsetCommit (28) Versions 0..2
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey DescribeAcls (29) Versions 0..1
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey CreateAcls (30) Versions 0..1
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey DeleteAcls (31) Versions 0..1
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey DescribeConfigs (32) Versions 0..2
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey AlterConfigs (33) Versions 0..1
%7|1708588258.533|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey AlterReplicaLogDirs (34) Versions 0..1
%7|1708588258.534|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey DescribeLogDirs (35) Versions 0..1
%7|1708588258.534|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey SaslAuthenticate (36) Versions 0..1
%7|1708588258.534|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey CreatePartitions (37) Versions 0..1
%7|1708588258.534|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey CreateDelegationToken (38) Versions 0..1
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey RenewDelegationToken (39) Versions 0..1
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey ExpireDelegationToken (40) Versions 0..1
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey DescribeDelegationToken (41) Versions 0..1
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey DeleteGroups (42) Versions 0..1
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: ApiKey ElectLeadersRequest (43) Versions 0..0
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature MsgVer1: Produce (2..2) supported by broker
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature MsgVer1: Fetch (2..2) supported by broker
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Enabling feature MsgVer1
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature MsgVer2: Produce (3..3) supported by broker
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature MsgVer2: Fetch (4..4) supported by broker
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Enabling feature MsgVer2
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature ApiVersion: ApiVersion (0..0) supported by broker
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Enabling feature ApiVersion
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature BrokerGroupCoordinator: FindCoordinator (0..0) supported by broker
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Enabling feature BrokerGroupCoordinator
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature BrokerBalancedConsumer: FindCoordinator (0..0) supported by broker
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature BrokerBalancedConsumer: OffsetCommit (1..2) supported by broker
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature BrokerBalancedConsumer: OffsetFetch (1..1) supported by broker
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature BrokerBalancedConsumer: JoinGroup (0..0) supported by broker
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature BrokerBalancedConsumer: SyncGroup (0..0) supported by broker
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature BrokerBalancedConsumer: Heartbeat (0..0) supported by broker
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature BrokerBalancedConsumer: LeaveGroup (0..0) supported by broker
%7|1708588258.537|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Enabling feature BrokerBalancedConsumer
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature ThrottleTime: Produce (1..2) supported by broker
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature ThrottleTime: Fetch (1..2) supported by broker
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Enabling feature ThrottleTime
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature Sasl: JoinGroup (0..0) supported by broker
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Enabling feature Sasl
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature SaslHandshake: SaslHandshake (0..0) supported by broker
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Enabling feature SaslHandshake
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature LZ4: FindCoordinator (0..0) supported by broker
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Enabling feature LZ4
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature OffsetTime: ListOffsets (1..1) supported by broker
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Enabling feature OffsetTime
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature IdempotentProducer: InitProducerId (0..0) supported by broker
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Enabling feature IdempotentProducer
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature ZSTD: Produce (7..7) supported by broker
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature ZSTD: Fetch (10..10) supported by broker
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Enabling feature ZSTD
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature SaslAuthReq: SaslHandshake (1..1) supported by broker
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Feature SaslAuthReq: SaslAuthenticate (0..1) supported by broker
%7|1708588258.538|APIVERSION|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Enabling feature SaslAuthReq
%7|1708588258.538|FEATURE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Updated enabled protocol features to MsgVer1,ApiVersion,BrokerBalancedConsumer,ThrottleTime,Sasl,SaslHandshake,BrokerGroupCoordinator,LZ4,OffsetTime,MsgVer2,IdempotentProducer,ZSTD,SaslAuthReq
%7|1708588258.538|STATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Broker changed state APIVERSION_QUERY -> UP
%7|1708588258.538|BROADCAST|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: Broadcasting state change
%7|1708588258.538|METADATA|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Request metadata for brokers only: connected
%7|1708588258.538|METADATA|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: Skipping metadata request: connected: full request already in-transit
%7|1708588258.589|RECV|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Received ApiVersionResponse (v0, 270 bytes, CorrId 2, rtt 68.61ms)
%7|1708588258.589|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Broker API support:
%7|1708588258.589|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey Produce (0) Versions 0..7
%7|1708588258.589|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey Fetch (1) Versions 0..10
%7|1708588258.589|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey ListOffsets (2) Versions 0..5
%7|1708588258.589|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey Metadata (3) Versions 0..7
%7|1708588258.589|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey LeaderAndIsr (4) Versions 0..2
%7|1708588258.589|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey StopReplica (5) Versions 0..1
%7|1708588258.589|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey UpdateMetadata (6) Versions 0..5
%7|1708588258.589|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey ControlledShutdown (7) Versions 0..2
%7|1708588258.589|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey OffsetCommit (8) Versions 0..6
%7|1708588258.589|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey OffsetFetch (9) Versions 0..5
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey FindCoordinator (10) Versions 0..2
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey JoinGroup (11) Versions 0..4
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey Heartbeat (12) Versions 0..2
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey LeaveGroup (13) Versions 0..2
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey SyncGroup (14) Versions 0..2
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey DescribeGroups (15) Versions 0..2
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey ListGroups (16) Versions 0..2
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey SaslHandshake (17) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey ApiVersion (18) Versions 0..2
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey CreateTopics (19) Versions 0..3
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey DeleteTopics (20) Versions 0..3
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey DeleteRecords (21) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey InitProducerId (22) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey OffsetForLeaderEpoch (23) Versions 0..2
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey AddPartitionsToTxn (24) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey AddOffsetsToTxn (25) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey EndTxn (26) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey WriteTxnMarkers (27) Versions 0..0
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey TxnOffsetCommit (28) Versions 0..2
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey DescribeAcls (29) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey CreateAcls (30) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey DeleteAcls (31) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey DescribeConfigs (32) Versions 0..2
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey AlterConfigs (33) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey AlterReplicaLogDirs (34) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey DescribeLogDirs (35) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey SaslAuthenticate (36) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey CreatePartitions (37) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey CreateDelegationToken (38) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey RenewDelegationToken (39) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey ExpireDelegationToken (40) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey DescribeDelegationToken (41) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey DeleteGroups (42) Versions 0..1
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: ApiKey ElectLeadersRequest (43) Versions 0..0
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature MsgVer1: Produce (2..2) supported by broker
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature MsgVer1: Fetch (2..2) supported by broker
%7|1708588258.590|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature MsgVer1
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature MsgVer2: Produce (3..3) supported by broker
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature MsgVer2: Fetch (4..4) supported by broker
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature MsgVer2
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature ApiVersion: ApiVersion (0..0) supported by broker
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature ApiVersion
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature BrokerGroupCoordinator: FindCoordinator (0..0) supported by broker
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature BrokerGroupCoordinator
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature BrokerBalancedConsumer: FindCoordinator (0..0) supported by broker
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature BrokerBalancedConsumer: OffsetCommit (1..2) supported by broker
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature BrokerBalancedConsumer: OffsetFetch (1..1) supported by broker
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature BrokerBalancedConsumer: JoinGroup (0..0) supported by broker
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature BrokerBalancedConsumer: SyncGroup (0..0) supported by broker
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature BrokerBalancedConsumer: Heartbeat (0..0) supported by broker
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature BrokerBalancedConsumer: LeaveGroup (0..0) supported by broker
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature BrokerBalancedConsumer
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature ThrottleTime: Produce (1..2) supported by broker
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature ThrottleTime: Fetch (1..2) supported by broker
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature ThrottleTime
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature Sasl: JoinGroup (0..0) supported by broker
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature Sasl
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature SaslHandshake: SaslHandshake (0..0) supported by broker
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature SaslHandshake
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature LZ4: FindCoordinator (0..0) supported by broker
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature LZ4
%7|1708588258.591|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature OffsetTime: ListOffsets (1..1) supported by broker
%7|1708588258.592|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature OffsetTime
%7|1708588258.592|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature IdempotentProducer: InitProducerId (0..0) supported by broker
%7|1708588258.592|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature IdempotentProducer
%7|1708588258.592|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature ZSTD: Produce (7..7) supported by broker
%7|1708588258.592|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature ZSTD: Fetch (10..10) supported by broker
%7|1708588258.592|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature ZSTD
%7|1708588258.592|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature SaslAuthReq: SaslHandshake (1..1) supported by broker
%7|1708588258.592|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Feature SaslAuthReq: SaslAuthenticate (0..1) supported by broker
%7|1708588258.592|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Enabling feature SaslAuthReq
%7|1708588258.592|FEATURE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Updated enabled protocol features to MsgVer1,ApiVersion,BrokerBalancedConsumer,ThrottleTime,Sasl,SaslHandshake,BrokerGroupCoordinator,LZ4,OffsetTime,MsgVer2,IdempotentProducer,ZSTD,SaslAuthReq
%7|1708588258.592|STATE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Broker changed state APIVERSION_QUERY -> UP
%7|1708588258.592|BROADCAST|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: Broadcasting state change
%7|1708588258.592|METADATA|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: Hinted cache of 1/1 topic(s) being queried
%7|1708588258.592|METADATA|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: Requesting metadata for 1/1 topics: connected
%7|1708588258.592|METADATA|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Request metadata for 1 topic(s): connected
%7|1708588258.592|SEND|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Sent MetadataRequest (v7, 58 bytes @ 0, CorrId 3)
%7|1708588258.602|RECV|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Received MetadataResponse (v7, 268 bytes, CorrId 3, rtt 70.21ms)
%7|1708588258.602|METADATA|rdkafka#producer-1| [thrd:main]: ssl://some-kafka-address-1:9094/bootstrap: ===== Received metadata: connected =====
%7|1708588258.602|METADATA|rdkafka#producer-1| [thrd:main]: ssl://some-kafka-address-1:9094/bootstrap: ClusterId: ZhFlyXjaT5y3tVXhF3ck-w, ControllerId: 2
%7|1708588258.602|METADATA|rdkafka#producer-1| [thrd:main]: ssl://some-kafka-address-1:9094/bootstrap: 3 brokers, 0 topics
%7|1708588258.602|METADATA|rdkafka#producer-1| [thrd:main]: ssl://some-kafka-address-1:9094/bootstrap: Broker #0/3: some-kafka-address-2:9094 NodeId 2
%7|1708588258.602|UPDATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: NodeId changed from -1 to 2
%7|1708588258.602|UPDATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/2: Name changed from ssl://some-kafka-address-2:9094/bootstrap to ssl://some-kafka-address-2:9094/2
%7|1708588258.602|LEADER|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/2: Mapped 0 partition(s) to broker
%7|1708588258.602|STATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/2: Broker changed state UP -> UPDATE
%7|1708588258.602|BROADCAST|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: Broadcasting state change
%7|1708588258.602|BROADCAST|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: Broadcasting state change
%7|1708588258.602|STATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/2: Broker changed state UPDATE -> UP
%7|1708588258.602|BROADCAST|rdkafka#producer-1| [thrd:ssl://some-kafka-address-2:]: Broadcasting state change
%7|1708588258.602|METADATA|rdkafka#producer-1| [thrd:main]: ssl://some-kafka-address-1:9094/bootstrap: Broker #1/3: some-kafka-address-3:9094 NodeId 3
%7|1708588258.602|UPDATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/bootstrap: NodeId changed from -1 to 3
%7|1708588258.603|UPDATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Name changed from ssl://some-kafka-address-3:9094/bootstrap to ssl://some-kafka-address-3:9094/3
%7|1708588258.603|LEADER|rdkafka#producer-1| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Mapped 0 partition(s) to broker
%7|1708588258.603|BROADCAST|rdkafka#producer-1| [thrd:ssl://some-kafka-address-3:]: Broadcasting state change
%7|1708588258.603|METADATA|rdkafka#producer-1| [thrd:main]: ssl://some-kafka-address-1:9094/bootstrap: Broker #2/3: some-kafka-address-1:9094 NodeId 1
%7|1708588258.603|UPDATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: NodeId changed from -1 to 1
%7|1708588258.603|UPDATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/1: Name changed from ssl://some-kafka-address-1:9094/bootstrap to ssl://some-kafka-address-1:9094/1
%7|1708588258.603|LEADER|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/1: Mapped 0 partition(s) to broker
%7|1708588258.603|STATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/1: Broker changed state UP -> UPDATE
%7|1708588258.603|BROADCAST|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: Broadcasting state change
%7|1708588258.603|BROADCAST|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: Broadcasting state change
%7|1708588258.603|STATE|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/1: Broker changed state UPDATE -> UP
%7|1708588258.603|BROADCAST|rdkafka#producer-1| [thrd:ssl://some-kafka-address-1:]: Broadcasting state change
%7|1708588258.603|CLUSTERID|rdkafka#producer-1| [thrd:main]: ssl://some-kafka-address-1:9094/1: ClusterId update "" -> "ZhFlyXjaT5y3tVXhF3ck-w"
%7|1708588258.603|CONTROLLERID|rdkafka#producer-1| [thrd:main]: ssl://some-kafka-address-1:9094/1: ControllerId update -1 -> 2
%7|1708588258.603|BROADCAST|rdkafka#producer-1| [thrd:main]: Broadcasting state change
%7|1708588258.663|RECV|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: Received MetadataResponse (v7, 457 bytes, CorrId 3, rtt 70.93ms)
%7|1708588258.664|METADATA|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/bootstrap: ===== Received metadata (for 1 requested topics): connected =====
%7|1708588258.664|METADATA|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/bootstrap: ClusterId: ZhFlyXjaT5y3tVXhF3ck-w, ControllerId: 2
%7|1708588258.664|METADATA|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/bootstrap: 3 brokers, 1 topics
%7|1708588258.664|METADATA|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/bootstrap: Broker #0/3: some-kafka-address-2:9094 NodeId 2
%7|1708588258.664|UPDATE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/bootstrap: NodeId changed from -1 to 2
%7|1708588258.664|UPDATE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/2: Name changed from ssl://some-kafka-address-2:9094/bootstrap to ssl://some-kafka-address-2:9094/2
%7|1708588258.664|LEADER|rdkafka#producer-2| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/2: Mapped 0 partition(s) to broker
%7|1708588258.664|BROADCAST|rdkafka#producer-2| [thrd:ssl://some-kafka-address-2:]: Broadcasting state change
%7|1708588258.665|METADATA|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/bootstrap: Broker #1/3: some-kafka-address-3:9094 NodeId 3
%7|1708588258.665|UPDATE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/bootstrap: NodeId changed from -1 to 3
%7|1708588258.665|UPDATE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Name changed from ssl://some-kafka-address-3:9094/bootstrap to ssl://some-kafka-address-3:9094/3
%7|1708588258.665|LEADER|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Mapped 0 partition(s) to broker
%7|1708588258.665|BROADCAST|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: Broadcasting state change
%7|1708588258.665|METADATA|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/bootstrap: Broker #2/3: some-kafka-address-1:9094 NodeId 1
%7|1708588258.665|UPDATE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/bootstrap: NodeId changed from -1 to 1
%7|1708588258.666|UPDATE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/1: Name changed from ssl://some-kafka-address-1:9094/bootstrap to ssl://some-kafka-address-1:9094/1
%7|1708588258.666|LEADER|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/1: Mapped 0 partition(s) to broker
%7|1708588258.666|STATE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/1: Broker changed state UP -> UPDATE
%7|1708588258.666|BROADCAST|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: Broadcasting state change
%7|1708588258.666|BROADCAST|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: Broadcasting state change
%7|1708588258.666|STATE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/1: Broker changed state UPDATE -> UP
%7|1708588258.666|BROADCAST|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: Broadcasting state change
%7|1708588258.666|METADATA|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/1: Topic my__topic with 3 partitions
%7|1708588258.666|STATE|rdkafka#producer-2| [thrd:main]: Topic my__topic changed state unknown -> exists
%7|1708588258.666|PARTCNT|rdkafka#producer-2| [thrd:main]: Topic my__topic partition count changed from 0 to 3
%7|1708588258.666|TOPPARNEW|rdkafka#producer-2| [thrd:main]: NEW my__topic [0] 0x7f9bf40055a0 refcnt 0x7f9bf4005630 (at rd_kafka_topic_partition_cnt_update:911)
%7|1708588258.666|TOPPARNEW|rdkafka#producer-2| [thrd:main]: NEW my__topic [1] 0x7f9bf4005b20 refcnt 0x7f9bf4005bb0 (at rd_kafka_topic_partition_cnt_update:911)
%7|1708588258.666|TOPPARNEW|rdkafka#producer-2| [thrd:main]: NEW my__topic [2] 0x7f9bf4006160 refcnt 0x7f9bf40061f0 (at rd_kafka_topic_partition_cnt_update:911)
%7|1708588258.666|METADATA|rdkafka#producer-2| [thrd:main]: Topic my__topic partition 0 Leader 3 Epoch -1
%7|1708588258.666|BRKDELGT|rdkafka#producer-2| [thrd:main]: my__topic [0]: delegate to broker ssl://some-kafka-address-3:9094/3 (rktp 0x7f9bf40055a0, term 0, ref 2)
%7|1708588258.667|BRKDELGT|rdkafka#producer-2| [thrd:main]: my__topic [0]: delegating to broker ssl://some-kafka-address-3:9094/3 for partition with 0 messages (0 bytes) queued
%7|1708588258.667|BRKMIGR|rdkafka#producer-2| [thrd:main]: Migrating topic my__topic [0] 0x7f9bf40055a0 from (none) to ssl://some-kafka-address-3:9094/3 (sending PARTITION_JOIN to ssl://some-kafka-address-3:9094/3)
%7|1708588258.667|METADATA|rdkafka#producer-2| [thrd:main]: Topic my__topic partition 1 Leader 1 Epoch -1
%7|1708588258.667|BRKDELGT|rdkafka#producer-2| [thrd:main]: my__topic [1]: delegate to broker ssl://some-kafka-address-1:9094/1 (rktp 0x7f9bf4005b20, term 0, ref 2)
%7|1708588258.667|BRKDELGT|rdkafka#producer-2| [thrd:main]: my__topic [1]: delegating to broker ssl://some-kafka-address-1:9094/1 for partition with 0 messages (0 bytes) queued
%7|1708588258.667|BRKMIGR|rdkafka#producer-2| [thrd:main]: Migrating topic my__topic [1] 0x7f9bf4005b20 from (none) to ssl://some-kafka-address-1:9094/1 (sending PARTITION_JOIN to ssl://some-kafka-address-1:9094/1)
%7|1708588258.667|METADATA|rdkafka#producer-2| [thrd:main]: Topic my__topic partition 2 Leader 2 Epoch -1
%7|1708588258.667|BRKDELGT|rdkafka#producer-2| [thrd:main]: my__topic [2]: delegate to broker ssl://some-kafka-address-2:9094/2 (rktp 0x7f9bf4006160, term 0, ref 2)
%7|1708588258.667|BRKDELGT|rdkafka#producer-2| [thrd:main]: my__topic [2]: delegating to broker ssl://some-kafka-address-2:9094/2 for partition with 0 messages (0 bytes) queued
%7|1708588258.667|BRKMIGR|rdkafka#producer-2| [thrd:main]: Migrating topic my__topic [2] 0x7f9bf4006160 from (none) to ssl://some-kafka-address-2:9094/2 (sending PARTITION_JOIN to ssl://some-kafka-address-2:9094/2)
%7|1708588258.667|PARTCNT|rdkafka#producer-2| [thrd:main]: Partitioning 1 unassigned messages in topic my__topic to 3 partitions
%7|1708588258.668|TOPBRK|rdkafka#producer-2| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/2: Topic my__topic [2]: joining broker (rktp 0x7f9bf4006160, 0 message(s) queued)
%7|1708588258.668|FETCHADD|rdkafka#producer-2| [thrd:ssl://some-kafka-address-2:]: ssl://some-kafka-address-2:9094/2: Added my__topic [2] to active list (1 entries, opv 0, 0 messages queued): joining
%7|1708588258.668|BROADCAST|rdkafka#producer-2| [thrd:ssl://some-kafka-address-2:]: Broadcasting state change
%7|1708588258.667|TOPBRK|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/1: Topic my__topic [1]: joining broker (rktp 0x7f9bf4005b20, 0 message(s) queued)
%7|1708588258.668|FETCHADD|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/1: Added my__topic [1] to active list (1 entries, opv 0, 0 messages queued): joining
%7|1708588258.667|TOPBRK|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Topic my__topic [0]: joining broker (rktp 0x7f9bf40055a0, 0 message(s) queued)
%7|1708588258.669|FETCHADD|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Added my__topic [0] to active list (1 entries, opv 0, 0 messages queued): joining
%7|1708588258.669|BROADCAST|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: Broadcasting state change
%7|1708588258.669|BROADCAST|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: Broadcasting state change
%7|1708588258.669|PARTITIONER|rdkafka#producer-2| [thrd:main]: my__topic [0] is the new sticky partition
%7|1708588258.669|UAS|rdkafka#producer-2| [thrd:main]: 1/1 messages were partitioned in topic my__topic
%7|1708588258.669|METADATA|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/1: 1/1 requested topic(s) seen in metadata
%7|1708588258.669|CLUSTERID|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/1: ClusterId update "" -> "ZhFlyXjaT5y3tVXhF3ck-w"
%7|1708588258.669|CONTROLLERID|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/1: ControllerId update -1 -> 2
%7|1708588258.669|BROADCAST|rdkafka#producer-2| [thrd:main]: Broadcasting state change
%7|1708588258.669|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) in xmit queue (1 added from partition queue)
%7|1708588258.669|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) queued but broker not up
%7|1708588258.669|STATE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Broker changed state INIT -> TRY_CONNECT
%7|1708588258.669|BROADCAST|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: Broadcasting state change
%7|1708588258.669|CONNECT|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: broker in state TRY_CONNECT connecting
%7|1708588258.669|STATE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Broker changed state TRY_CONNECT -> CONNECT
%7|1708588258.670|BROADCAST|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: Broadcasting state change
%7|1708588258.752|CONNECT|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Connecting to ipv4#10.200.6.118:9094 (ssl) with socket 23
%7|1708588258.753|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) in xmit queue (0 added from partition queue)
%7|1708588258.753|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) queued but broker not up
%7|1708588258.753|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) in xmit queue (0 added from partition queue)
%7|1708588258.753|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) queued but broker not up
%7|1708588258.753|QRYLEADER|rdkafka#producer-2| [thrd:main]: Topic my__topic [0]: broker is down: re-query
%7|1708588258.753|METADATA|rdkafka#producer-2| [thrd:main]: Requesting metadata for 1/1 topics: refresh unavailable topics
%7|1708588258.753|METADATA|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/1: Request metadata for 1 topic(s): refresh unavailable topics
%7|1708588258.753|SEND|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/1: Sent MetadataRequest (v7, 58 bytes @ 0, CorrId 4)
%7|1708588258.827|RECV|rdkafka#producer-2| [thrd:ssl://some-kafka-address-1:]: ssl://some-kafka-address-1:9094/1: Received MetadataResponse (v7, 457 bytes, CorrId 4, rtt 73.39ms)
%7|1708588258.827|METADATA|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/1: ===== Received metadata (for 1 requested topics): refresh unavailable topics =====
%7|1708588258.827|METADATA|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/1: ClusterId: ZhFlyXjaT5y3tVXhF3ck-w, ControllerId: 2
%7|1708588258.827|METADATA|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/1: 3 brokers, 1 topics
%7|1708588258.827|METADATA|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/1: Broker #0/3: some-kafka-address-2:9094 NodeId 2
%7|1708588258.827|METADATA|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/1: Broker #1/3: some-kafka-address-3:9094 NodeId 3
%7|1708588258.827|METADATA|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/1: Broker #2/3: some-kafka-address-1:9094 NodeId 1
%7|1708588258.827|METADATA|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/1: Topic my__topic with 3 partitions
%7|1708588258.827|METADATA|rdkafka#producer-2| [thrd:main]: Topic my__topic partition 0 Leader 3 Epoch -1
%7|1708588258.827|METADATA|rdkafka#producer-2| [thrd:main]: Topic my__topic partition 1 Leader 1 Epoch -1
%7|1708588258.827|METADATA|rdkafka#producer-2| [thrd:main]: Topic my__topic partition 2 Leader 2 Epoch -1
%7|1708588258.827|METADATA|rdkafka#producer-2| [thrd:main]: ssl://some-kafka-address-1:9094/1: 1/1 requested topic(s) seen in metadata
%7|1708588258.828|CONNECT|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Connected to ipv4#10.200.6.118:9094
%7|1708588258.828|STATE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Broker changed state CONNECT -> SSL_HANDSHAKE
%7|1708588258.828|BROADCAST|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: Broadcasting state change
%7|1708588258.828|ENDPOINT|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Enabled endpoint identification using hostname some-kafka-address-3
%7|1708588258.828|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) in xmit queue (0 added from partition queue)
%7|1708588258.828|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) queued but broker not up
%7|1708588258.904|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) in xmit queue (0 added from partition queue)
%7|1708588258.904|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) queued but broker not up
%7|1708588258.904|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) in xmit queue (0 added from partition queue)
%7|1708588258.904|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) queued but broker not up
%7|1708588258.909|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) in xmit queue (0 added from partition queue)
%7|1708588258.909|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) queued but broker not up
%7|1708588258.980|CONNECTED|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Connected (#1)
%7|1708588258.980|FEATURE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Updated enabled protocol features +ApiVersion to ApiVersion
%7|1708588258.980|STATE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Broker changed state SSL_HANDSHAKE -> APIVERSION_QUERY
%7|1708588258.980|BROADCAST|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: Broadcasting state change
%7|1708588258.980|SEND|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Sent ApiVersionRequest (v3, 66 bytes @ 0, CorrId 1)
%7|1708588258.980|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) in xmit queue (0 added from partition queue)
%7|1708588258.980|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) queued but broker not up
%7|1708588259.050|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) in xmit queue (0 added from partition queue)
%7|1708588259.050|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) queued but broker not up
%7|1708588259.051|RECV|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Received ApiVersionResponse (v3, 6 bytes, CorrId 1, rtt 70.84ms)
%7|1708588259.051|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiVersionRequest v3 failed due to UNSUPPORTED_VERSION: retrying with v0
%7|1708588259.051|SEND|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Sent ApiVersionRequest (v0, 21 bytes @ 0, CorrId 2)
%7|1708588259.051|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) in xmit queue (0 added from partition queue)
%7|1708588259.051|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) queued but broker not up
%7|1708588259.123|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) in xmit queue (0 added from partition queue)
%7|1708588259.123|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) queued but broker not up
%7|1708588259.124|RECV|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Received ApiVersionResponse (v0, 270 bytes, CorrId 2, rtt 72.42ms)
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Broker API support:
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey Produce (0) Versions 0..7
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey Fetch (1) Versions 0..10
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey ListOffsets (2) Versions 0..5
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey Metadata (3) Versions 0..7
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey LeaderAndIsr (4) Versions 0..2
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey StopReplica (5) Versions 0..1
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey UpdateMetadata (6) Versions 0..5
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey ControlledShutdown (7) Versions 0..2
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey OffsetCommit (8) Versions 0..6
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey OffsetFetch (9) Versions 0..5
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey FindCoordinator (10) Versions 0..2
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey JoinGroup (11) Versions 0..4
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey Heartbeat (12) Versions 0..2
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey LeaveGroup (13) Versions 0..2
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey SyncGroup (14) Versions 0..2
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey DescribeGroups (15) Versions 0..2
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey ListGroups (16) Versions 0..2
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey SaslHandshake (17) Versions 0..1
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey ApiVersion (18) Versions 0..2
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey CreateTopics (19) Versions 0..3
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey DeleteTopics (20) Versions 0..3
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey DeleteRecords (21) Versions 0..1
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey InitProducerId (22) Versions 0..1
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey OffsetForLeaderEpoch (23) Versions 0..2
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey AddPartitionsToTxn (24) Versions 0..1
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey AddOffsetsToTxn (25) Versions 0..1
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey EndTxn (26) Versions 0..1
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey WriteTxnMarkers (27) Versions 0..0
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey TxnOffsetCommit (28) Versions 0..2
%7|1708588259.124|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey DescribeAcls (29) Versions 0..1
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey CreateAcls (30) Versions 0..1
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey DeleteAcls (31) Versions 0..1
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey DescribeConfigs (32) Versions 0..2
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey AlterConfigs (33) Versions 0..1
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey AlterReplicaLogDirs (34) Versions 0..1
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey DescribeLogDirs (35) Versions 0..1
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey SaslAuthenticate (36) Versions 0..1
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey CreatePartitions (37) Versions 0..1
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey CreateDelegationToken (38) Versions 0..1
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey RenewDelegationToken (39) Versions 0..1
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey ExpireDelegationToken (40) Versions 0..1
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey DescribeDelegationToken (41) Versions 0..1
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey DeleteGroups (42) Versions 0..1
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: ApiKey ElectLeadersRequest (43) Versions 0..0
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature MsgVer1: Produce (2..2) supported by broker
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature MsgVer1: Fetch (2..2) supported by broker
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Enabling feature MsgVer1
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature MsgVer2: Produce (3..3) supported by broker
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature MsgVer2: Fetch (4..4) supported by broker
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Enabling feature MsgVer2
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature ApiVersion: ApiVersion (0..0) supported by broker
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Enabling feature ApiVersion
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature BrokerGroupCoordinator: FindCoordinator (0..0) supported by broker
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Enabling feature BrokerGroupCoordinator
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature BrokerBalancedConsumer: FindCoordinator (0..0) supported by broker
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature BrokerBalancedConsumer: OffsetCommit (1..2) supported by broker
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature BrokerBalancedConsumer: OffsetFetch (1..1) supported by broker
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature BrokerBalancedConsumer: JoinGroup (0..0) supported by broker
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature BrokerBalancedConsumer: SyncGroup (0..0) supported by broker
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature BrokerBalancedConsumer: Heartbeat (0..0) supported by broker
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature BrokerBalancedConsumer: LeaveGroup (0..0) supported by broker
%7|1708588259.125|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Enabling feature BrokerBalancedConsumer
%7|1708588259.126|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature ThrottleTime: Produce (1..2) supported by broker
%7|1708588259.126|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature ThrottleTime: Fetch (1..2) supported by broker
%7|1708588259.126|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Enabling feature ThrottleTime
%7|1708588259.126|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature Sasl: JoinGroup (0..0) supported by broker
%7|1708588259.126|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Enabling feature Sasl
%7|1708588259.126|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature SaslHandshake: SaslHandshake (0..0) supported by broker
%7|1708588259.126|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Enabling feature SaslHandshake
%7|1708588259.126|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature LZ4: FindCoordinator (0..0) supported by broker
%7|1708588259.126|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Enabling feature LZ4
%7|1708588259.126|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature OffsetTime: ListOffsets (1..1) supported by broker
%7|1708588259.126|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Enabling feature OffsetTime
%7|1708588259.126|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature IdempotentProducer: InitProducerId (0..0) supported by broker
%7|1708588259.126|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Enabling feature IdempotentProducer
%7|1708588259.138|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature ZSTD: Produce (7..7) supported by broker
%7|1708588259.138|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature ZSTD: Fetch (10..10) supported by broker
%7|1708588259.138|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Enabling feature ZSTD
%7|1708588259.138|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature SaslAuthReq: SaslHandshake (1..1) supported by broker
%7|1708588259.138|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Feature SaslAuthReq: SaslAuthenticate (0..1) supported by broker
%7|1708588259.138|APIVERSION|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Enabling feature SaslAuthReq
%7|1708588259.138|FEATURE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Updated enabled protocol features to MsgVer1,ApiVersion,BrokerBalancedConsumer,ThrottleTime,Sasl,SaslHandshake,BrokerGroupCoordinator,LZ4,OffsetTime,MsgVer2,IdempotentProducer,ZSTD,SaslAuthReq
%7|1708588259.138|STATE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Broker changed state APIVERSION_QUERY -> UP
%7|1708588259.138|BROADCAST|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: Broadcasting state change
%7|1708588259.138|METADATA|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: Skipping metadata refresh of 1 topic(s): connected: already being requested
%7|1708588259.138|TOPPAR|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0] 1 message(s) in xmit queue (0 added from partition queue)
%7|1708588259.138|PRODUCE|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0]: Produce MessageSet with 1 message(s) (444 bytes, ApiVersion 7, MsgVersion 2, MsgId 0, BaseSeq -1, PID{Invalid}, uncompressed)
%7|1708588259.139|SEND|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Sent ProduceRequest (v7, 521 bytes @ 0, CorrId 3)
%7|1708588259.223|RECV|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: Received ProduceResponse (v7, 74 bytes, CorrId 3, rtt 84.24ms)
%7|1708588259.223|MSGSET|rdkafka#producer-2| [thrd:ssl://some-kafka-address-3:]: ssl://some-kafka-address-3:9094/3: my__topic [0]: MessageSet with 1 message(s) (MsgId 0, BaseSeq -1) delivered
Description
I'm using Celery to parallelize a process of ingesting data to a Kafka topic. What I've noticed is that some messages do not end up in Kafka even though the library emits no issues. I've tried several configuration attempts to mitigate it but to no avail. I'm using version 2.2.0 of the library with Kafka 3.0.1 with Celery 5.3.4. Both kafka and the producer run in Docker.
How to reproduce
Celery config:
Producer code:
Celery run command
celery -A main worker --autoscale=20,0 --loglevel=INFO -O fair --max-tasks-per-child 50
Checklist
Please provide the following information:
confluent_kafka.version()
andconfluent_kafka.libversion()
):{...}
'debug': '..'
as necessary)