Errors are consumed as messages

[alexcocinda]

Description

I am implementing an idempotent Elasticsearch sink with bulk indexing support.

Occasionally, the consumer receives errors as messages from Kafka. For example, when a topic is not available, this message is consumed:

Subscribed topic not available: not-existent-topic: Broker: Unknown topic or partition.

Is this the expected behavior? If so, how can I disable this kind of message?

How to reproduce

1. Start Kafka.

   docker run -d --rm --net=host --name=kafka -e ALLOW_PLAINTEXT_LISTENER=yes -v /opt/bitnami/kafka/logs:/opt/bitnami/kafka/logs bitnami/kafka:3.5.0

2. Start consuming messages for a topic that does not exist.
3. Notice that a fake message is consumed.

Checklist

Please provide the following information:

• [x] librdkafka version (release number or git tag): 2.1.1
• [x] Apache Kafka version: 3.5.0
• [x] librdkafka client configuration: auto.reset.offset=latest, enable.auto.commit=false, enable.auto.offset.store=false>
• [x] Operating system: Debian 11
• [x] Provide logs (with debug=.. as necessary) from librdkafka
• [ ] Provide broker log excerpts
• [ ] Critical issue

Consumer logs:

%7|1688114389.320|MEMBERID|rdkafka#consumer-1| [thrd:app]: Group "dev2": updating member id "(not-set)" -> ""
%7|1688114389.321|WAKEUPFD|rdkafka#consumer-1| [thrd:app]: GroupCoordinator: Enabled low-latency ops queue wake-ups
%7|1688114389.321|BROKER|rdkafka#consumer-1| [thrd:app]: GroupCoordinator: Added new broker with NodeId -1
%7|1688114389.321|BRKMAIN|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator: Enter main broker thread
%7|1688114389.321|CGRPSTATE|rdkafka#consumer-1| [thrd:main]: Group "dev2" changed state init -> query-coord (join-state init)
%7|1688114389.321|BROADCAST|rdkafka#consumer-1| [thrd:main]: Broadcasting state change
%7|1688114389.321|WAKEUPFD|rdkafka#consumer-1| [thrd:app]: localhost:9092/bootstrap: Enabled low-latency ops queue wake-ups
%7|1688114389.321|BRKMAIN|rdkafka#consumer-1| [thrd::0/internal]: :0/internal: Enter main broker thread
%7|1688114389.321|BROKER|rdkafka#consumer-1| [thrd:app]: localhost:9092/bootstrap: Added new broker with NodeId -1
%7|1688114389.321|INIT|rdkafka#consumer-1| [thrd:app]: librdkafka v2.1.1 (0x20101ff) rdkafka#consumer-1 initialized (builtin.features gzip,snappy,ssl,sasl,regex,lz4,sasl_plain,sasl_scram,plugins,sasl_oauthbearer,http,oidc, CC CXX PKGCONFIG INSTALL GNULD LDS C11THREADS LIBDL PLUGINS ZLIB SSL CURL HDRHISTOGRAM SYSLOG SNAPPY SOCKEM SASL_SCRAM SASL_OAUTHBEARER OAUTHBEARER_OIDC CRC32C_HW, debug 0xfffff)
%7|1688114389.321|CONNECT|rdkafka#consumer-1| [thrd:main]: localhost:9092/bootstrap: Selected for cluster connection: coordinator query (broker has 0 connection attempt(s))
%7|1688114389.321|CONF|rdkafka#consumer-1| [thrd:app]: Client configuration:
%7|1688114389.321|CONF|rdkafka#consumer-1| [thrd:app]:   client.software.version = 2.1.1
%7|1688114389.321|CONF|rdkafka#consumer-1| [thrd:app]:   metadata.broker.list = localhost:9092
%7|1688114389.321|CONF|rdkafka#consumer-1| [thrd:app]:   debug = generic,broker,topic,metadata,feature,queue,msg,protocol,cgrp,security,fetch,interceptor,plugin,consumer,admin,eos,mock,assignor,conf,all
%7|1688114389.321|BRKMAIN|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enter main broker thread
%7|1688114389.321|CONF|rdkafka#consumer-1| [thrd:app]:   opaque = 0x5636a42e9d48
%7|1688114389.321|CONF|rdkafka#consumer-1| [thrd:app]:   security.protocol = plaintext
%7|1688114389.321|CONF|rdkafka#consumer-1| [thrd:app]:   ssl_key = [redacted]
%7|1688114389.321|CONF|rdkafka#consumer-1| [thrd:app]:   group.id = dev2
%7|1688114389.321|CGRPQUERY|rdkafka#consumer-1| [thrd:main]: Group "dev2": no broker available for coordinator query: intervaled in state query-coord
%7|1688114389.321|CONF|rdkafka#consumer-1| [thrd:app]:   enable.auto.commit = false
%7|1688114389.321|CONNECT|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Received CONNECT op
%7|1688114389.321|STATE|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Broker changed state INIT -> TRY_CONNECT
%7|1688114389.321|BROADCAST|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: Broadcasting state change
%7|1688114389.321|CONF|rdkafka#consumer-1| [thrd:app]:   enable.auto.offset.store = false
%7|1688114389.321|CONNECT|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: broker in state TRY_CONNECT connecting
%7|1688114389.321|STATE|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Broker changed state TRY_CONNECT -> CONNECT
%7|1688114389.321|BROADCAST|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: Broadcasting state change
%7|1688114389.321|CGRPOP|rdkafka#consumer-1| [thrd:main]: Group "dev2" received op SUBSCRIBE in state query-coord (join-state init)
%7|1688114389.321|SUBSCRIBE|rdkafka#consumer-1| [thrd:main]: Group "dev2": subscribe to new subscription of 1 topics (join-state init)
%7|1688114389.321|CONNECT|rdkafka#consumer-1| [thrd:main]: Not selecting any broker for cluster connection: still suppressed for 49ms: coordinator query
%7|1688114389.321|CGRPQUERY|rdkafka#consumer-1| [thrd:main]: Group "dev2": no broker available for coordinator query: intervaled in state query-coord
[2023-06-30 08:39:49.321] [info] Started conveying messages.
%7|1688114389.321|CONNECT|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Connecting to ipv4#127.0.0.1:9092 (plaintext) with socket 9
%7|1688114389.321|CONNECT|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Connected to ipv4#127.0.0.1:9092
%7|1688114389.321|CONNECTED|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Connected (#1)
%7|1688114389.321|FEATURE|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Updated enabled protocol features +ApiVersion to ApiVersion
%7|1688114389.321|STATE|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Broker changed state CONNECT -> APIVERSION_QUERY
%7|1688114389.321|BROADCAST|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: Broadcasting state change
%7|1688114389.321|SEND|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Sent ApiVersionRequest (v3, 40 bytes @ 0, CorrId 1)
%7|1688114389.322|RECV|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Received ApiVersionResponse (v3, 411 bytes, CorrId 1, rtt 0.62ms)
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Broker API support:
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey Produce (0) Versions 0..9
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey Fetch (1) Versions 0..13
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey ListOffsets (2) Versions 0..7
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey Metadata (3) Versions 0..12
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey OffsetCommit (8) Versions 0..8
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey OffsetFetch (9) Versions 0..8
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey FindCoordinator (10) Versions 0..4
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey JoinGroup (11) Versions 0..9
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey Heartbeat (12) Versions 0..4
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey LeaveGroup (13) Versions 0..5
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey SyncGroup (14) Versions 0..5
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey DescribeGroups (15) Versions 0..5
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey ListGroups (16) Versions 0..4
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey SaslHandshake (17) Versions 0..1
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey ApiVersion (18) Versions 0..3
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey CreateTopics (19) Versions 0..7
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey DeleteTopics (20) Versions 0..6
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey DeleteRecords (21) Versions 0..2
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey InitProducerId (22) Versions 0..4
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey OffsetForLeaderEpoch (23) Versions 0..4
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey AddPartitionsToTxn (24) Versions 0..3
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey AddOffsetsToTxn (25) Versions 0..3
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey EndTxn (26) Versions 0..3
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey WriteTxnMarkers (27) Versions 0..1
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey TxnOffsetCommit (28) Versions 0..3
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey DescribeAcls (29) Versions 0..3
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey CreateAcls (30) Versions 0..3
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey DeleteAcls (31) Versions 0..3
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey DescribeConfigs (32) Versions 0..4
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey AlterConfigs (33) Versions 0..2
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey AlterReplicaLogDirs (34) Versions 0..2
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey DescribeLogDirs (35) Versions 0..4
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey SaslAuthenticate (36) Versions 0..2
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey CreatePartitions (37) Versions 0..3
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey DeleteGroups (42) Versions 0..2
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey ElectLeadersRequest (43) Versions 0..2
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey IncrementalAlterConfigsRequest (44) Versions 0..1
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey AlterPartitionReassignmentsRequest (45) Versions 0..0
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey ListPartitionReassignmentsRequest (46) Versions 0..0
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey OffsetDeleteRequest (47) Versions 0..0
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey DescribeClientQuotasRequest (48) Versions 0..1
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey AlterClientQuotasRequest (49) Versions 0..1
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey DescribeQuorumRequest (55) Versions 0..1
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey UpdateFeaturesRequest (57) Versions 0..1
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey DescribeCluster (60) Versions 0..0
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey DescribeProducers (61) Versions 0..0
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey UnregisterBroker (64) Versions 0..0
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey DescribeTransactions (65) Versions 0..0
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:   ApiKey ListTransactions (66) Versions 0..0
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature MsgVer1: Produce (2..2) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature MsgVer1: Fetch (2..2) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature MsgVer1
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature MsgVer2: Produce (3..3) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature MsgVer2: Fetch (4..4) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature MsgVer2
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature ApiVersion: ApiVersion (0..0) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature ApiVersion
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature BrokerGroupCoordinator: FindCoordinator (0..0) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature BrokerGroupCoordinator
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature BrokerBalancedConsumer: FindCoordinator (0..0) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature BrokerBalancedConsumer: OffsetCommit (1..2) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature BrokerBalancedConsumer: OffsetFetch (1..1) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature BrokerBalancedConsumer: JoinGroup (0..0) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature BrokerBalancedConsumer: SyncGroup (0..0) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature BrokerBalancedConsumer: Heartbeat (0..0) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature BrokerBalancedConsumer: LeaveGroup (0..0) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature BrokerBalancedConsumer
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature ThrottleTime: Produce (1..2) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature ThrottleTime: Fetch (1..2) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature ThrottleTime
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature Sasl: JoinGroup (0..0) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature Sasl
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature SaslHandshake: SaslHandshake (0..0) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature SaslHandshake
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature LZ4: FindCoordinator (0..0) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature LZ4
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature OffsetTime: ListOffsets (1..1) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature OffsetTime
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature IdempotentProducer: InitProducerId (0..0) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature IdempotentProducer
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature ZSTD: Produce (7..7) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature ZSTD: Fetch (10..10) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature ZSTD
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature SaslAuthReq: SaslHandshake (1..1) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap:  Feature SaslAuthReq: SaslAuthenticate (0..0) supported by broker
%7|1688114389.322|APIVERSION|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature SaslAuthReq
%7|1688114389.322|FEATURE|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Updated enabled protocol features to MsgVer1,ApiVersion,BrokerBalancedConsumer,ThrottleTime,Sasl,SaslHandshake,BrokerGroupCoordinator,LZ4,OffsetTime,MsgVer2,IdempotentProducer,ZSTD,SaslAuthReq
%7|1688114389.322|STATE|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Broker changed state APIVERSION_QUERY -> UP
%7|1688114389.322|BROADCAST|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: Broadcasting state change
%7|1688114389.322|METADATA|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Request metadata for brokers only: connected
%7|1688114389.322|CGRPQUERY|rdkafka#consumer-1| [thrd:main]: localhost:9092/bootstrap: Group "dev2": querying for coordinator: intervaled in state query-coord
%7|1688114389.322|SEND|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Sent MetadataRequest (v9, 27 bytes @ 0, CorrId 2)
%7|1688114389.322|CGRPSTATE|rdkafka#consumer-1| [thrd:main]: Group "dev2" changed state query-coord -> wait-coord (join-state init)
%7|1688114389.322|BROADCAST|rdkafka#consumer-1| [thrd:main]: Broadcasting state change
%7|1688114389.322|SEND|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Sent FindCoordinatorRequest (v2, 28 bytes @ 0, CorrId 3)
%7|1688114389.323|RECV|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Received MetadataResponse (v9, 56 bytes, CorrId 2, rtt 0.28ms)
%7|1688114389.323|METADATA|rdkafka#consumer-1| [thrd:main]: localhost:9092/bootstrap: ===== Received metadata: connected =====
%7|1688114389.323|METADATA|rdkafka#consumer-1| [thrd:main]: localhost:9092/bootstrap: ClusterId: uvB9BFuSSGOEgeSjOl4KdA, ControllerId: 1
%7|1688114389.323|METADATA|rdkafka#consumer-1| [thrd:main]: localhost:9092/bootstrap: 1 brokers, 0 topics
%7|1688114389.323|METADATA|rdkafka#consumer-1| [thrd:main]: localhost:9092/bootstrap:   Broker #0/1: ubutus:9092 NodeId 1
%7|1688114389.323|WAKEUPFD|rdkafka#consumer-1| [thrd:main]: ubutus:9092/1: Enabled low-latency ops queue wake-ups
%7|1688114389.323|BROKER|rdkafka#consumer-1| [thrd:main]: ubutus:9092/1: Added new broker with NodeId 1
%7|1688114389.323|CLUSTERID|rdkafka#consumer-1| [thrd:main]: localhost:9092/bootstrap: ClusterId update "" -> "uvB9BFuSSGOEgeSjOl4KdA"
%7|1688114389.323|CONTROLLERID|rdkafka#consumer-1| [thrd:main]: localhost:9092/bootstrap: ControllerId update -1 -> 1
%7|1688114389.323|BROADCAST|rdkafka#consumer-1| [thrd:main]: Broadcasting state change
%7|1688114389.323|BRKMAIN|rdkafka#consumer-1| [thrd:ubutus:9092/1]: ubutus:9092/1: Enter main broker thread
%7|1688114389.323|RECV|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Received FindCoordinatorResponse (v2, 28 bytes, CorrId 3, rtt 0.61ms)
%7|1688114389.323|CGRPCOORD|rdkafka#consumer-1| [thrd:main]: localhost:9092/bootstrap: Group "dev2" coordinator is ubutus:9092 id 1
%7|1688114389.323|CGRPCOORD|rdkafka#consumer-1| [thrd:main]: Group "dev2" changing coordinator -1 -> 1
%7|1688114389.323|COORDSET|rdkafka#consumer-1| [thrd:main]: Group "dev2" coordinator set to broker ubutus:9092/1
%7|1688114389.323|CGRPSTATE|rdkafka#consumer-1| [thrd:main]: Group "dev2" changed state wait-coord -> wait-broker-transport (join-state init)
%7|1688114389.323|BROADCAST|rdkafka#consumer-1| [thrd:main]: Broadcasting state change
%7|1688114389.323|NODENAME|rdkafka#consumer-1| [thrd:main]: GroupCoordinator: Broker nodename changed from "" to "ubutus:9092"
%7|1688114389.323|NODEID|rdkafka#consumer-1| [thrd:main]: GroupCoordinator: Broker nodeid changed from -1 to 1
%7|1688114389.323|CONNECT|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator: Received CONNECT op
%7|1688114389.323|CGRPQUERY|rdkafka#consumer-1| [thrd:main]: localhost:9092/bootstrap: Group "dev2": querying for coordinator: intervaled in state wait-broker-transport
%7|1688114389.323|STATE|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator: Broker changed state INIT -> TRY_CONNECT
%7|1688114389.323|BROADCAST|rdkafka#consumer-1| [thrd:GroupCoordinator]: Broadcasting state change
%7|1688114389.323|CONNECT|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: broker in state TRY_CONNECT connecting
%7|1688114389.323|STATE|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator: Broker changed state TRY_CONNECT -> CONNECT
%7|1688114389.323|BROADCAST|rdkafka#consumer-1| [thrd:GroupCoordinator]: Broadcasting state change
%7|1688114389.323|SEND|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Sent FindCoordinatorRequest (v2, 28 bytes @ 0, CorrId 4)
%7|1688114389.323|CONNECT|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Connecting to ipv4#127.0.1.1:9092 (plaintext) with socket 12
%7|1688114389.323|CONNECT|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Connected to ipv4#127.0.1.1:9092
%7|1688114389.323|CONNECTED|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Connected (#1)
%7|1688114389.323|FEATURE|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Updated enabled protocol features +ApiVersion to ApiVersion
%7|1688114389.323|STATE|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator: Broker changed state CONNECT -> APIVERSION_QUERY
%7|1688114389.323|BROADCAST|rdkafka#consumer-1| [thrd:GroupCoordinator]: Broadcasting state change
%7|1688114389.323|SEND|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Sent ApiVersionRequest (v3, 40 bytes @ 0, CorrId 1)
%7|1688114389.323|RECV|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Received FindCoordinatorResponse (v2, 28 bytes, CorrId 4, rtt 0.33ms)
%7|1688114389.323|CGRPCOORD|rdkafka#consumer-1| [thrd:main]: localhost:9092/bootstrap: Group "dev2" coordinator is ubutus:9092 id 1
%7|1688114389.324|RECV|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Received ApiVersionResponse (v3, 411 bytes, CorrId 1, rtt 0.53ms)
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Broker API support:
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey Produce (0) Versions 0..9
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey Fetch (1) Versions 0..13
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey ListOffsets (2) Versions 0..7
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey Metadata (3) Versions 0..12
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey OffsetCommit (8) Versions 0..8
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey OffsetFetch (9) Versions 0..8
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey FindCoordinator (10) Versions 0..4
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey JoinGroup (11) Versions 0..9
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey Heartbeat (12) Versions 0..4
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey LeaveGroup (13) Versions 0..5
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey SyncGroup (14) Versions 0..5
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey DescribeGroups (15) Versions 0..5
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey ListGroups (16) Versions 0..4
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey SaslHandshake (17) Versions 0..1
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey ApiVersion (18) Versions 0..3
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey CreateTopics (19) Versions 0..7
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey DeleteTopics (20) Versions 0..6
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey DeleteRecords (21) Versions 0..2
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey InitProducerId (22) Versions 0..4
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey OffsetForLeaderEpoch (23) Versions 0..4
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey AddPartitionsToTxn (24) Versions 0..3
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey AddOffsetsToTxn (25) Versions 0..3
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey EndTxn (26) Versions 0..3
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey WriteTxnMarkers (27) Versions 0..1
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey TxnOffsetCommit (28) Versions 0..3
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey DescribeAcls (29) Versions 0..3
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey CreateAcls (30) Versions 0..3
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey DeleteAcls (31) Versions 0..3
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey DescribeConfigs (32) Versions 0..4
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey AlterConfigs (33) Versions 0..2
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey AlterReplicaLogDirs (34) Versions 0..2
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey DescribeLogDirs (35) Versions 0..4
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey SaslAuthenticate (36) Versions 0..2
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey CreatePartitions (37) Versions 0..3
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey DeleteGroups (42) Versions 0..2
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey ElectLeadersRequest (43) Versions 0..2
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey IncrementalAlterConfigsRequest (44) Versions 0..1
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey AlterPartitionReassignmentsRequest (45) Versions 0..0
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey ListPartitionReassignmentsRequest (46) Versions 0..0
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey OffsetDeleteRequest (47) Versions 0..0
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey DescribeClientQuotasRequest (48) Versions 0..1
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey AlterClientQuotasRequest (49) Versions 0..1
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey DescribeQuorumRequest (55) Versions 0..1
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey UpdateFeaturesRequest (57) Versions 0..1
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey DescribeCluster (60) Versions 0..0
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey DescribeProducers (61) Versions 0..0
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey UnregisterBroker (64) Versions 0..0
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey DescribeTransactions (65) Versions 0..0
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:   ApiKey ListTransactions (66) Versions 0..0
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature MsgVer1: Produce (2..2) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature MsgVer1: Fetch (2..2) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Enabling feature MsgVer1
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature MsgVer2: Produce (3..3) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature MsgVer2: Fetch (4..4) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Enabling feature MsgVer2
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature ApiVersion: ApiVersion (0..0) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Enabling feature ApiVersion
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature BrokerGroupCoordinator: FindCoordinator (0..0) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Enabling feature BrokerGroupCoordinator
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature BrokerBalancedConsumer: FindCoordinator (0..0) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature BrokerBalancedConsumer: OffsetCommit (1..2) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature BrokerBalancedConsumer: OffsetFetch (1..1) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature BrokerBalancedConsumer: JoinGroup (0..0) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature BrokerBalancedConsumer: SyncGroup (0..0) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature BrokerBalancedConsumer: Heartbeat (0..0) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature BrokerBalancedConsumer: LeaveGroup (0..0) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Enabling feature BrokerBalancedConsumer
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature ThrottleTime: Produce (1..2) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature ThrottleTime: Fetch (1..2) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Enabling feature ThrottleTime
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature Sasl: JoinGroup (0..0) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Enabling feature Sasl
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature SaslHandshake: SaslHandshake (0..0) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Enabling feature SaslHandshake
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature LZ4: FindCoordinator (0..0) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Enabling feature LZ4
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature OffsetTime: ListOffsets (1..1) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Enabling feature OffsetTime
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature IdempotentProducer: InitProducerId (0..0) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Enabling feature IdempotentProducer
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature ZSTD: Produce (7..7) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature ZSTD: Fetch (10..10) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Enabling feature ZSTD
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature SaslAuthReq: SaslHandshake (1..1) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1:  Feature SaslAuthReq: SaslAuthenticate (0..0) supported by broker
%7|1688114389.324|APIVERSION|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Enabling feature SaslAuthReq
%7|1688114389.324|FEATURE|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Updated enabled protocol features to MsgVer1,ApiVersion,BrokerBalancedConsumer,ThrottleTime,Sasl,SaslHandshake,BrokerGroupCoordinator,LZ4,OffsetTime,MsgVer2,IdempotentProducer,ZSTD,SaslAuthReq
%7|1688114389.324|STATE|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator: Broker changed state APIVERSION_QUERY -> UP
%7|1688114389.324|BROADCAST|rdkafka#consumer-1| [thrd:GroupCoordinator]: Broadcasting state change
%7|1688114389.324|METADATA|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Request metadata for brokers only: connected
%7|1688114389.324|CGRPSTATE|rdkafka#consumer-1| [thrd:main]: Group "dev2" changed state wait-broker-transport -> up (join-state init)
%7|1688114389.324|BROADCAST|rdkafka#consumer-1| [thrd:main]: Broadcasting state change
%7|1688114389.324|JOIN|rdkafka#consumer-1| [thrd:main]: Group "dev2": join with 0 subscribed topic(s)
%7|1688114389.324|METADATA|rdkafka#consumer-1| [thrd:main]: Hinted cache of 1/1 topic(s) being queried
%7|1688114389.324|SEND|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Sent MetadataRequest (v9, 27 bytes @ 0, CorrId 2)
%7|1688114389.324|CGRPMETADATA|rdkafka#consumer-1| [thrd:main]: consumer join: metadata for subscription only available for 0/1 topics (-1ms old)
%7|1688114389.324|METADATA|rdkafka#consumer-1| [thrd:main]: localhost:9092/bootstrap: Request metadata for 1 topic(s): consumer join
%7|1688114389.324|JOIN|rdkafka#consumer-1| [thrd:main]: Group "dev2": postponing join until up-to-date metadata is available
%7|1688114389.324|CGRPJOINSTATE|rdkafka#consumer-1| [thrd:main]: Group "dev2" changed join state init -> wait-metadata (state up)
%7|1688114389.324|DUMP|rdkafka#consumer-1| [thrd:main]: Assignment dump (started_cnt=0, wait_stop_cnt=0)
%7|1688114389.324|DUMP_ALL|rdkafka#consumer-1| [thrd:main]: List with 0 partition(s):
%7|1688114389.324|DUMP_PND|rdkafka#consumer-1| [thrd:main]: List with 0 partition(s):
%7|1688114389.324|DUMP_QRY|rdkafka#consumer-1| [thrd:main]: List with 0 partition(s):
%7|1688114389.324|DUMP_REM|rdkafka#consumer-1| [thrd:main]: List with 0 partition(s):
%7|1688114389.324|ASSIGNDONE|rdkafka#consumer-1| [thrd:main]: Group "dev2": assignment operations done in join-state wait-metadata (rebalance rejoin=false)
%7|1688114389.324|SEND|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Sent MetadataRequest (v9, 46 bytes @ 0, CorrId 5)
%7|1688114389.324|RECV|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator/1: Received MetadataResponse (v9, 56 bytes, CorrId 2, rtt 0.30ms)
%7|1688114389.324|METADATA|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/1: ===== Received metadata: connected =====
%7|1688114389.324|METADATA|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/1: ClusterId: uvB9BFuSSGOEgeSjOl4KdA, ControllerId: 1
%7|1688114389.324|METADATA|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/1: 1 brokers, 0 topics
%7|1688114389.324|METADATA|rdkafka#consumer-1| [thrd:main]: GroupCoordinator/1:   Broker #0/1: ubutus:9092 NodeId 1
%7|1688114389.325|RECV|rdkafka#consumer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Received MetadataResponse (v9, 83 bytes, CorrId 5, rtt 0.37ms)
%7|1688114389.325|METADATA|rdkafka#consumer-1| [thrd:main]: localhost:9092/bootstrap: ===== Received metadata (for 1 requested topics): consumer join =====
%7|1688114389.325|METADATA|rdkafka#consumer-1| [thrd:main]: localhost:9092/bootstrap: ClusterId: uvB9BFuSSGOEgeSjOl4KdA, ControllerId: 1
%7|1688114389.325|METADATA|rdkafka#consumer-1| [thrd:main]: localhost:9092/bootstrap: 1 brokers, 1 topics
%7|1688114389.325|METADATA|rdkafka#consumer-1| [thrd:main]: localhost:9092/bootstrap:   Topic cradle-es-frame-2 with 0 partitions: Broker: Unknown topic or partition
%7|1688114389.325|METADATA|rdkafka#consumer-1| [thrd:main]: localhost:9092/bootstrap:   Broker #0/1: ubutus:9092 NodeId 1
%7|1688114389.325|METADATA|rdkafka#consumer-1| [thrd:main]: localhost:9092/bootstrap: 1/1 requested topic(s) seen in metadata
%7|1688114389.325|TOPICERR|rdkafka#consumer-1| [thrd:main]: Subscribed topic not available: cradle-es-frame-2: Broker: Unknown topic or partition
%7|1688114389.325|SUBSCRIPTION|rdkafka#consumer-1| [thrd:main]: Group "dev2": no topics in metadata matched subscription
%7|1688114389.325|REJOIN|rdkafka#consumer-1| [thrd:main]: Group "dev2": subscription updated from metadata change: rejoining group in state wait-metadata
%7|1688114389.325|GRPLEADER|rdkafka#consumer-1| [thrd:main]: Group "dev2": resetting group leader info: group (re)join
%7|1688114389.325|REJOIN|rdkafka#consumer-1| [thrd:main]: Group "dev2" (re)joining in join-state wait-metadata with 0 assigned partition(s): Metadata for subscribed topic(s) has changed
%7|1688114389.325|REBALANCE|rdkafka#consumer-1| [thrd:main]: Group "dev2" initiating rebalance (NONE) in state up (join-state wait-metadata) with 0 assigned partition(s): Metadata for subscribed topic(s) has changed
%7|1688114389.325|REJOIN|rdkafka#consumer-1| [thrd:main]: Group "dev2": Rejoining group without an assignment: Metadata for subscribed topic(s) has changed
%7|1688114389.325|CGRPJOINSTATE|rdkafka#consumer-1| [thrd:main]: Group "dev2" changed join state wait-metadata -> init (state up)
%7|1688114389.325|JOIN|rdkafka#consumer-1| [thrd:main]: Group "dev2": join with 0 subscribed topic(s)
%7|1688114389.325|CGRPMETADATA|rdkafka#consumer-1| [thrd:main]: consumer join: metadata for subscription is up to date (0ms old)
%7|1688114389.325|SUBSCRIPTION|rdkafka#consumer-1| [thrd:main]: Group "dev2": no topics in metadata matched subscription
%7|1688114389.325|JOIN|rdkafka#consumer-1| [thrd:main]: Group "dev2": no matching topics based on 0ms old metadata: next metadata refresh in 300000ms
%7|1688114389.325|SUBSCRIPTION|rdkafka#consumer-1| [thrd:main]: Group "dev2": no topics in metadata matched subscription
[2023-06-30 08:39:49.325] [debug] Added message with id bebeeaa5-b635-4969-ab81-99568bb36d72 from cradle-es-frame-2 to bulk request: Subscribed topic not available: cradle-es-frame-2: Broker: Unknown topic or partition.

[emasab]

Hello, that is the expected behavior in case of non-existing topic. These kind of errors are expected behavior because they are the way to signal non-retriable errors to the application.