search

wbarnha / kafka-python-ng

Fork for Python client for Apache Kafka
https://wbarnha.github.io/kafka-python-ng/
Apache License 2.0
77 stars 10 forks source link

Socket EVENT_READ without in-flight-requests #29

Open wbarnha opened 8 months ago

wbarnha commented 8 months ago

I'm getting this error for a week now trying to move a project to Docker. I've tried kafka-python==1.4.4, kafka-python-1.4.7, and kafka-python==2.0.2, with a kafka 2.11 2.1.0 and cluster version 0.10.2.1.

The application has been working fine in production for many years, but for some reason, as soon as I add it into a Docker container (regardless of the kafka client and python version (3.6, 3.7, 3.8, 3.9)), it refuses to connect.

Other services have no problem connecting to the same kafka cluster at the same time. No duplicate subscriptions are happening.

Basically this is the code to reproduce (I use one Process for each kafka topic):

class EventReader:
    def __init__(self):
        ...

    def run(self):
        self.create_consumer()

        while True:
            try:
                self.try_to_read()
            except Exception as e:
                ...

    def try_to_read(self):
        for message in self.consumer:
            try:
                ...
            except Exception as e:
                ...

    def create_consumer(self):
        self.consumer = KafkaConsumer(
            'some-topic',
            group_id=group_id,
            bootstrap_servers=bootstrap_servers,
            enable_auto_commit=True,
            auto_offset_reset="latest",
            connections_max_idle_ms=9 * ONE_MINUTE,  # default: 9min
            max_poll_interval_ms=10 * ONE_MINUTE,  # default: 5min
            session_timeout_ms=ONE_MINUTE,  # default: 10s
            max_poll_records=50,  # default: 500,
            max_in_flight_requests_per_connection=10,  # default: 5
        )

for topic, handlers in topic_to_handlers.items():
    reader = EventReader(topic, handlers)
    process = Process(target=reader.run)
    process.start()

    gn_env.event_readers[topic] = process

Below is the info log. Debug doesn't reveal anything more relevant than the "Socket EVENT_READ without in-flight-requests".

P.S. I do not run Kafka in docker, which was the biggest reason I found for why people got this error:

2023-06-22 10:45:09,050 - kafka.client       - WARNING - Node 2 connection failed -- refreshing metadata
2023-06-22 10:45:09,052 - kafka.cluster      - INFO    - Group coordinator for event-v2-ddtg-cms-detect is BrokerMetadata(nodeId='coordinator-1', host='maggie-kafka-2.somedomain.lab', port=9094, rack=None)
2023-06-22 10:45:09,052 - kafka.coordinator  - INFO    - Discovered coordinator coordinator-1 for group event-v2-ddtg-cms-detect
2023-06-22 10:45:09,102 - kafka.conn         - INFO    - <BrokerConnection node_id=1 host=maggie-kafka-2.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.125', 9094)]>: connecting to maggie-kafka-2.somedomain.lab:9094 [('10.60.1.125', 9094) IPv4]
2023-06-22 10:45:09,104 - kafka.cluster      - INFO    - Group coordinator for event-v2-rrps-faceverify-detect is BrokerMetadata(nodeId='coordinator-1', host='maggie-kafka-2.somedomain.lab', port=9094, rack=None)
2023-06-22 10:45:09,105 - kafka.coordinator  - INFO    - Discovered coordinator coordinator-1 for group event-v2-rrps-faceverify-detect
2023-06-22 10:45:09,105 - kafka.coordinator  - WARNING - Marking the coordinator dead (node coordinator-1) for group event-v2-rrps-faceverify-detect: Node Disconnected.
2023-06-22 10:45:09,106 - kafka.conn         - INFO    - <BrokerConnection node_id=1 host=maggie-kafka-2.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.125', 9094)]>: Connection complete.
2023-06-22 10:45:09,108 - kafka.conn         - INFO    - <BrokerConnection node_id=coordinator-1 host=maggie-kafka-2.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.125', 9094)]>: connecting to maggie-kafka-2.somedomain.lab:9094 [('10.60.1.125', 9094) IPv4]
2023-06-22 10:45:09,109 - kafka.conn         - INFO    - <BrokerConnection node_id=coordinator-1 host=maggie-kafka-2.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.125', 9094)]>: Connection complete.
2023-06-22 10:45:09,110 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.126', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,110 - kafka.client       - WARNING - Node 2 connection failed -- refreshing metadata
2023-06-22 10:45:09,111 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.126', 9094)]>: connecting to maggie-kafka-3.somedomain.lab:9094 [('10.60.1.126', 9094) IPv4]
2023-06-22 10:45:09,112 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.126', 9094)]>: Connection complete.
2023-06-22 10:45:09,113 - kafka.cluster      - INFO    - Group coordinator for event-v2-rrps-faceverify-detect is BrokerMetadata(nodeId='coordinator-1', host='maggie-kafka-2.somedomain.lab', port=9094, rack=None)
2023-06-22 10:45:09,113 - kafka.coordinator  - INFO    - Discovered coordinator coordinator-1 for group event-v2-rrps-faceverify-detect
2023-06-22 10:45:09,117 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.126', 9094)]>: connecting to maggie-kafka-3.somedomain.lab:9094 [('10.60.1.126', 9094) IPv4]
2023-06-22 10:45:09,117 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.126', 9094)]>: Connection complete.
2023-06-22 10:45:09,119 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.126', 9094)]>: connecting to maggie-kafka-3.somedomain.lab:9094 [('10.60.1.126', 9094) IPv4]
2023-06-22 10:45:09,119 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.126', 9094)]>: Connection complete.
2023-06-22 10:45:09,126 - kafka.conn         - INFO    - <BrokerConnection node_id=0 host=maggie-kafka-1.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.124', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,127 - kafka.client       - WARNING - Node 0 connection failed -- refreshing metadata
2023-06-22 10:45:09,128 - kafka.conn         - INFO    - <BrokerConnection node_id=0 host=maggie-kafka-1.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.124', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,129 - kafka.client       - WARNING - Node 0 connection failed -- refreshing metadata
2023-06-22 10:45:09,184 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.126', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,184 - kafka.client       - WARNING - Node 2 connection failed -- refreshing metadata
2023-06-22 10:45:09,185 - kafka.conn         - INFO    - <BrokerConnection node_id=0 host=maggie-kafka-1.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.124', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,186 - kafka.client       - WARNING - Node 0 connection failed -- refreshing metadata
2023-06-22 10:45:09,186 - kafka.conn         - INFO    - <BrokerConnection node_id=1 host=maggie-kafka-2.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.125', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,186 - kafka.cluster      - INFO    - Group coordinator for event-v2-rrps-nude-detect is BrokerMetadata(nodeId='coordinator-0', host='maggie-kafka-1.somedomain.lab', port=9094, rack=None)
2023-06-22 10:45:09,186 - kafka.coordinator  - INFO    - Discovered coordinator coordinator-0 for group event-v2-rrps-nude-detect
2023-06-22 10:45:09,186 - kafka.client       - WARNING - Node 1 connection failed -- refreshing metadata
2023-06-22 10:45:09,187 - kafka.coordinator  - WARNING - Marking the coordinator dead (node coordinator-0) for group event-v2-rrps-nude-detect: Node Disconnected.
2023-06-22 10:45:09,189 - kafka.conn         - INFO    - <BrokerConnection node_id=coordinator-0 host=maggie-kafka-1.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.124', 9094)]>: connecting to maggie-kafka-1.somedomain.lab:9094 [('10.60.1.124', 9094) IPv4]
2023-06-22 10:45:09,190 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.126', 9094)]>: Closing connection. KafkaConnectionError: v
2023-06-22 10:45:09,190 - kafka.conn         - INFO    - <BrokerConnection node_id=coordinator-0 host=maggie-kafka-1.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.124', 9094)]>: Connection complete.
2023-06-22 10:45:09,191 - kafka.client       - WARNING - Node 2 connection failed -- refreshing metadata
2023-06-22 10:45:09,193 - kafka.cluster      - INFO    - Group coordinator for event-v2-rrps-nude-detect is BrokerMetadata(nodeId='coordinator-0', host='maggie-kafka-1.somedomain.lab', port=9094, rack=None)
2023-06-22 10:45:09,193 - kafka.coordinator  - INFO    - Discovered coordinator coordinator-0 for group event-v2-rrps-nude-detect
2023-06-22 10:45:09,193 - kafka.conn         - INFO    - <BrokerConnection node_id=0 host=maggie-kafka-1.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.124', 9094)]>: connecting to maggie-kafka-1.somedomain.lab:9094 [('10.60.1.124', 9094) IPv4]
2023-06-22 10:45:09,194 - kafka.conn         - INFO    - <BrokerConnection node_id=0 host=maggie-kafka-1.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.124', 9094)]>: Connection complete.
2023-06-22 10:45:09,195 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.126', 9094)]>: connecting to maggie-kafka-3.somedomain.lab:9094 [('10.60.1.126', 9094) IPv4]
2023-06-22 10:45:09,196 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.126', 9094)]>: Connection complete.
2023-06-22 10:45:09,201 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.126', 9094)]>: connecting to maggie-kafka-3.somedomain.lab:9094 [('10.60.1.126', 9094) IPv4]
2023-06-22 10:45:09,201 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.126', 9094)]>: Connection complete.
2023-06-22 10:45:09,211 - kafka.conn         - INFO    - <BrokerConnection node_id=0 host=maggie-kafka-1.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.124', 9094)]>: connecting to maggie-kafka-1.somedomain.lab:9094 [('10.60.1.124', 9094) IPv4]
2023-06-22 10:45:09,212 - kafka.conn         - INFO    - <BrokerConnection node_id=0 host=maggie-kafka-1.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.124', 9094)]>: Connection complete.
2023-06-22 10:45:09,215 - kafka.conn         - INFO    - <BrokerConnection node_id=0 host=maggie-kafka-1.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.124', 9094)]>: connecting to maggie-kafka-1.somedomain.lab:9094 [('10.60.1.124', 9094) IPv4]
2023-06-22 10:45:09,215 - kafka.conn         - INFO    - <BrokerConnection node_id=0 host=maggie-kafka-1.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.124', 9094)]>: Connection complete.
2023-06-22 10:45:09,243 - kafka.conn         - INFO    - <BrokerConnection node_id=coordinator-1 host=maggie-kafka-2.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.125', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,243 - kafka.client       - WARNING - Node coordinator-1 connection failed -- refreshing metadata
2023-06-22 10:45:09,243 - kafka.coordinator  - WARNING - Marking the coordinator dead (node coordinator-1) for group event-v2-ddtg-cms-detect: Node Disconnected.
2023-06-22 10:45:09,244 - kafka.conn         - INFO    - <BrokerConnection node_id=1 host=maggie-kafka-2.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.125', 9094)]>: connecting to maggie-kafka-2.somedomain.lab:9094 [('10.60.1.125', 9094) IPv4]
2023-06-22 10:45:09,244 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.126', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,245 - kafka.client       - WARNING - Node 2 connection failed -- refreshing metadata
2023-06-22 10:45:09,245 - kafka.conn         - INFO    - <BrokerConnection node_id=1 host=maggie-kafka-2.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.125', 9094)]>: Connection complete.
2023-06-22 10:45:09,260 - kafka.cluster      - INFO    - Group coordinator for event-v2-ttds-nude-detect is BrokerMetadata(nodeId='coordinator-2', host='maggie-kafka-3.somedomain.lab', port=9094, rack=None)
2023-06-22 10:45:09,261 - kafka.coordinator  - INFO    - Discovered coordinator coordinator-2 for group event-v2-ttds-nude-detect
2023-06-22 10:45:09,261 - kafka.coordinator  - WARNING - Marking the coordinator dead (node coordinator-2) for group event-v2-ttds-nude-detect: Node Disconnected.
2023-06-22 10:45:09,262 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.126', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,262 - kafka.client       - WARNING - Node 2 connection failed -- refreshing metadata
2023-06-22 10:45:09,264 - kafka.conn         - INFO    - <BrokerConnection node_id=coordinator-2 host=maggie-kafka-3.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.126', 9094)]>: connecting to maggie-kafka-3.somedomain.lab:9094 [('10.60.1.126', 9094) IPv4]
2023-06-22 10:45:09,265 - kafka.conn         - INFO    - <BrokerConnection node_id=coordinator-0 host=maggie-kafka-1.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.124', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,265 - kafka.conn         - INFO    - <BrokerConnection node_id=coordinator-2 host=maggie-kafka-3.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.126', 9094)]>: Connection complete.
2023-06-22 10:45:09,266 - kafka.client       - WARNING - Node coordinator-0 connection failed -- refreshing metadata
2023-06-22 10:45:09,266 - kafka.coordinator  - WARNING - Marking the coordinator dead (node coordinator-0) for group event-v2-rrps-fskbatch-detect: Node Disconnected.
2023-06-22 10:45:09,267 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.126', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,267 - kafka.client       - WARNING - Node 2 connection failed -- refreshing metadata
2023-06-22 10:45:09,268 - kafka.cluster      - INFO    - Group coordinator for event-v2-ttds-nude-detect is BrokerMetadata(nodeId='coordinator-2', host='maggie-kafka-3.somedomain.lab', port=9094, rack=None)
2023-06-22 10:45:09,268 - kafka.coordinator  - INFO    - Discovered coordinator coordinator-2 for group event-v2-ttds-nude-detect
2023-06-22 10:45:09,270 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.126', 9094)]>: connecting to maggie-kafka-3.somedomain.lab:9094 [('10.60.1.126', 9094) IPv4]
2023-06-22 10:45:09,271 - kafka.conn         - INFO    - <BrokerConnection node_id=0 host=maggie-kafka-1.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.124', 9094)]>: connecting to maggie-kafka-1.somedomain.lab:9094 [('10.60.1.124', 9094) IPv4]
2023-06-22 10:45:09,271 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.126', 9094)]>: Connection complete.
2023-06-22 10:45:09,272 - kafka.conn         - INFO    - <BrokerConnection node_id=0 host=maggie-kafka-1.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.124', 9094)]>: Connection complete.
2023-06-22 10:45:09,315 - kafka.conn         - INFO    - <BrokerConnection node_id=0 host=maggie-kafka-1.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.124', 9094)]>: connecting to maggie-kafka-1.somedomain.lab:9094 [('10.60.1.124', 9094) IPv4]
2023-06-22 10:45:09,315 - kafka.conn         - INFO    - <BrokerConnection node_id=0 host=maggie-kafka-1.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.124', 9094)]>: Connection complete.
2023-06-22 10:45:09,319 - kafka.conn         - INFO    - <BrokerConnection node_id=1 host=maggie-kafka-2.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.125', 9094)]>: connecting to maggie-kafka-2.somedomain.lab:9094 [('10.60.1.125', 9094) IPv4]
2023-06-22 10:45:09,319 - kafka.conn         - INFO    - <BrokerConnection node_id=1 host=maggie-kafka-2.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.125', 9094)]>: Connection complete.
2023-06-22 10:45:09,323 - kafka.conn         - INFO    - <BrokerConnection node_id=0 host=maggie-kafka-1.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.124', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,323 - kafka.client       - WARNING - Node 0 connection failed -- refreshing metadata
2023-06-22 10:45:09,350 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.126', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,350 - kafka.conn         - INFO    - <BrokerConnection node_id=0 host=maggie-kafka-1.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.124', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,350 - kafka.client       - WARNING - Node 2 connection failed -- refreshing metadata
2023-06-22 10:45:09,350 - kafka.client       - WARNING - Node 0 connection failed -- refreshing metadata
2023-06-22 10:45:09,351 - kafka.cluster      - INFO    - Group coordinator for event-v2-happ-nude-detect is BrokerMetadata(nodeId='coordinator-0', host='maggie-kafka-1.somedomain.lab', port=9094, rack=None)
2023-06-22 10:45:09,351 - kafka.coordinator  - INFO    - Discovered coordinator coordinator-0 for group event-v2-happ-nude-detect
2023-06-22 10:45:09,351 - kafka.coordinator  - WARNING - Marking the coordinator dead (node coordinator-0) for group event-v2-happ-nude-detect: Node Disconnected.
2023-06-22 10:45:09,353 - kafka.conn         - INFO    - <BrokerConnection node_id=coordinator-0 host=maggie-kafka-1.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.124', 9094)]>: connecting to maggie-kafka-1.somedomain.lab:9094 [('10.60.1.124', 9094) IPv4]
2023-06-22 10:45:09,354 - kafka.conn         - INFO    - <BrokerConnection node_id=coordinator-0 host=maggie-kafka-1.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.124', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,354 - kafka.conn         - INFO    - <BrokerConnection node_id=coordinator-0 host=maggie-kafka-1.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.124', 9094)]>: Connection complete.
2023-06-22 10:45:09,355 - kafka.client       - WARNING - Node coordinator-0 connection failed -- refreshing metadata
2023-06-22 10:45:09,355 - kafka.coordinator  - WARNING - Marking the coordinator dead (node coordinator-0) for group event-koko-nude-detect: Node Disconnected.
2023-06-22 10:45:09,357 - kafka.cluster      - INFO    - Group coordinator for event-v2-happ-nude-detect is BrokerMetadata(nodeId='coordinator-0', host='maggie-kafka-1.somedomain.lab', port=9094, rack=None)
2023-06-22 10:45:09,357 - kafka.coordinator  - INFO    - Discovered coordinator coordinator-0 for group event-v2-happ-nude-detect
2023-06-22 10:45:09,361 - kafka.conn         - INFO    - <BrokerConnection node_id=0 host=maggie-kafka-1.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.124', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,361 - kafka.client       - WARNING - Node 0 connection failed -- refreshing metadata
2023-06-22 10:45:09,375 - kafka.conn         - INFO    - <BrokerConnection node_id=2 host=maggie-kafka-3.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.126', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,375 - kafka.client       - WARNING - Node 2 connection failed -- refreshing metadata
2023-06-22 10:45:09,376 - kafka.cluster      - INFO    - Group coordinator for event-test-request is BrokerMetadata(nodeId='coordinator-1', host='maggie-kafka-2.somedomain.lab', port=9094, rack=None)
2023-06-22 10:45:09,376 - kafka.coordinator  - INFO    - Discovered coordinator coordinator-1 for group event-test-request
2023-06-22 10:45:09,376 - kafka.coordinator  - WARNING - Marking the coordinator dead (node coordinator-1) for group event-test-request: Node Disconnected.
2023-06-22 10:45:09,378 - kafka.conn         - INFO    - <BrokerConnection node_id=coordinator-1 host=maggie-kafka-2.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.125', 9094)]>: connecting to maggie-kafka-2.somedomain.lab:9094 [('10.60.1.125', 9094) IPv4]
2023-06-22 10:45:09,379 - kafka.conn         - INFO    - <BrokerConnection node_id=0 host=maggie-kafka-1.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.124', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,379 - kafka.conn         - INFO    - <BrokerConnection node_id=coordinator-1 host=maggie-kafka-2.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.125', 9094)]>: Connection complete.
2023-06-22 10:45:09,379 - kafka.client       - WARNING - Node 0 connection failed -- refreshing metadata
2023-06-22 10:45:09,382 - kafka.cluster      - INFO    - Group coordinator for event-test-request is BrokerMetadata(nodeId='coordinator-1', host='maggie-kafka-2.somedomain.lab', port=9094, rack=None)
2023-06-22 10:45:09,382 - kafka.coordinator  - INFO    - Discovered coordinator coordinator-1 for group event-test-request
2023-06-22 10:45:09,383 - kafka.conn         - INFO    - <BrokerConnection node_id=1 host=maggie-kafka-2.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.125', 9094)]>: connecting to maggie-kafka-2.somedomain.lab:9094 [('10.60.1.125', 9094) IPv4]
2023-06-22 10:45:09,383 - kafka.conn         - INFO    - <BrokerConnection node_id=1 host=maggie-kafka-2.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.125', 9094)]>: connecting to maggie-kafka-2.somedomain.lab:9094 [('10.60.1.125', 9094) IPv4]
2023-06-22 10:45:09,384 - kafka.conn         - INFO    - <BrokerConnection node_id=1 host=maggie-kafka-2.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.125', 9094)]>: Connection complete.
2023-06-22 10:45:09,384 - kafka.conn         - INFO    - <BrokerConnection node_id=1 host=maggie-kafka-2.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.125', 9094)]>: Connection complete.
2023-06-22 10:45:09,427 - kafka.conn         - INFO    - <BrokerConnection node_id=1 host=maggie-kafka-2.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.125', 9094)]>: connecting to maggie-kafka-2.somedomain.lab:9094 [('10.60.1.125', 9094) IPv4]
2023-06-22 10:45:09,428 - kafka.conn         - INFO    - <BrokerConnection node_id=1 host=maggie-kafka-2.somedomain.lab:9094 <connecting> [IPv4 ('10.60.1.125', 9094)]>: Connection complete.
2023-06-22 10:45:09,435 - kafka.conn         - INFO    - <BrokerConnection node_id=1 host=maggie-kafka-2.somedomain.lab:9094 <connected> [IPv4 ('10.60.1.125', 9094)]>: Closing connection. KafkaConnectionError: Socket EVENT_READ without in-flight-requests
2023-06-22 10:45:09,436 - kafka.client       - WARNING - Node 1 connection failed -- refreshing metadata
Cloopen-ReLiNK commented 5 months ago

have you solved this problem； same error
Socket EVENT_READ without in-flight-request