confluentinc / librdkafka

The Apache Kafka C/C++ library
Other
214 stars 3.14k forks source link

Kafka broker connection failure issues when doing k6 testing #4773

Open dev2021-ctrl opened 3 months ago

dev2021-ctrl commented 3 months ago

Description
Getting an error when producing messages at a high rate and sending 30k messages using k6 load testing scripts within span of 30 seconds to Kafka using Docker, and getting a broker error. Attached is the php script that is called at a high rate by k6 scripts

prod.txt

Please let me know if more debugging logs required. Facing this issue since 1 week.Please help at the earliest by providing some solution

Will share the docker files if needed.

Error

rdkafka#producer-2834| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Failed to connect to broker at kafka-3.7.kafka-producer-k6-test_default:9092:  
ll (after 12ms in state CONNECT)

Please provide the following information:

Basiclaly I am using docker and above are the versions

dev2021-ctrl commented 3 months ago

Any updates please very urgent facing this issue in prod too.

emasab commented 2 months ago

Do you always get a connection error or after a period where connections work? It seems like you're connecting to kafka:9092 as bootstrap but the advertised listener is kafka-3.7.kafka-producer-k6-test_default:9092. Make sure you can connect to that endpoint too.