tulios / kafkajs

A modern Apache Kafka client for node.js
https://kafka.js.org
MIT License
3.75k stars 527 forks source link

Broker drops connection when sending messages with mixed acks flags with the same producer (0 and default -1) #1595

Open dgyimesi opened 1 year ago

dgyimesi commented 1 year ago

Describe the bug Broker drops connection when sending messages with mixed acks flags (0 and default -1) at volume.

To Reproduce

  1. Run a producer that continuously produces messages to a topic without acks being specified.
  2. With the same producer continuously produce messages to a different topic with acks set to 0.
  3. Relatively soon connections will get dropped with 'Connection closed' or 'EPIPE' errors.

Expected behavior Producer connection remains intact when sending messages with different acks flag.

Environment:

richard-pal commented 1 year ago

Reproduced with Kafkajs 2.2.4 at a request rate about 200 reqs/second.

The related KafkaJS error logs: {"level":"ERROR","timestamp":"2023-10-17T10:37:59.980Z","logger":"kafkajs","message":"[Producer] Failed to send messages: Closed connection","retryCount":0,"retryTime":815}

Using a separate producer instance for ack:-1 and ack: 0 looks to solve the problem.