Closed iercan closed 3 years ago
Hey, ieran. Currently, nats-kafka only exposes a limited set of configuration, documented here: https://github.com/nats-io/nats-kafka/blob/master/docs/config.md
However, I think we would be open to exposing more options.
We're using Shopify/sarama under the hood as the client library for Kafka.
Searching for batch.size
and linger.ms
in Sarama, I've found these two relevant issues:
Producer.Flush.Frequency
, as linger.ms
is not available@variadico We have found what causes the issue. Document states that nats queue should be supplied with natsqueue
keyword but we investigated code and realized that it should be supplied as queuename
keyword. As a result our load balancing does not work properly.
Hi. We are using NatsToKafka connector to relay data from nats to kafka. The subject we are trying to read is sending about 10k messages per second. In nats-kafka logs, we are seeing too many of the messages below:
I did some research and see it is suggested to tune
batch.size
andlinger.ms
for kafka producers but I couldn't figure out how to set these parameters on nats-kafka. I'd appreciate with any suggestion.