Closed julianharty closed 6 years ago
max.in.flight.requests.per.connection=1
to prevent out-of-order messages (default is 5)
acks=all
(default is 1)
retries
>= 1 (default is 0)
replication.factor=3
(assuming you have a cluster of at least 3 Kafka nodes, else set it to the number of nodes.)
See various documents including the free to download whitepaper from Confluent https://www.confluent.io/white-paper/optimizing-your-apache-kafka-deployment/
Note: replication.factor
is not currently practical to set within the current functionality of pepper-box. In our testing we'll configure the server to create sufficient replicas instead by using default.replication.factor
see http://kafka.apache.org/documentation.html#prodconfig for more info.
Now I've changed the default settings to reduce the risk of missing or out-of-order messages it's time to close this issue since I'm not aware of any missing settings at this stage.
For our testing we want to minimise the risk of duplicates and out-of-order messages, etc. The defaults used by the Kafka client libraries don't suit our needs. This enhancement is intended to add the various parameters that'd help our testing.